Image coding method and device, electronic equipment and storage medium
By predicting and fusion of multi-coded templates on image blocks and selecting encoding results based on encoding loss values, the problem of poor image encoding performance in the prior art is solved, and the data volume and performance improvement are achieved.
Patent Information
- Application Number
- CN202310557091.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-22
AI Technical Summary
The existing image encoding methods lead to a large amount of data in the image encoding result and poor encoding performance.
By acquiring the image blocks, using multiple encoding templates to predict them, the fusion template weights are weighted and summed to predicted image blocks, and the fusion image blocks are generated, and the encoding result is determined based on the encoding loss value of the fusion image block. If the loss value is not high, For the prediction image block, a fused image block is used, otherwise a prediction image block with the lowest encoding loss value is used.
The amount of data of image encoding results is reduced and the image encoding performance is improved.
Smart Images

Figure CN119011847A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and more specifically, to an image encoding method, device, electronic device and storage medium. Background Art
[0002] In the field of computer vision processing technology, images are a common type of data. By encoding images, the amount of image data can be compressed, the storage space required to store images can be reduced, and the channel resources occupied by transmitting images can be reduced.
[0003] In the related art, an image can be divided into multiple image blocks. For any image block, the predicted image block of the image block can be determined by a set coding template, and then the image block is encoded according to the predicted image block corresponding to the image block to obtain the encoding result of the image block, and then the encoding results of each image block are summarized to obtain the image encoding result of the image.
[0004] However, the data volume of the encoding result of the image block determined by this method is still large, so that the data volume of the image encoding result of the image is large, resulting in poor encoding performance of the image. Summary of the invention
[0005] In view of this, the embodiments of the present application propose an image encoding method, device, electronic device and storage medium.
[0006] In a first aspect, an embodiment of the present application provides an image coding method, the method comprising: obtaining any image block of a plurality of image blocks included in an image as a target image block; predicting the target image block according to a plurality of coding templates respectively to obtain predicted image blocks corresponding to the plurality of coding templates; fusing the predicted image blocks corresponding to the plurality of target coding templates according to the template weights corresponding to the plurality of target coding templates to obtain a fused image block; the plurality of coding templates include a plurality of target coding templates; determining a coding loss value corresponding to the target predicted image block, the coding loss value of the target predicted image block being used to indicate the degree of difference between the target predicted image block and the target image block; the target predicted image block is a predicted image block or a fused image block; if the coding loss value of the fused image block is not higher than the coding loss values of the respective predicted image blocks, determining the coding result of the target image block according to the fused image block; if the coding loss value of the fused image block is higher than the coding loss value of one of the predicted image blocks, determining the coding result of the target image block according to the predicted image block with the lowest coding loss value; traversing the image, obtaining and summarizing the coding results of the plurality of image blocks respectively to obtain the image coding result of the image.
[0007] In a second aspect, an embodiment of the present application provides an image encoding device, the device comprising: an acquisition module, used to acquire any image block of a plurality of image blocks included in an image as a target image block; a prediction module, used to predict the target image block according to a plurality of encoding templates, respectively, to obtain the predicted image blocks corresponding to the plurality of encoding templates; a fusion module, used to fuse the predicted image blocks corresponding to the plurality of target encoding templates according to the template weights corresponding to the plurality of target encoding templates, to obtain the fused image block; the plurality of encoding templates include a plurality of target encoding templates; a determination module, used to determine the encoding loss value corresponding to the target predicted image block, the target predicted image block The coding loss value is used to indicate the degree of difference between the target predicted image block and the target image block; the target predicted image block is a predicted image block or a fused image block; a first coding module is used to determine the coding result of the target image block according to the fused image block if the coding loss value of the fused image block is not higher than the coding loss values of each predicted image block; a second coding module is used to determine the coding result of the target image block according to the predicted image block with the lowest coding loss value if the coding loss value of the fused image block is higher than the coding loss value of one of the predicted image blocks; an image coding module is used to traverse the image, obtain and summarize the respective coding results of multiple image blocks, and obtain the image coding result of the image.
[0008] Optionally, the fusion module is also used to perform weighted summation of the predicted image blocks corresponding to the multiple target coding templates according to the template weights corresponding to the multiple target coding templates to obtain an intermediate image block; and perform dimensionality reduction processing on the pixel values of each pixel point in the intermediate image block to obtain a fused image block.
[0009] Optionally, the template weight of the target coding template includes the weight corresponding to each original pixel in the target image block; the fusion module is also used to, for each original pixel, perform weighted summation on the pixel values of the target pixels of each target coding template according to the weights of the target coding templates corresponding to the original pixel points, to obtain the sum of the pixel values of the original pixel points, and the target pixel point of the target code coding template refers to the predicted pixel point corresponding to the original pixel point in the predicted image block corresponding to the target coding template; according to the summation of the pixel values of each original pixel point, the pixel value of each original pixel point is replaced to obtain an intermediate image block.
[0010] Optionally, the fusion module is also used to calculate the sum of the pixel value of each pixel point in the intermediate image block and the target pixel value as the intermediate pixel value of each pixel point in the intermediate image block; right-shift the intermediate pixel value of each pixel point in the intermediate image block by the target number of bits to obtain the right-shift processing result of each pixel point in the intermediate image block; replace the pixel value of each pixel point in the intermediate image block according to the right-shift processing result of each pixel point in the intermediate image block to obtain a fused image block.
[0011] Optionally, the coding template includes a prediction direction; the device also includes a weight acquisition module, which is used to determine the prediction accuracy of each original pixel in the target image block according to the prediction direction of the target coding template; and determine the template weight corresponding to the target coding template according to the size information of the target image block and the prediction accuracy of each original pixel in the target image block.
[0012] Optionally, the determination module is further used to determine the encoding loss value corresponding to the target prediction image block based on the target prediction image block and the target image block.
[0013] Optionally, the target predicted image block includes predicted pixel points corresponding to each original pixel point in the target image block; the determination module is also used to determine the loss value of each predicted pixel point based on the pixel value of each predicted pixel point and the pixel value of the original pixel point corresponding to each predicted pixel point; and determine the encoding loss value corresponding to the target predicted image block based on the loss value of each predicted pixel point in the predicted image block.
[0014] Optionally, the determination module is further used to determine the square of the difference between the pixel value of each predicted pixel point and the pixel value of the original pixel point corresponding to each predicted pixel point, as the loss value of each predicted pixel point.
[0015] Optionally, the coding template includes a template shape, a search direction and a prediction direction; the prediction module is further used to obtain, for each coding template, an area from the image that matches the template shape of the coding template, matches the size information of the target image block, and is adjacent to the target image block in the search direction of the coding template as a reconstructed block of the coding template; determine from the image an area that matches the respective reconstructed block of each target coding template as the respective matched reconstructed block of each coding template; for each coding template, obtain from the image an area that matches the size information of the target image block and is adjacent to the matched reconstructed block of the coding template in the prediction direction of the coding template as the predicted image block of the coding template.
[0016] Optionally, the image encoding module is further used to determine the encoding method of each image block according to the encoding result of each image block; obtain and summarize the encoding results and encoding methods of each of the multiple image blocks to obtain the image encoding result of the image.
[0017] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.
[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is executed by a processor.
[0019] In a fifth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above method.
[0020] An image encoding method, apparatus, electronic device and storage medium are provided in the embodiments of the present application. In the present application, a fused image block of a target image block is determined by using the template weights of respective target encoding templates and a predicted image block, and when the encoding loss value of the fused image block is not higher than the encoding loss values of each predicted image block, the encoding result of the target image block is determined according to the fused image block. The encoding result of the target image block determined according to the fused image block has a smaller amount of data, thereby reducing the amount of data of the image encoding result of the image and improving the image encoding performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0022] Figure 1 A schematic diagram showing an application scenario proposed in an embodiment of the present application;
[0023] Figure 2 A flowchart of an image encoding method proposed in one embodiment of the present application is shown;
[0024] Figure 3 A schematic diagram showing a reconstructed block corresponding to a vertical coding template in an embodiment of the present application is shown;
[0025] Figure 4 A schematic diagram showing a reconstructed block corresponding to a horizontal coding template in an embodiment of the present application is shown;
[0026] Figure 5 A schematic diagram showing a reconstructed block corresponding to an L-shaped coding template in an embodiment of the present application is shown;
[0027] Figure 6 A schematic diagram showing an encoding process corresponding to an L-shaped encoding template in an embodiment of the present application is shown;
[0028] Figure 7 A flowchart of an image encoding method proposed in another embodiment of the present application is shown;
[0029] Figure 8 A flowchart of an image encoding method proposed in another embodiment of the present application is shown;
[0030] Fig. 9 A schematic diagram showing an image block encoding process in an embodiment of the present application is shown;
[0031] Fig.10 A block diagram of an image encoding device proposed in one embodiment of the present application is shown;
[0032] Fig.11 A structural block diagram of an electronic device for executing an image encoding method according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. According to the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0034] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0036] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0037] It should also be noted that in the embodiments of the present application, the acquisition of information such as the user's job type, education level, age, life stage (for example, whether married), work location, and favorite products requires the user's permission or consent, and the collection, use, processing, and storage of information such as the user's job type, education level, age, life stage (for example, whether married), work location, and favorite products must comply with the regulations of the region where the user is located.
[0038] The present application discloses an image encoding method, device, electronic device and storage medium, and relates to cloud technology, etc.
[0039] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.
[0040] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.
[0041] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.
[0042] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.
[0043] The process of allocating physical storage space to a logical volume by the storage system. Specifically, the physical storage space is divided into stripes in advance according to the estimated capacity of the objects stored in the logical volume (this estimate is often relatively large compared to the actual capacity of the objects to be stored) and the group of independent redundant disk arrays (RAID). A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.
[0044] like Figure 1 As shown, the application scenario applicable to the embodiment of the present application includes a terminal 20 and a server 10, and the terminal 20 and the server 10 are connected through a wired network or a wireless network. The terminal 20 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart home appliance, a vehicle terminal, an aircraft, a wearable device terminal, a virtual reality device, and other terminal devices that can display pages, or run other applications that can call page display applications (such as instant messaging applications, shopping applications, search applications, game applications, forum applications, map traffic applications, etc.).
[0045] The server 10 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 10 may be used to provide services for applications running on the terminal 20.
[0046] The server 10 can obtain an image and divide the image into multiple image blocks of the same size according to a set size, and then encode each image block according to different encoding templates to obtain the encoding result of each image block, and record the encoding method of each image block. Then, the server 10 summarizes the encoding results and encoding methods of each image block as the image encoding result of the image. The set size can be set based on demand, such as 64×64.
[0047] After the server 10 obtains the image encoding result, it can also send the image encoding result to a terminal 20. The terminal 20 determines the encoding template of each image block according to the encoding method of each image block in the image encoding result, determines the decoding method according to the encoding template of each image block, decodes the encoding result of each image block according to the decoding method, and obtains the decoding result of each image block. The terminal 20 then summarizes the decoding results of each image block to obtain the image decoding result of the image. The image decoding result is an image corresponding to the image before encoding (the corresponding here can refer to an image that is completely the same or has a high degree of similarity).
[0048] After obtaining the image encoding result, the server 10 may store the image encoding result in the distributed cloud storage system, and the terminal 20 obtains the image encoding result to be decoded from the distributed cloud storage system and decodes it.
[0049] In one implementation, the server 10 may obtain a video stream to be processed, and determine each video frame in the video stream to be processed as an image, and then encode the image to obtain an image encoding result, and then summarize the image encoding results of each image to obtain a video encoding result. Afterwards, the server 10 may send the video encoding result to the terminal 20, and the terminal 20 obtains the image encoding result of each image from the video encoding result, obtains the image decoding result of each image according to the image encoding result of each image, and then summarizes the image decoding results of each image to obtain a video decoding result.
[0050] In another embodiment, the terminal 20 can obtain an image and divide the image into multiple image blocks of the same size according to a set size, and then encode each image block according to different encoding templates to obtain the encoding result of each image block, and record the encoding method of each image block. After that, the terminal 20 summarizes the encoding results and encoding methods of each image block as the image encoding result of the image.
[0051] After the terminal 20 obtains the image encoding result, it can also send the image encoding result to the server 10. The server 10 determines the encoding template of each image block according to the encoding method of each image block in the image encoding result, determines the decoding method according to the encoding template of each image block, decodes the encoding result of each image block according to the decoding method, obtains the decoding result of each image block, and then summarizes the decoding results of each image block to obtain the image decoding result of the image.
[0052] As another embodiment, the terminal 20 may include multiple terminals, among which terminal a and terminal b are connected through the server 10. Terminal a can encode the acquired image to obtain an image encoding result, and then send the image encoding result to the server 10. The server 10 forwards the image encoding result to another terminal b, and terminal b decodes the image encoding result to obtain an image decoding result, thereby realizing rapid transmission of images between multiple terminals.
[0053] For the convenience of description, in the following embodiments, image encoding is performed by an electronic device as an example for description.
[0054] See also Figure 2 , Figure 2 A flowchart of an image encoding method proposed in one embodiment of the present application is shown. The method can be applied to an electronic device, which can be Figure 1 The terminal 20 or the server 10 in the embodiment of the present invention comprises:
[0055] S110. Acquire any image block among a plurality of image blocks included in the image as a target image block.
[0056] In this embodiment, the image may refer to an image to be encoded, and the image may be a color image or a grayscale image. The image to be encoded may be an image obtained by the electronic device from a network, an image obtained by the electronic device from another electronic device, or an image taken by the electronic device through a camera.
[0057] As an implementation method, the electronic device can obtain a video stream and use each video frame in the video stream as an image to be encoded to obtain multiple images corresponding to the video stream, and encode the multiple images separately, thereby encoding the video stream.
[0058] The image can be divided according to the set size to obtain multiple image blocks of the same size. For any image block to be encoded at an encoding time, the image block to be encoded at the encoding time can be used as the target image block. For example, if the image size is 1920×1080 and the set size is 60×60, the image can be divided into 576 image blocks, and the image block to be encoded at time t1 is the 500th block, and the 500th image block is used as the target image block at time t1.
[0059] S120 , predicting the target image block according to the multiple coding templates respectively, to obtain predicted image blocks corresponding to the multiple coding templates respectively.
[0060] The coding template may be a coding template set based on demand, and the coding template may include an L-type coding template, a horizontal coding template, and a vertical coding template. For each coding template, the target image block is predicted according to the coding template to determine the area matching the target image block in the image as the predicted image block corresponding to the coding template, and the predicted image block has the same size as the target image block, for example, if the target image block is 60×60, the predicted image block is also 60×60.
[0061] For example, image B is divided into 100 image blocks: b1, b2, ..., b100. For one of the image blocks b20, image block b20 is predicted by a horizontal coding template, and region c1 matching image block b20 is determined in image B as a predicted image block corresponding to the horizontal coding template; image block b20 is predicted by a vertical coding template, and region c2 matching image block b20 is determined in image B as a predicted image block corresponding to the vertical coding template; image block b20 is predicted by an L-type coding template, and region c3 matching image block b20 is determined in image B as a predicted image block corresponding to the L-type coding template.
[0062] As an implementation manner, the coding template includes a template shape, a search direction, and a prediction direction; S120 may include: for each coding template, obtaining from the image an area that matches the template shape of the coding template, matches the size information of the target image block, and is adjacent to the target image block in the search direction of the coding template as a reconstructed block of the coding template; determining from the image an area that matches the respective reconstructed block of each target coding template as the respective matched reconstructed block of each coding template; for each coding template, obtaining from the image an area that matches the size information of the target image block and is adjacent to the matched reconstructed block of the coding template in the prediction direction of the coding template as the predicted image block of the coding template.
[0063] The coding template is used to determine the reconstructed block of the target image block in the image. The reconstructed block is determined based on the search direction, the template shape and the size of the target image block. The search direction refers to the direction of the reconstructed block relative to the target image block.
[0064] When the coding template is a vertical coding template, the coding shape of the vertical coding template can be a rectangle (the reconstructed block is a rectangle, and usually the shape of the vertical coding template is a rectangle with a width smaller than the height, that is, the width of the reconstructed image block is smaller than the height), the search direction is the left side of the target image block, and matching the size information of the target image block means that the height of the determined reconstructed block is the same as the height of the target image block, that is, the determined reconstructed block is a rectangular image block in the image adjacent to the left border of the target image block, and the height of the right side of the rectangular image block coincides with the height of the left side of the target image block. For example, Figure 3 As shown, the reconstructed block corresponding to the target image block 301 (unfilled area) is determined to be 302 (filled area) according to the vertical coding template.
[0065] When the coding template is a horizontal coding template, the coding shape of the horizontal coding template can be a rectangle (the reconstructed block is a rectangle, and usually the shape of the horizontal coding template is a rectangle with a width greater than a height, that is, the width of the reconstructed image block is greater than the height), the search direction is directly above the target image block, and matching the size information of the target image block means that the width of the determined reconstructed block is the same as the width of the target image block, that is, the determined reconstructed block is a rectangular image block in the image adjacent to the upper border of the target image block, and the width of the bottom of the rectangular image block coincides with the width of the top of the target image block. For example, Figure 4 As shown, the reconstructed block corresponding to the target image block 401 (unfilled area) is determined to be 402 (filled area) according to the horizontal coding template.
[0066] When the coding template is an L-shaped coding template, the coding shape of the L-shaped coding template can be L-shaped (the reconstructed block is L-shaped), the search direction is the upper left of the target image block, and matching the size information of the target image block means that the inner width of the determined reconstructed block is the same as the width of the target image block, and the inner height of the determined reconstructed block is the same as the height of the target image block, that is, the determined reconstructed block is an L-shaped image block adjacent to the left border and the upper border of the target image block in the image, and the inner width of the L-shaped image block coincides with the width above the target image block, and the inner height of the L-shaped image block coincides with the height on the left side of the target image block. For example, Figure 5 As shown, the reconstructed block corresponding to the target image block 501 (unfilled area) is determined to be 502 (filled area) according to the horizontal coding template.
[0067] The matching reconstructed block may refer to an area in the image that has the highest similarity to the reconstructed block and has the same size as the reconstructed block. A search box with the same size as the reconstructed block may be determined, and the search box may be slid in the image to determine the area within the search box after each slide as a candidate block (each candidate block has the same size as the reconstructed block), and the similarity between the reconstructed block and each candidate block may be determined based on the pixel value of each pixel in the reconstructed block and the pixel value of each pixel in each candidate block, and the candidate block with the greatest similarity may be determined as the matching reconstructed block.
[0068] As an implementation method, the pixel values of each pixel in the candidate block can be summed to obtain the summed result of the candidate block, and the pixel values of each pixel in the reconstructed block can be summed to obtain the summed result of the reconstructed block. The difference between the summed result of the candidate block and the summed result of the reconstructed block is calculated as the similarity between the reconstructed block and the candidate block.
[0069] In another embodiment, for each pixel in the reconstructed block, the pixel in the candidate block that matches the pixel is determined, and the difference between the pixel value of each pixel in the reconstructed block and the pixel value of the pixel in the candidate block that matches the pixel is calculated (which can also be a mean square error or an average error, etc.), as the similarity of each pixel in the reconstructed block, and the similarities of each pixel in the reconstructed block are summed to obtain the similarity between the reconstructed block and the candidate block. Among them, the pixel points with the same pixel coordinates in the candidate block and the reconstructed block are matched, and the pixel coordinates of the pixel point refer to the pixel coordinates of the pixel point in the image block to which it belongs. For example, the pixel coordinates of pixel x1 in the reconstructed block are (30,21), and the pixel coordinates of pixel x2 in the candidate block are (30,21), and it is determined that pixel x1 matches pixel x2.
[0070] The prediction direction refers to the direction of the determined predicted image block relative to the matching reconstructed block. The prediction direction is relative to the search direction. For example, in the horizontal coding template, the search direction is the left side of the target image block, and the prediction direction is the right side of the matching reconstructed block. In the vertical coding template, the search direction is the top of the target image block, and the prediction direction is the bottom of the matching reconstructed block. In the L-type coding template, the search direction is the top left of the target image block, and the prediction direction is the bottom right of the matching reconstructed block.
[0071] The predicted image block that matches the size information of the target image block means that the predicted image block has the same size as the target image block.
[0072] In the vertical coding template, being adjacent to the matching reconstructed block means that the height on the left side of the predicted image block coincides with the height on the right side of the matching reconstructed block, for example Figure 3 As shown, if 302 is a matching reconstructed block, the determined predicted image block is 301; in the horizontal coding template, being adjacent to the matching reconstructed block means that the width of the top of the predicted image block coincides with the width of the bottom of the matching reconstructed block, for example Figure 4 As shown, if 402 is a matching reconstructed block, the determined predicted image block is 401; in the L-type coding template, being adjacent to the matching reconstructed block means that the height of the left side of the predicted image block coincides with the height of the inner side of the matching reconstructed block and the width of the predicted image block coincides with the width of the inner side of the matching reconstructed block, for example Figure 5 As shown, if 502 is a matching reconstructed block, the determined predicted image block is 501.
[0073] For example, Figure 6 As shown, the coding template is an L-type coding template, the search direction of the L-type coding template is the upper left, and the prediction direction is the lower right. For the target image block 601, the upper left reconstructed block 602 is determined, and the matching reconstructed block 603 is determined in the image 60, and the lower right predicted image block 604 is determined according to the matching reconstructed block 503, and the target image block 601 and the predicted image block 604 have the same size.
[0074] In this embodiment, an image is divided into a plurality of image blocks, and the plurality of image blocks are arranged in sequence according to position information in the image. The image blocks are encoded according to the arrangement order of the image blocks. For each image block, when encoding the image block, a reconstructed block of the image block is searched in the image blocks that have been encoded.
[0075] As an implementation method, an image is divided into multiple image blocks, and the multiple image blocks are arranged in sequence according to position information in the image, and the image blocks are encoded according to the arrangement order of each image block. For each image block, when encoding the image block, a partial area is determined in the image block that has been encoded as a search area, and a reconstructed block of the image block is searched within the search area, wherein the size and dimension of the search area can be set based on demand and is not limited in this application.
[0076] S130 . According to the template weights corresponding to the multiple target coding templates, the predicted image blocks corresponding to the multiple target coding templates are fused to obtain a fused image block.
[0077] The target coding template may be a coding template among multiple coding templates, for example, the target coding template may be a horizontal coding template and a vertical coding template. The template weight of each target coding template may be determined, and the predicted image blocks corresponding to each of the multiple target coding templates may be weighted and summed by the template weights corresponding to each of the target coding templates to obtain a fused image block. The fused image block and the target image block also have the same size.
[0078] Specifically, the pixel value of each pixel in the predicted image block corresponding to the target coding template is multiplied by the template weight corresponding to the coding template to obtain the product result of the target coding template, and then the product results of each target coding template are summed to obtain a fused image block.
[0079] For example, the multiple target coding templates include a horizontal coding template and a vertical coding template, the template weight of the horizontal coding template is d1, and the template weight of the vertical coding template is d2. For the target image block c1, the predicted image block of the horizontal coding template is c2, and the predicted image block of the vertical coding template is c3. At this time, the fused image block is (c2×d1+c3×d2).
[0080] As another implementation, each pixel in the target image block can be used as an original pixel, the template weight of the target coding template includes the weight corresponding to each original pixel, and each pixel in the predicted image block corresponding to each coding template is used as a predicted pixel. For each original pixel, determine the predicted pixel corresponding to the original pixel in the predicted image blocks corresponding to different target templates as the matching pixel corresponding to the original pixel; for each target coding template, use the weight of the original pixel in the target coding template as the weight of the matching pixel of the original pixel under the target coding template; for each original pixel, perform weighted summation on the matching pixel corresponding to the original pixel by the weight of each matching pixel corresponding to the original pixel to obtain the summation result of the original pixel, and thus traverse each original pixel in the target image block to obtain the summation result of each original pixel, adjust the pixel value of each original pixel to the summation result of the pixel, and obtain a fused image block.
[0081] When the pixel coordinates of the original pixel and the predicted pixel are the same, the predicted pixel is determined as the matching pixel of the original pixel. For example, if the coordinates of the original pixel in the target image block are (20, 21), and the coordinates of the predicted pixel in the target predicted image block are also (20, 21), the predicted pixel is determined as the matching pixel of the original pixel.
[0082] Exemplarily, the matching pixel point e2 of the horizontal coding template corresponding to the original pixel point e1 is, the matching pixel point e3 of the vertical coding template corresponding to the original pixel point e1 is, the weight of the original pixel point e1 in the horizontal coding template is f1, and the weight of the original pixel point e1 in the vertical coding template is f2. f1 is used as the weight of the matching pixel point e2, and f2 is used as the weight of the matching pixel point e3. At this time, the sum of the original pixel point e1 is (e2×f1+e3×f2), and the pixel value of the original pixel point e1 is adjusted to (e2×f1+e3×f2). In this way, each original pixel point in the target image block is traversed to obtain a fused image block.
[0083] S140, determining a coding loss value corresponding to the target prediction image block.
[0084] The coding loss value of the target prediction image block is used to indicate the degree of difference between the target prediction image block and the target image block; the target prediction image block is a prediction image block or a fused image block.
[0085] As an implementation method, the pixel values of each pixel in the target prediction image block can be summed to obtain a first summation result, and the pixel values of each original pixel in the target image block can be summed to obtain a second summation result, and the difference between the first summation result and the second summation result can be calculated as the coding loss value of the target prediction image block.
[0086] As another implementation method, for each predicted pixel point in each target predicted image block, the original pixel point in the target image block corresponding to the predicted pixel point can be determined, and the difference between the pixel values of each predicted pixel point and the corresponding original pixel point is calculated as the predicted difference of the predicted pixel point. The predicted differences of each predicted pixel point of the target predicted image block are averaged, mean squared error is calculated, and average error is calculated, and the operation result of the target predicted image block is obtained as the coding loss value of the target predicted image block.
[0087] S150. If the coding loss value of the fused image block is not higher than the coding loss values of each predicted image block, determine the coding result of the target image block according to the fused image block; if the coding loss value of the fused image block is higher than the coding loss value of one of the predicted image blocks, determine the coding result of the target image block according to the predicted image block with the lowest coding loss value.
[0088] If the coding loss value corresponding to the fused image block is not higher than the coding loss values of each predicted image block, it is determined that the coding loss value corresponding to the fused image block is the smallest, indicating that the difference between the fused image block and the target image block is the smallest. The target image block is encoded by the fused image block to obtain the encoding result of the target image block.
[0089] If the coding loss value corresponding to the fused image block is higher than the coding loss value of one of the predicted image blocks, it is determined that the coding loss value corresponding to the fused image block is not the smallest, indicating that the difference between the fused image block and the target image block is not the smallest, and a predicted image with the lowest coding loss value is obtained. The difference between the predicted image block and the target image block is the smallest, and the target image block is encoded using the predicted image block with the lowest coding loss value to obtain the encoding result of the target image block.
[0090] Based on the above, it can be seen that when encoding the target image block, the image block with the lowest coding loss value is selected from the predicted image blocks and the fused image blocks corresponding to each target template to encode the target image block. The difference between the image block with the lowest coding loss value and the target image block is small, so that when encoding is performed according to the image block with the lowest coding loss value, the coding accuracy is higher, thereby improving the coding performance of the image block.
[0091] S160 , traverse the image, obtain and summarize the encoding results of the multiple image blocks, and obtain an image encoding result of the image.
[0092] For each image block, the corresponding encoding result is determined according to S110-S150. For any image block, the encoding result of the image block may be obtained based on a fused image block corresponding to the image block, or may be obtained based on a predicted image block corresponding to the image block.
[0093] After obtaining the encoding results of all the image blocks in the image, the encoding results of the image blocks in the image are aggregated to obtain the image encoding result of the image.
[0094] As an implementation manner, S160 may include: determining the encoding method of each image block according to the encoding result of each image block; acquiring and summarizing the encoding results and encoding methods of multiple image blocks to obtain the image encoding result of the image.
[0095] In the present application, the coding basis block for encoding the image block can be determined according to the coding result of the image block, and the coding method can be determined according to the obtained coding basis block. If the target image block is encoded according to the fused image block, the coding basis block is the fused image block, and if the target image block is encoded according to the predicted image block with the lowest coding loss value, the predicted image block with the lowest coding loss value is the coding basis block of the target image block.
[0096] The encoding method may refer to a method for determining an encoding basis block. For example, when a predicted image block determined by an L-type encoding template is used as the encoding basis block, the encoding method may be encoding by using an L-type encoding template. For another example, when a fused image block determined by a fusion of a horizontal encoding template and a vertical encoding template is used as the encoding basis block, the encoding method may be encoding by fusion of a horizontal encoding template and a vertical encoding template.
[0097] In this embodiment, a respective encoding mode identifier may be set for each encoding mode, and the encoding mode identifier indicates the encoding mode of the target image block. For example, the encoding modes include encoding by a horizontal encoding template, encoding by a vertical encoding template, encoding by an L-type encoding template, and encoding by a fusion of a horizontal encoding template and a vertical encoding template. The encoding mode identifier may be a binary number, and each encoding mode identifier refers to Table 1, which is as follows:
[0098] Table 1
[0099]
[0100] It can be understood that when the encoding method is referred to by the encoding method identifier, the encoding method identifier of each image block can be obtained; the encoding results of each of the multiple image blocks and the encoding method identifiers are obtained and summarized to obtain the image encoding result of the image.
[0101] After obtaining the image encoding result, the encoding method identifier of each image block in the image encoding result can be obtained, and the encoding template of each image block can be determined according to the encoding method identifier of each image block. The decoding method is determined according to the encoding template of each image block. Each image block is decoded according to the decoding method and encoding result of each image block to obtain the decoding result of each image block.
[0102] For example, for any image block g1, its corresponding encoding method is determined to be encoded using a vertical coding template, and the coding template is determined to be a vertical coding template. The decoding method corresponding to the vertical coding template is determined according to the vertical coding template. The encoding result of the image block g1 is decoded using the decoding method corresponding to the vertical coding template to obtain a decoding result of the image block g1.
[0103] In this embodiment, a fused image block of the target image block is determined by using the template weights of each target coding template and the predicted image block, and when the coding loss value of the fused image block is not higher than the coding loss values of each predicted image block, the coding result of the target image block is determined based on the fused image block. The coding result of the target image block determined based on the fused image block has a smaller amount of data, thereby reducing the amount of data of the image coding result of the image and improving the image coding performance.
[0104] See also Figure 7 , Figure 7 A flowchart of another image encoding method proposed in an embodiment of the present application is shown. The method can be applied to an electronic device, which can be Figure 1 The terminal 20 or the server 10 in the embodiment of the present invention comprises:
[0105] S210, obtaining any image block among a plurality of image blocks included in the image as a target image block; predicting the target image block according to a plurality of coding templates respectively, to obtain predicted image blocks corresponding to the plurality of coding templates respectively.
[0106] The description of S210 refers to the description of S110 to S120 above, which will not be repeated here.
[0107] S220 . Perform weighted summation on the predicted image blocks corresponding to the multiple target coding templates according to the template weights corresponding to the multiple target coding templates, to obtain an intermediate image block.
[0108] In summary, one possible implementation method is to multiply the pixel value of each pixel in the predicted image block corresponding to the target coding template by the template weight corresponding to the coding template to obtain the product result of the target coding template, and then sum the product results of each target coding template to obtain the intermediate image block.
[0109] As another possible implementation, each pixel in the target image block can be used as an original pixel, the template weight of the target coding template includes the weight corresponding to each original pixel, and each pixel in the target predicted image block is used as a predicted pixel. For each original pixel, according to the weight of each target coding template corresponding to the original pixel, the pixel values of the target pixels of each target coding template are weighted summed to obtain the sum of the pixel values of the original pixel. The target pixel of the target code coding template refers to the predicted pixel corresponding to the original pixel in the predicted image block corresponding to the target coding template; according to the sum of the pixel values of each original pixel, the pixel values of each original pixel are replaced to obtain an intermediate image block.
[0110] When the pixel coordinates of the original pixel point and the predicted pixel point are the same, the predicted pixel point is determined as the target pixel point of the original pixel point, and the pixel coordinates of the pixel point may refer to the pixel coordinates of the pixel point in the image block where the pixel point is located. For example, the coordinates of the original pixel point in the target image block are (20, 21), and the coordinates of the predicted pixel point in the target predicted image block are also (20, 21), and the predicted pixel point is determined as the target pixel point of the original pixel point.
[0111] The above process of determining the sum of the pixel values of the original pixels can be expressed as Formula 1, which is as follows:
[0112] h(i,j)=p 1 (i,j)w 1 (i,j)+p 2 (i,j)w 2 (i,j)
[0113] Among them, h(i,j) is the sum of the pixel values of the original pixel point with pixel coordinates (i,j) in the target image block, and w 1 (i, j) is the weight of the original pixel with pixel coordinates (i, j) in the horizontal encoding template, w 2 (i, j) is the weight of the original pixel with pixel coordinates (i, j) in the vertical coding template, p 1 (i, j) is the predicted pixel point with pixel coordinates (i, j) in the predicted image block corresponding to the horizontal coding template, p 2(i, j) is the predicted pixel point with pixel coordinates (i, j) in the predicted image block corresponding to the vertical coding template.
[0114] Exemplarily, the target pixel point of the horizontal coding template corresponding to the original pixel point e4 is e5, and the target pixel point of the vertical coding template corresponding to the original pixel point e4 is e6. The weight of the original pixel point e4 in the horizontal coding template is f3, and the weight of the original pixel point e4 in the vertical coding template is f4. f3 is used as the weight of the target pixel point e5, and f4 is used as the weight of the target pixel point e6. At this time, the sum of the original pixel point e4 is (e5×f3+e6×f4), and the pixel value of the original pixel point e3 is adjusted to (e5×f3+e6×f4). In this way, each original pixel point in the target image block is traversed to obtain the intermediate image block.
[0115] S230 , performing dimensionality reduction processing on the pixel value of each pixel point in the intermediate image block to obtain a fused image block.
[0116] After the intermediate image block is obtained, the pixel values of the pixels in the intermediate image block are subjected to dimensionality reduction processing to reduce the size of the pixel values of the pixels to avoid excessive pixel values of the pixels in the intermediate image block, thereby reducing the amount of data in the intermediate image block. The dimensionality reduction processing may include processing such as subtracting a fixed constant from the pixel value and normalization.
[0117] As an implementation method, the dimensionality reduction processing may include: calculating the sum of the pixel value of each pixel point in the intermediate image block and the target pixel value as the intermediate pixel value of each pixel point in the intermediate image block; right-shifting the intermediate pixel value of each pixel point in the intermediate image block by the target number of bits to obtain the right-shift processing result of each pixel point in the intermediate image block; replacing the pixel value of each pixel point in the intermediate image block according to the right-shift processing result of each pixel point in the intermediate image block to obtain a fused image block.
[0118] The target pixel value may be determined based on the requirements and the size of the target image block, for example, the target pixel value may be 32. The target bit number may be determined based on the target pixel value, for example, when the target pixel value is 32, the target bit number for right shift may be 6.
[0119] The above process of determining the right shift processing result of each pixel point in the intermediate image block can be expressed as Formula 2, which is as follows:
[0120] h 0 (i,j)=(h(i,j)+M)>>N (II)
[0121] Among them, h 0(i, j) is the right shift processing result corresponding to the pixel at pixel coordinates (i, j) in the intermediate image block, h(i, j) is the pixel value of the pixel at pixel coordinates (i, j) in the intermediate image block, M is the target pixel value, and N is the target number of bits. Since the intermediate image block is obtained by replacing the pixel values of the original pixels in the target image block according to the sum of the pixel values of the respective original pixels, the pixel values of the pixels in the intermediate image block are essentially the sum of the pixel values of the original pixels in the target image block.
[0122] After obtaining the right shift processing results of each pixel in the intermediate image block, according to the right shift processing results of each pixel in the intermediate image block, replace the pixel values of each pixel in the intermediate image block to obtain the fused image block.
[0123] Combined with the steps of S220 - S230, the process of determining the fused image block can be summarized as Equation Three, and Equation Three is as follows:
[0124] h 0 (i,j) = (p 1 (i,j)w 1 (i,j) + p 2 (i,j)w 2 (i,j) + M) >> N (Three)
[0125] Among them, since the fused image block is obtained by replacing the pixel values of each pixel in the intermediate image block according to the right shift processing results of the pixels in the intermediate image block, the pixel values of the pixels in the fused image block are essentially the right shift processing results of the pixels in the intermediate image block.
[0126] S240. Determine the coding loss value corresponding to the target prediction image block; if the coding loss value of the fused image block is not higher than the coding loss values of each prediction image block, determine the coding result of the target image block according to the fused image block; if the coding loss value of the fused image block is higher than the coding loss value of one of the prediction image blocks, determine the coding result of the target image block according to the prediction image block with the lowest coding loss value; traverse the image, obtain and summarize the coding results of each of the multiple image blocks to obtain the image coding result of the image.
[0127] Among them, the description of S240 refers to the description of S140 - S160 above, and will not be elaborated here.
[0128] In this embodiment, the predicted image blocks corresponding to the multiple target coding templates are weighted and summed up respectively through the template weights corresponding to the multiple target coding templates, and the intermediate image blocks after the weighted summation are subjected to dimensionality reduction processing, so that the fused image blocks after dimensionality reduction have a higher similarity with the target image blocks, thereby improving the accuracy of the fused image blocks, and further improving the accuracy of encoding the target image blocks according to the fused image blocks.
[0129] See also Figure 8 , Figure 8 A flowchart of an image encoding method proposed in another embodiment of the present application is shown. The method can be applied to an electronic device, which can be Figure 1 The terminal 20 or the server 10 in the embodiment of the present invention comprises:
[0130] S310, obtaining any image block among a plurality of image blocks included in the image as a target image block; predicting the target image block according to a plurality of coding templates respectively, to obtain predicted image blocks corresponding to the plurality of coding templates respectively.
[0131] The description of S310 refers to the description of S110 to S120 above, which will not be repeated here.
[0132] S320, determining the prediction accuracy of each original pixel in the target image block according to the prediction direction of the target coding template; determining the template weight corresponding to the target coding template according to the size information of the target image block and the prediction accuracy of each original pixel in the target image block.
[0133] Among them, the prediction direction of the coding template refers to the description of S120 above and is not repeated here. When the target coding template is a horizontal coding template, the predicted image block predicted by the target coding template has a higher accuracy in the part close to the matching reconstructed block determined by the target coding template (that is, the pixels near the width of the predicted image block that overlaps with the matching reconstructed block), and the accuracy of determining the pixels of this part is higher; and when the target coding template is a vertical coding template, the predicted image block predicted by the target coding template has a higher accuracy in the part close to the matching reconstructed block determined by the target coding template (that is, the pixels near the height of the predicted image block that overlaps with the matching reconstructed block), and the accuracy of determining the pixels of this part is higher.
[0134] Therefore, for the horizontal coding template, according to the prediction accuracy of each original pixel in the target image block, Formula 4 is determined as the weight template calculation formula, and the weight template is calculated by Formula 4 and the size information of the target image block. Formula 4 is as follows:
[0135]
[0136] Among them, w 1is the weight of the original pixel with pixel coordinates (i, j) in the template weight of the horizontal encoding template, W is the width of the target image block, and H is the height of the target image block.
[0137] It can be seen from Formula 4 that for pixels in the same column (a column with the same i), the larger the j value is (the larger the j value is, the farther the pixel is from the matching reconstructed block), the lower the weight is.
[0138] At the same time, for the vertical coding template, according to the prediction accuracy of each original pixel in the target image block, Formula 5 is determined as the weight template calculation formula, and the weight template is calculated by Formula 5 and the size information of the target image block. Formula 4 is as follows:
[0139]
[0140] It can be seen from Formula 5 that for pixels in the same row (a row with the same j), the larger the i value is (the larger the i value is, the farther the pixel is from the matching reconstructed block), the lower the weight is.
[0141] S330. Determine a coding loss value corresponding to the target prediction image block according to the target prediction image block and the target image block.
[0142] After the target prediction image block is obtained, the coding loss value corresponding to the target prediction image block is calculated according to the pixel value of each pixel point in the target prediction image block and the pixel value of each pixel point in the target image block.
[0143] Specifically, the target predicted image block includes predicted pixel points corresponding to each original pixel point in the target image block; the loss value of each predicted pixel point is determined according to the pixel value of each predicted pixel point and the pixel value of the original pixel point corresponding to each predicted pixel point; the encoding loss value corresponding to the target predicted image block is determined according to the loss value of each predicted pixel point in the predicted image block.
[0144] As described above, the target predicted image block (predicted image block or fused image block) has the same size as the target image block, the target predicted image block and the target image block include the same number of pixels, the pixels in the target image block are original pixels, the pixels in the target predicted image block are predicted pixels, the original pixels and the predicted pixels correspond one-to-one, wherein the original pixels and the predicted pixels with the same pixel coordinates correspond one-to-one.
[0145] For each predicted pixel in the target predicted image block, the loss value of the predicted pixel can be determined based on the pixel value of the predicted pixel and the pixel value of the original pixel corresponding to the predicted pixel, and then the loss values of each predicted pixel in the target predicted image block are summed to obtain the encoding loss value of the target predicted image block.
[0146] As an implementation method, the process of determining the loss value of each predicted pixel point includes: determining the square of the difference between the pixel value of each predicted pixel point and the pixel value of the original pixel point corresponding to each predicted pixel point, as the loss value of each predicted pixel point.
[0147] Based on the above description, the process of determining the encoding loss value of the target prediction image block can be simplified to Formula 6, which is as follows:
[0148]
[0149] Among them, loss is the encoding loss value, W is the width of the target prediction image block, H is the height of the target prediction image block, pred(x,y) is the pixel value of the predicted pixel with pixel coordinates (x,y), and orig(x,y) is the pixel value of the original pixel with pixel coordinates (x,y).
[0150] S340. If the coding loss value of the fused image block is not higher than the coding loss values of each predicted image block, determine the coding result of the target image block according to the fused image block; if the coding loss value of the fused image block is higher than the coding loss value of one of the predicted image blocks, determine the coding result of the target image block according to the predicted image block with the lowest coding loss value; traverse the image, obtain and summarize the respective coding results of multiple image blocks, and obtain the image coding result of the image.
[0151] The description of S340 refers to the description of S150-S160 above and will not be repeated here.
[0152] In this embodiment, the coding loss value is determined according to the pixel value of each original pixel in the target image block and the pixel value of each predicted pixel in the target predicted image block. The coding loss value can accurately characterize the gap between the target predicted image block and the target image block, so that when the target image block is encoded according to the image block with the lowest coding loss value determined according to the coding loss value, the coding effect of the target image block is better and the coding accuracy is higher.
[0153] At the same time, according to the prediction direction of the target coding template, the prediction accuracy of each original pixel in the target image block is determined, and then according to the prediction accuracy of each original pixel in the target image block, the template weight corresponding to the target coding template is determined. The accuracy of the template weight of the target coding template is also high, thereby improving the fusion effect and accuracy of the fused image block.
[0154] In order to explain the technical solution of the present application more clearly, the image encoding method of the present application is explained below in conjunction with an exemplary scenario. In this scenario, terminal o1 is connected to terminal o2 through a server communication, and terminal o1 sends a video stream to terminal o2 in real time through the server. Each video frame in the video stream of terminal o1 is regarded as an image, and each image is divided into 10×10 image blocks (including 10 rows of image blocks, and each row of image blocks includes 10 image blocks). Each image block is encoded in sequence from left to right and from top to bottom in a Z-shaped order.
[0155] The coding template includes an L-type coding template, a vertical coding template and a horizontal coding template, and the target coding template is a horizontal coding template and a vertical coding template. Among them, the coding mode corresponding to the L-type coding template is identified as 110, the coding mode corresponding to the vertical coding template is identified as 10, the coding mode corresponding to the horizontal coding template is identified as 1, and the coding mode of the horizontal coding target and the vertical coding template fusion is identified as 111.
[0156] like Fig. 9 As shown, for the 80th image block T80 ( Fig. 9 901 in the figure), the reconstructed block C80 corresponding to T80 is determined by using the L-type coding template ( Fig. 9 902 in ), and then determine the partial area 903 (the area composed of four dotted rectangular boxes) in the area composed of the first 79 image blocks as the search area, and determine the matching reconstructed image block PC80 ( Fig. 9 904 in the figure), and then determine the predicted image block YL80 (corresponding to the L-type coding template) according to the reconstructed image block PC80. Fig. 9 905 in ).
[0157] For image block T80, the predicted image block YS80 corresponding to the horizontal coding template and the predicted image block YC80 corresponding to the vertical coding template are determined respectively, and the template weight corresponding to the horizontal coding template (which can be determined according to Formula 4) and the template weight corresponding to the vertical coding template (which can be determined according to Formula 5) are obtained. Through the template weight corresponding to the horizontal coding template and the template weight corresponding to the vertical coding template, the pixel value of each pixel in the fused image block is determined according to Formula 3 to obtain the fused image block YR80.
[0158] The coding loss value is determined according to the predicted image block YL80 and the image block T80 in accordance with Formula 6, as the coding loss value of the predicted image block YL80; the coding loss value is determined according to the predicted image block YS80 and the image block T80 in accordance with Formula 6, as the coding loss value of the predicted image block YS80; the coding loss value is determined according to the predicted image block YC80 and the image block T80 in accordance with Formula 6, as the coding loss value of the predicted image block YC80; the coding loss value is determined according to the fused image block YR80 and the image block T80 in accordance with Formula 6, as the coding loss value of the fused image block YR80.
[0159] Afterwards, it is determined that the encoding loss value of the fused image block YR80 is the lowest, and the image block T80 is encoded by the fused image block YR80 to obtain the encoding result of the image block T80, and the encoding method of the fused image block is obtained as the encoding method of the fusion of the horizontal encoding template and the vertical encoding template, and the encoding method identifier is determined to be 111.
[0160] In this way, all 100 image blocks are traversed to obtain the encoding results and encoding method identifiers of the 100 image blocks respectively, and the encoding results and encoding method identifiers of the 100 image blocks respectively are summarized to obtain the image encoding result of the image.
[0161] See also Fig.10 , Fig.10 A block diagram of an image encoding device proposed in an embodiment of the present application is shown, and the device 1000 includes:
[0162] An acquisition module 1010 is used to acquire any image block among a plurality of image blocks included in the image as a target image block;
[0163] A prediction module 1020, configured to predict the target image block according to the plurality of coding templates respectively, and obtain the predicted image blocks corresponding to the plurality of coding templates respectively;
[0164] A fusion module 1030 is used to fuse the predicted image blocks corresponding to the multiple target coding templates according to the template weights corresponding to the multiple target coding templates to obtain a fused image block; the multiple coding templates include the multiple target coding templates;
[0165] A determination module 1040 is used to determine a coding loss value corresponding to a target prediction image block, where the coding loss value of the target prediction image block is used to indicate a degree of difference between the target prediction image block and the target image block; the target prediction image block is a prediction image block or a fused image block;
[0166] A first encoding module 1050, configured to determine an encoding result of a target image block according to the fused image block if the encoding loss value of the fused image block is not higher than the encoding loss values of each predicted image block;
[0167] A second encoding module 1060, configured to determine an encoding result of the target image block according to the predicted image block with the lowest encoding loss value if the encoding loss value of the fused image block is higher than the encoding loss value of one of the predicted image blocks;
[0168] The image encoding module 1070 is used to traverse the image, obtain and summarize the encoding results of each of the multiple image blocks, and obtain the image encoding result of the image.
[0169] Optionally, the fusion module 1030 is further used to perform weighted summation of the predicted image blocks corresponding to the multiple target coding templates according to the template weights corresponding to the multiple target coding templates to obtain an intermediate image block; and perform dimensionality reduction processing on the pixel values of each pixel point in the intermediate image block to obtain a fused image block.
[0170] Optionally, the template weight of the target coding template includes the weight corresponding to each original pixel in the target image block; the fusion module 1030 is also used to perform weighted summation on the pixel values of the target pixels of each target coding template for each original pixel according to the weights of the target coding templates corresponding to the original pixel, to obtain the sum of the pixel values of the original pixel, the target pixel of the target code coding template refers to the predicted pixel corresponding to the original pixel in the predicted image block corresponding to the target coding template; according to the sum of the pixel values of each original pixel, the pixel value of each original pixel is replaced to obtain an intermediate image block.
[0171] Optionally, the fusion module 1030 is further used to calculate the sum of the pixel value of each pixel point in the intermediate image block and the target pixel value as the intermediate pixel value of each pixel point in the intermediate image block; right-shift the intermediate pixel value of each pixel point in the intermediate image block by the target number of bits to obtain the right-shift processing result of each pixel point in the intermediate image block; and replace the pixel value of each pixel point in the intermediate image block according to the right-shift processing result of each pixel point in the intermediate image block to obtain a fused image block.
[0172] Optionally, the coding template includes a prediction direction; the device also includes a weight acquisition module, which is used to determine the prediction accuracy of each original pixel in the target image block according to the prediction direction of the target coding template; and determine the template weight corresponding to the target coding template according to the size information of the target image block and the prediction accuracy of each original pixel in the target image block.
[0173] Optionally, the determination module 1040 is further configured to determine a coding loss value corresponding to the target prediction image block according to the target prediction image block and the target image block.
[0174] Optionally, the target predicted image block includes predicted pixel points corresponding to each original pixel point in the target image block; the determination module 1040 is also used to determine the loss value of each predicted pixel point based on the pixel value of each predicted pixel point and the pixel value of the original pixel point corresponding to each predicted pixel point; and determine the encoding loss value corresponding to the target predicted image block based on the loss value of each predicted pixel point in the predicted image block.
[0175] Optionally, the determination module 1040 is further used to determine the square of the difference between the pixel value of each predicted pixel point and the pixel value of the original pixel point corresponding to each predicted pixel point, as the loss value of each predicted pixel point.
[0176] Optionally, the coding template includes a template shape, a search direction and a prediction direction; the prediction module 1020 is further used to obtain, for each coding template, an area from the image that matches the template shape of the coding template, matches the size information of the target image block, and is adjacent to the target image block in the search direction of the coding template as a reconstructed block of the coding template; determine from the image an area that matches the respective reconstructed block of each target coding template as the respective matched reconstructed block of each coding template; for each coding template, obtain from the image an area that matches the size information of the target image block and is adjacent to the matched reconstructed block of the coding template in the prediction direction of the coding template as the predicted image block of the coding template.
[0177] Optionally, the image encoding module 1070 is further used to determine the encoding method of each image block according to the encoding result of each image block; obtain and summarize the encoding results and encoding methods of each of the multiple image blocks to obtain the image encoding result of the image.
[0178] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.
[0179] Fig.11 A structural block diagram of an electronic device for executing an image encoding method according to an embodiment of the present application is shown. The electronic device may be Figure 1 The terminal 20 or the server 10, etc., need to be explained that, Fig.11 The computer system 1200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0180] like Fig.11As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM) 1203, such as executing the method in the above embodiment. In RAM 1203, various programs and data required for system operation are also stored. CPU 1201, ROM 1202 and RAM 1203 are connected to each other through bus 1204. Input / output (I / O) interface 1205 is also connected to bus 1204.
[0181] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read therefrom is installed into the storage section 1208 as needed.
[0182] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1209, and / or installed from a removable medium 1211. When the computer program is executed by a central processing unit (CPU) 1201, various functions defined in the system of the present application are executed.
[0183] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0184] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0185] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0186] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions, and when the computer-readable storage instructions are executed by a processor, the method in any of the above embodiments is implemented.
[0187] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method in any of the above embodiments.
[0188] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.
[0189] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable an electronic device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.
[0190] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. It should be understood that the present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
[0191] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for encoding an image, It is characterized in that The method comprises: Acquire any image block among a plurality of image blocks included in the image as a target image block; Predicting the target image block according to multiple coding templates respectively to obtain predicted image blocks corresponding to the multiple coding templates; According to the template weights corresponding to the multiple target coding templates, the predicted image blocks corresponding to the multiple target coding templates are fused to obtain a fused image block; the multiple coding templates include the multiple target coding templates; Determine a coding loss value corresponding to a target prediction image block, wherein the coding loss value of the target prediction image block is used to indicate a degree of difference between the target prediction image block and the target image block; the target prediction image block is the prediction image block or the fused image block; If the coding loss value of the fused image block is not higher than the coding loss values of each predicted image block, determining the coding result of the target image block according to the fused image block; If the coding loss value of the fused image block is higher than the coding loss value of one of the predicted image blocks, determining the coding result of the target image block according to the predicted image block with the lowest coding loss value; The image is traversed, and encoding results of each of the plurality of image blocks are acquired and summarized to obtain an image encoding result of the image.
2. The method according to claim 1, It is characterized in that The step of fusing the predicted image blocks corresponding to the plurality of target coding templates respectively according to the template weights corresponding to the plurality of target coding templates respectively to obtain the fused image block comprises: According to the template weights corresponding to the multiple target coding templates, weighted summation is performed on the predicted image blocks corresponding to the multiple target coding templates to obtain an intermediate image block; Dimensionality reduction processing is performed on the pixel values of each pixel point in the intermediate image block to obtain the fused image block.
3. The method according to claim 2, It is characterized in that The template weight of the target coding template includes the weight corresponding to each original pixel in the target image block; and performing weighted summation on the predicted image blocks corresponding to the multiple target coding templates according to the template weights corresponding to the multiple target coding templates to obtain the intermediate image block, including: For each of the original pixels, according to the weights of the target coding templates corresponding to the original pixels, weighted summation is performed on the pixel values of the target pixels of the target coding templates to obtain the pixel value summation result of the original pixels, wherein the target pixel of the target coding template refers to the predicted pixel corresponding to the original pixel in the predicted image block corresponding to the target coding template; According to the summation result of the pixel values of each of the original pixel points, the pixel value of each of the original pixel points is replaced to obtain the intermediate image block.
4. The method according to claim 2, It is characterized in that The step of performing dimensionality reduction processing on the pixel values of each pixel point in the intermediate image block to obtain the fused image block includes: Calculating the sum of the pixel value of each pixel point in the intermediate image block and the target pixel value as the intermediate pixel value of each pixel point in the intermediate image block; Right-shifting the intermediate pixel value of each pixel point in the intermediate image block by a target number of bits to obtain a right-shift processing result of each pixel point in the intermediate image block; According to the right-shift processing results of each pixel point in the intermediate image block, the pixel value of each pixel point in the intermediate image block is replaced to obtain the fused image block.
5. The method according to claim 1, It is characterized in that The coding template includes a prediction direction; and the method for acquiring the template weight of the target coding template includes: Determining the prediction accuracy of each original pixel in the target image block according to the prediction direction of the target coding template; The template weight corresponding to the target coding template is determined according to the size information of the target image block and the prediction accuracy of each original pixel in the target image block.
6. The method according to claim 1, It is characterized in that The determining the encoding loss value corresponding to the target predicted image block includes: According to the target prediction image block and the target image block, a coding loss value corresponding to the target prediction image block is determined.
7. The method according to claim 6, It is characterized in that The target predicted image block includes predicted pixel points corresponding to each original pixel point in the target image block; and determining the coding loss value corresponding to the target predicted image block according to the target predicted image block and the target image block includes: Determine a loss value of each predicted pixel point according to a pixel value of each predicted pixel point and a pixel value of an original pixel point corresponding to each predicted pixel point; According to the loss values of the respective predicted pixels in the predicted image block, the encoding loss value corresponding to the target predicted image block is determined.
8. The method according to claim 7, It is characterized in that The determining, according to the pixel value of each predicted pixel and the pixel value of the original pixel corresponding to each predicted pixel, the loss value of each predicted pixel comprises: Determine the square of the difference between the pixel value of each predicted pixel point and the pixel value of the original pixel point corresponding to each predicted pixel point as the loss value of each predicted pixel point.
9. The method according to claim 1, It is characterized in that The coding template includes a template shape, a search direction, and a prediction direction; the target image block is predicted according to a plurality of coding templates to obtain predicted image blocks corresponding to the plurality of coding templates, respectively, including: For each of the coding templates, obtaining from the image a region that matches the template shape of the coding template, matches the size information of the target image block, and is adjacent to the target image block in the search direction of the coding template as a reconstructed block of the coding template; Determine from the image an area matching the reconstructed block of each target coding template as a matching reconstructed block of each coding template; For each of the coding templates, an area matching the size information of the target image block and adjacent to the matching reconstructed block of the coding template in the prediction direction of the coding template is obtained from the image as the predicted image block of the coding template.
10. The method according to claim 1, It is characterized in that The acquiring and summarizing the encoding results of the respective image blocks to obtain the image encoding result of the image includes: Determining an encoding method for each of the image blocks according to an encoding result of each of the image blocks; The encoding results and encoding methods of each of the multiple image blocks are obtained and summarized to obtain an image encoding result of the image.
11. A method for encoding an image, It is characterized in that The method comprises: An acquisition module, used for acquiring any image block among a plurality of image blocks included in the image as a target image block; A prediction module, used to predict the target image block according to multiple coding templates respectively, to obtain predicted image blocks corresponding to the multiple coding templates respectively; a fusion module, configured to fuse the predicted image blocks corresponding to the multiple target coding templates according to the template weights corresponding to the multiple target coding templates, to obtain a fused image block; the multiple coding templates include the multiple target coding templates; a determination module, configured to determine a coding loss value corresponding to a target prediction image block, wherein the coding loss value of the target prediction image block is used to indicate a degree of difference between the target prediction image block and the target image block; the target prediction image block is the prediction image block or the fused image block; A first encoding module, configured to determine an encoding result of the target image block according to the fused image block if the encoding loss value of the fused image block is not higher than the encoding loss values of each predicted image block; A second encoding module, configured to determine an encoding result of the target image block according to the predicted image block with the lowest encoding loss value if the encoding loss value of the fused image block is higher than the encoding loss value of one of the predicted image blocks; The image encoding module is used to traverse the image, obtain and summarize the encoding results of each of the multiple image blocks, and obtain the image encoding result of the image.
12. An electronic device, It is characterized in that include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to execute the method according to any one of claims 1-10.
13. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 10.
Citation Information
Cited By
Working condition identification method, device and equipment of rotary kiln and medium
CN119274003A
Rotary kiln working condition identification method, device, equipment and medium
CN119274003B
