Image processing method, apparatus and system, and related device

Abstract

An image processing method, apparatus and system, and a related device, which relate to the technical field of data processing. The method comprises: acquiring multiple images, and grouping the multiple images on the basis of pixel features of the multiple images, so as to obtain multiple groups, wherein the similarity between different images in the same group is higher than the similarity between images in different groups; and using a first pixel library corresponding to a first group among the multiple groups to encode images in the first group so as to obtain image encoding results corresponding to the first group, wherein the first pixel library comprises pixels from some images in the first group, and the image encoding results corresponding to the first group is used for restoring images in the first group on the basis of the first pixel library. Thus, by using the similarity between different images in the same group, the data volume of images in each group can be fully compressed, such that the volume of encoded image data is far less than the volume of the image data before being encoded, thereby reducing the storage space required for storing multiple images.

Description

Image processing methods, apparatus, systems and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202410910756.3, filed on July 8, 2024, entitled "Image Processing Method, Apparatus, System and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology, and in particular to an image processing method, apparatus, system and related equipment. Background Technology

[0003] Images have become important data in applications such as the Internet, scientific research, and industrial production.

[0004] As image resolution increases, the amount of image data also increases. Therefore, images are encoded into image data of a specified format, such as using entropy coding techniques. The encoded image data is usually smaller than the original image data, thus reducing the storage space required for image storage. The process of encoding an image is essentially representing as much image information as possible with as little data as possible.

[0005] In practical applications, the reduction rate of image data after encoding is not high, such as only 2% to 10%, which makes the data volume of the encoded image still large, and thus image storage still occupies a large amount of storage space. Summary of the Invention

[0006] This application provides an image processing method to reduce the storage space required for image storage. In addition, this application also provides an image processing apparatus, an audio processing method, an audio processing apparatus, a computing device, a computer-readable storage medium, and a computer program product.

[0007] Firstly, this application provides an image processing method. Specifically, it involves acquiring multiple images to be processed, which may be images of the same or different formats, and which may be sent by a user-side client. Then, based on the pixel features of the acquired images, the images are grouped into multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups; that is, similar images are grouped into the same group, and dissimilar images are grouped into different groups. Finally, using a first pixel library corresponding to the first group, the images within the first group are encoded to obtain an image encoding result corresponding to the first group. The first group is one of the multiple groups obtained, and the first pixel library includes pixels from some images within the first group. The image encoding result corresponding to the first group is used to reconstruct the images within the first group using the first pixel library. For example, when a subsequent application requests images stored in the first group, the images within the first group can be reconstructed using the image encoding result corresponding to the first group and the first pixel library. Similarly, for other groups besides the first group, encoding results corresponding to each group can be generated in the same manner.

[0008] Thus, because multiple images within the same group have a high degree of similarity, encoding multiple images within that group based on pixels from a subset of those images can fully leverage this similarity to improve the encoding performance. In other words, by utilizing image similarity, the data size of each group can be significantly compressed, resulting in a much smaller amount of encoded image data compared to the original. Consequently, when storing multiple images, only the smaller encoded data needs to be stored, thereby reducing the storage space required for storing multiple images and improving the image reduction rate. The reduction rate is the ratio of the reduced data size to the original data size.

[0009] In one possible implementation, if the multiple images to be processed include images of different formats, the method further includes converting the multiple images into a standard format to obtain multiple images in a standard format. This standard format could be, for example, a pre-configured HUIF format or another format. Furthermore, when grouping the multiple images based on their pixel features, the grouping can specifically be based on the pixel features of the standard format images. Thus, through format conversion, multiple images of different formats can be converted into images of a unified format, facilitating subsequent encoding processing of these images and improving encoding performance (eliminating the need to execute different encoding logic for images of different formats).

[0010] In one possible implementation, the pixel library corresponding to the first group further includes reference metadata, which refers to the metadata used for reference during the encoding of image metadata. The method then further includes: using the reference metadata to encode the metadata of the standard-format images within the first group, obtaining the metadata encoding result corresponding to the first group. This metadata encoding result is used to reconstruct the metadata of the standard-format images within the first group from the reference metadata. Thus, using the reference metadata in the pixel library to encode the metadata of the images within the group can reduce the amount of metadata, thereby reducing the storage space required for subsequent storage of this metadata.

[0011] In one possible implementation, when encoding the images within the first group using the first pixel library corresponding to the first group, specifically, the pixels in the images within the first group are compared with the pixels in the first pixel library to obtain pixel difference information. This pixel difference information describes the differences between the pixels in the images within the first group and the pixels in the first pixel library. Encoding this pixel difference information yields the image encoding result corresponding to the first group, which includes the encoding results of each image within the first group. Since the pixels in the images within the first group are similar to the pixels in the first pixel library, the difference information obtained by comparing the pixels in the images with those in the pixel library usually exhibits a certain degree of similarity and regularity. Therefore, after encoding the difference information with regularity and similarity, the data size of the resulting encoding result is effectively smaller than the number of difference information points and also smaller than the data size of the images within the first group. This allows for effective compression of the image data, reducing the storage space required for subsequent image storage.

[0012] In one possible implementation, the first pixel library includes pixels from multiple reference pixel blocks, where each reference pixel block is a pixel block in an image, and the pixel difference information is a residual. When comparing pixels in the image within the first group with pixels in the first pixel library, the image within the first group can be first divided into multiple original pixel blocks. Based on the first pixel library, reference pixel blocks that satisfy matching conditions with each of the original pixel blocks are determined. These matching conditions include a pixel similarity greater than a threshold between the two pixel blocks. The residual between the original pixel blocks satisfying the matching conditions and the reference pixel blocks is then calculated, yielding the pixel difference information. Thus, by encoding the image within the first group at the pixel block granularity, subsequent reconstruction of individual pixel blocks in the image only requires the reference pixel blocks corresponding to that pixel library, eliminating the need to read all reference pixel blocks in the pixel library (and the reference pixel blocks in the pixel libraries of other groups), thereby avoiding read magnification issues.

[0013] In one possible implementation, each of the plurality of reference pixel blocks is configured with a priority, which indicates the degree to which a reference pixel block is used to determine whether it satisfies the matching condition with the original pixel block. Therefore, when determining the reference pixel block that satisfies the matching condition with each of the plurality of original pixel blocks, specifically, the reference pixel block that satisfies the matching condition with each of the plurality of original pixel blocks can be determined from a first pixel library based on the priority of each reference pixel block. In this way, for each original pixel block, higher-priority reference pixel blocks can be selected for priority matching. This allows for rapid traversal of reference pixel blocks that satisfy the matching condition with the original pixel block, eliminating the need to traverse all reference pixel blocks in the pixel library and perform calculations for that original pixel block. This reduces the resource consumption required to determine the reference pixel block and improves the efficiency of determining the reference pixel block.

[0014] In one possible implementation, the plurality of original pixel blocks includes a first original pixel block. When determining reference pixel blocks that satisfy matching conditions with each of the plurality of original pixel blocks according to the first pixel library, specifically, it can be done by first checking if a reference pixel block exists in the first pixel library that satisfies the matching conditions with the first original pixel block. If no reference pixel block exists in the first pixel library, the first original pixel block is added to the first pixel library as a reference pixel block. Thus, during the encoding of the image within the first group, by adding a reference pixel block to the pixel library, other original pixel blocks within the first group can be effectively compressed using this newly added reference pixel block, thereby improving the encoding effect for the image within the first group.

[0015] In one possible implementation, when encoding pixel difference information, the data features of the pixel difference information can be extracted first. Based on these features, at least one encoding algorithm matching the data features is determined from a variety of encoding algorithms. Then, the encoding operation for the pixel difference information is executed sequentially using at least one encoding algorithm. In this way, selecting suitable multiple encoding algorithms to encode the pixel difference information ensures the encoding effect for that pixel difference information, meaning that the amount of data from the encoding results for multiple images can be minimized, thereby effectively reducing the storage space required to store the multiple images.

[0016] Secondly, this application provides an image processing apparatus, comprising: an acquisition module for acquiring multiple images to be processed; a grouping module for grouping the multiple images into multiple groups based on pixel features, wherein the similarity between different images within the same group is higher than the similarity between images in different groups, and the multiple groups include a first group; and an encoding module for encoding the images within the first group using a first pixel library corresponding to the first group, thereby obtaining an image encoding result corresponding to the first group, wherein the first pixel library includes pixels from some images within the first group, and the image encoding result corresponding to the first group is used to recover the images within the first group from the first pixel library.

[0017] In one possible implementation, the multiple images to be processed include images of different formats, and the apparatus further includes: a conversion module for converting the multiple images to a standard format to obtain multiple images of a standard format; and a grouping module specifically used to group the multiple images of a standard format according to the pixel features of the multiple images of the standard format.

[0018] In one possible implementation, the pixel library corresponding to the first group further includes reference metadata; the encoding module is further configured to use the reference metadata to encode the metadata of the standard format image within the first group to obtain the metadata encoding result corresponding to the first group, and the metadata encoding result corresponding to the first group is used to recover the metadata of the standard format image within the first group with the reference metadata.

[0019] In one possible implementation, the encoding module is configured to: compare pixels in the image within the first group with pixels in the first pixel library to obtain pixel difference information, wherein the pixel difference information is used to describe the difference between pixels in the image within the first group and pixels in the first pixel library; and encode the pixel difference information to obtain the image encoding result corresponding to the first group.

[0020] In one possible implementation, the first pixel library includes pixels from multiple reference pixel blocks, where each reference pixel block is a pixel block in an image, and the pixel difference information is a residual. The encoding module is configured to: divide the image within the first group to obtain multiple original pixel blocks; determine, according to the first pixel library, reference pixel blocks that satisfy matching conditions with each of the multiple original pixel blocks, wherein the matching conditions include a pixel similarity between the two pixel blocks being greater than a threshold; and calculate the residual between the original pixel blocks that satisfy the matching conditions and the reference pixel blocks.

[0021] In one possible implementation, each of the plurality of reference pixel blocks is configured with a priority, the priority being used to indicate the degree to which a reference pixel block is used to determine whether it satisfies the matching condition with the original pixel block; the encoding module is used to determine, according to the priority of each reference pixel block, reference pixel blocks from the first pixel library that satisfy the matching condition with each of the plurality of original pixel blocks respectively.

[0022] In one possible implementation, the plurality of original pixel blocks include a first original pixel block; the encoding module is configured to: search whether there is a reference pixel block in the first pixel library that meets the matching conditions with the first original pixel block; when there is no reference pixel block in the first pixel library that meets the matching conditions with the first original pixel block, add the first original pixel block as a reference pixel block to the first pixel library.

[0023] In one possible implementation, the encoding module is configured to: extract data features of pixel difference information; determine at least one encoding algorithm that matches the data features from a variety of encoding algorithms based on the data features; and sequentially perform encoding operations on the pixel difference information using at least one encoding algorithm.

[0024] The image processing apparatus provided in the second aspect corresponds to the image processing method provided in the first aspect. Therefore, the technical effects of any implementation in the second aspect can be found in the relevant descriptions of the technical effects of the corresponding implementation in the first aspect, and will not be repeated here.

[0025] Thirdly, this application provides an image processing method, comprising: acquiring a first image to be processed; determining a first pixel library from multiple pixel libraries, wherein pixels in the first pixel library and pixels in the first image satisfy a similarity condition, and pixels in different pixel libraries do not satisfy the similarity condition; comparing the first image with at least one reference pixel set in the first pixel library to obtain difference information, the difference information being used to describe pixel differences between the first image and the at least one reference pixel set; encoding the difference information to obtain an encoding result corresponding to the first image, the encoding result being used to reconstruct the first image with the at least one reference pixel set.

[0026] Since the pixels in the first image and the pixels in the reference pixel set in the first pixel library satisfy similarity conditions (i.e., the pixels in the first image and the pixels in one or more reference pixel sets in the first pixel library have a certain similarity), the difference information obtained by comparing the first image and the reference pixel set based on this similarity usually also has certain similarities and patterns. Therefore, after encoding the difference information with patterns and similarities, the amount of data in the encoded result will be effectively smaller than the number of difference information items and also effectively smaller than the amount of data in the first image. Thus, when storing the first image subsequently, only the smaller encoded result needs to be stored, thereby reducing the storage space required for image storage. Simultaneously, since the encoded result and the reference pixel set can reconstruct the first image, storing the smaller encoded result is equivalent to storing the first image. Furthermore, when the difference information is encoded using a lossless encoding method, the complete first image can usually be reconstructed based on the encoded result and the reference pixel set, thereby reducing the storage space required for image storage while achieving lossless storage of the first image.

[0027] In one possible implementation, the pixels in the reference pixel set belong to at least one image, and the difference information is used to describe the pixel differences between the first image and an image in the first pixel library; or, the reference pixel set includes pixels in a reference pixel block, the reference pixel block being a pixel block in an image, and the difference information is used to describe the pixel differences between each pixel block in the first image and a reference pixel block in the first pixel library.

[0028] In one possible implementation, the reference pixel set includes pixels in reference pixel blocks, and the difference information is a residual; the step of comparing the first image with at least one reference pixel set in the first pixel library to obtain the difference information includes: dividing the first image to obtain multiple original pixel blocks; determining reference pixel blocks from the first pixel library that satisfy matching conditions with each of the multiple original pixel blocks, the matching conditions including a similarity between pixels in the two pixel blocks being greater than a threshold; and calculating the residual between pixels in the original pixel blocks that satisfy the matching conditions and pixels in the reference pixel blocks.

[0029] In one possible implementation, each reference pixel block in the first pixel library is configured with a priority, the priority being used to indicate the degree of priority of the reference pixel block in determining whether it meets the matching condition with the original pixel block; the step of determining the reference pixel block that meets the matching condition with each of the plurality of original pixel blocks from the first pixel library includes: determining the reference pixel block that meets the matching condition with each of the plurality of original pixel blocks from the first pixel library according to the priority of each reference pixel block.

[0030] In one possible implementation, determining the first pixel library from a plurality of pixel libraries includes: extracting pixel features of the first image; and determining the first pixel library from the plurality of pixel libraries based on the pixel features of the first image, wherein the reference pixel features of the first pixel library have the greatest similarity to the pixel features of the first image among the plurality of pixel libraries.

[0031] In one possible implementation, encoding the difference information includes: extracting data features of the difference information; determining at least one encoding algorithm that matches the data features from a variety of encoding algorithms based on the data features; and encoding the difference information using the at least one encoding algorithm.

[0032] In one possible implementation, determining the first pixel library from a plurality of pixel libraries includes: converting the format of the first image to obtain a first image in a standard format; and determining the first pixel library from the plurality of pixel libraries for the first image in the standard format.

[0033] In one possible implementation, the difference information includes metadata difference information and pixel difference information, and the first pixel library further includes reference metadata; the step of comparing the standard format image with at least one reference pixel set in the first pixel library includes: comparing the metadata of the standard format image with the reference metadata in the first pixel library to obtain the metadata difference information; and comparing the standard format image with at least one reference pixel set in the first pixel library to obtain the pixel difference information.

[0034] Fourthly, this application provides an image processing apparatus, characterized in that the image processing apparatus comprises: an acquisition module for acquiring a first image to be processed; a determination module for determining a first pixel library from a plurality of pixel libraries, wherein pixels in the first pixel library and pixels in the first image satisfy a similarity condition, and pixels in different pixel libraries do not satisfy the similarity condition; a comparison module for comparing the first image with at least one reference pixel set in the first pixel library to obtain difference information, wherein the difference information is used to describe pixel differences between the first image and the at least one reference pixel set; and an encoding module for encoding the difference information to obtain an encoding result corresponding to the first image, wherein the encoding result is used to recover the first image from the at least one reference pixel set.

[0035] In one possible implementation, the pixels in the reference pixel set belong to at least one image, and the difference information is used to describe the pixel differences between the first image and an image in the first pixel library; or, the reference pixel set includes pixels in a reference pixel block, the reference pixel block being a pixel block in an image, and the difference information is used to describe the pixel differences between each pixel block in the first image and a reference pixel block in the first pixel library.

[0036] In one possible implementation, the reference pixel set includes pixels in reference pixel blocks, and the difference information is a residual; the comparison module is configured to: divide the first image to obtain a plurality of original pixel blocks; determine from the first pixel library a reference pixel block that satisfies a matching condition with each of the plurality of original pixel blocks, the matching condition including that the similarity between pixels in the two pixel blocks is greater than a threshold; and calculate the residual between pixels in the original pixel block that satisfies the matching condition and pixels in the reference pixel block.

[0037] In one possible implementation, each reference pixel block in the first pixel library is configured with a priority, the priority being used to indicate the degree of priority of the reference pixel block in determining whether it meets the matching condition with the original pixel block; the determining module is used to determine, according to the priority of each reference pixel block, a reference pixel block from the first pixel library that meets the matching condition with each of the plurality of original pixel blocks.

[0038] In one possible implementation, the determining module is configured to: extract pixel features of the first image; and determine a first pixel library from the plurality of pixel libraries based on the pixel features of the first image, wherein the reference pixel features of the first pixel library have the greatest similarity to the pixel features of the first image among the plurality of pixel libraries.

[0039] In one possible implementation, the encoding module is configured to: extract data features of the difference information; determine at least one encoding algorithm that matches the data features from a variety of encoding algorithms based on the data features; and encode the difference information using the at least one encoding algorithm.

[0040] In one possible implementation, the determining module is configured to: convert the format of the first image to obtain a first image in a standard format; and, for the first image in the standard format, determine the first pixel library from the plurality of pixel libraries.

[0041] In one possible implementation, the difference information includes metadata difference information and pixel difference information, and the first pixel library further includes reference metadata; the comparison module is configured to: compare the metadata of the first image in the standard format with the reference metadata in the first pixel library to obtain the metadata difference information; and compare the first image in the standard format with at least one set of reference pixels in the first pixel library to obtain the pixel difference information.

[0042] Fifthly, this application also provides an audio processing method, characterized in that the method includes: acquiring audio to be processed; dividing the audio to obtain multiple original audio segments; comparing each of the multiple original audio segments with at least one reference audio segment in an audio library to obtain difference information corresponding to the multiple original audio segments, wherein the at least one reference audio segment in the audio library includes a portion of the multiple original audio segments, and the difference information is used to describe the audio difference between the audio and the at least one reference audio segment; encoding the difference information to obtain an encoding result corresponding to the audio, wherein the encoding result is used to recover the audio with the at least one reference audio segment.

[0043] Because the reference audio segments in the audio library include some of the original audio segments in the audio, there is a certain similarity between the original audio segments and the reference audio segments in the audio library. This similarity means that the difference information obtained by comparing the original audio segments and the reference audio segments usually also exhibits certain similarities and patterns. Therefore, encoding this patterned and similar difference information results in a data size that is effectively smaller than the number of difference information segments and also smaller than the total audio data size. This means that when storing audio images, only the smaller encoded data needs to be stored, thus reducing the storage space required for audio data storage. Furthermore, since the encoded data and the audio library can reconstruct the audio, storing the smaller encoded data effectively stores the audio itself.

[0044] Sixthly, this application also provides an audio processing apparatus, characterized in that the apparatus comprises: an acquisition module for acquiring audio to be processed; a segmentation module for segmenting the audio to obtain multiple original audio segments; a comparison module for comparing each of the multiple original audio segments with at least one reference audio segment in an audio library to obtain difference information corresponding to the multiple original audio segments, wherein the at least one reference audio segment in the audio library includes a portion of the multiple original audio segments, and the difference information is used to describe the audio difference between the audio and the at least one reference audio segment; and an encoding module for encoding the difference information to obtain an encoding result corresponding to the audio, wherein the encoding result is used to recover the audio with the at least one reference audio segment.

[0045] Seventhly, this application provides a computing device, which includes a processor and a memory. The processor and the memory communicate with each other. The processor is used to execute instructions stored in the memory to cause the computing device to perform the operation steps of the image processing method described in the first aspect or any implementation thereof, or to cause the computing device to perform the operation steps of the image processing method described in the third aspect or any implementation thereof. It should be noted that the memory may be integrated into the processor or may be independent of the processor. The computing device may also include a bus. The processor is connected to the memory via the bus. The memory may include readable storage memory and random access memory.

[0046] Eighthly, this application provides a storage system comprising a processor and at least one storage medium. The processor is configured to execute the operation steps of the image processing method described in the first aspect or any implementation thereof, or to execute the operation steps of the image processing method described in the third aspect or any implementation thereof. The at least one storage medium is configured to persistently store image encoding results.

[0047] Ninthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computing device, cause the computing device to perform the operation steps of the image processing method described in the first aspect or any implementation thereof, or to perform the operation steps of the image processing method described in the third aspect or any implementation thereof.

[0048] In a tenth aspect, this application provides a computer program product containing instructions that, when run on a computing device, causes the computing device to perform the operation steps of the image processing method described in the first aspect or any implementation thereof, or to perform the operation steps of the image processing method described in the third aspect or any implementation thereof.

[0049] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0050] Figure 1 is a schematic diagram of an exemplary storage system;

[0051] Figure 2 is a flowchart illustrating an image processing method provided in this application;

[0052] Figure 3 shows how the processor 100 uniformly converts images of various formats into standard formats.

[0053] Figure 4 is a schematic diagram showing the similarity of some pixel regions in different images;

[0054] Figure 5 is a schematic diagram of the data volume reduction of pixel data by processor 100;

[0055] Figure 6 is a schematic diagram of the difference sequence obtained by comparing the original pixel block and the reference pixel block;

[0056] Figure 7 is a schematic diagram of the process by which the processor 100 reduces the amount of metadata.

[0057] Figure 8 is a flowchart illustrating another image processing method provided in this application;

[0058] Figure 9 is a schematic diagram of another exemplary storage system;

[0059] Figure 10 is a flowchart illustrating an audio processing method provided in this application;

[0060] Figure 11 is a schematic diagram of the structure of an image processing device provided in this application;

[0061] Figure 12 is a schematic diagram of another image processing device provided in this application;

[0062] Figure 13 is a schematic diagram of an audio processing device provided in this application;

[0063] Figure 14 is a schematic diagram of the hardware structure of a computing device provided in this application. Detailed Implementation

[0064] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, various non-limiting embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained based on the embodiments in this application and based on the above content are within the scope of protection of this application.

[0065] Referring to Figure 1, which is a schematic diagram of an exemplary storage system, the storage system 10 includes a processor 100 and at least one storage medium. Figure 1 illustrates this using N storage media as an example, referred to as storage media 1 to storage media N. Each storage medium can be, for example, a solid-state drive (SSD), a hard disk drive (HDD), or a magnetic disk, and can be used for persistent data storage. Furthermore, the storage system 10 can include different types of storage media.

[0066] For example, processor 100 may be a general-purpose processor, such as a central processing unit (CPU); or processor 100 may be a controller that can encode images and write them to a storage medium; or processor 100 may be an accelerator card, such as a graphics processing unit (GPU) or a tensor processing unit (TPU); or processor 100 may be any one of the following processors or any combination thereof: application-specific integrated circuit (ASIC), programmable logic device (PLD), complex programmable logical device (CPLD), field-programmable gate array (FPGA), generic array logic (GAL), system on chip (SoC), software-defined infrastructure (SDI) chip, artificial intelligence (AI) chip, data processing unit (DPU).

[0067] In practical applications, the processor 100 can be configured with memory 301, as shown in Figure 1. The processor 100 can then use memory 301 to store data and perform corresponding processing operations based on the data in memory 301. Alternatively, the storage system 10 can also be configured with a separate memory 302, as shown in Figure 1. The data required or generated by the processor 100 for performing data processing operations can be saved through memory 302.

[0068] Furthermore, the storage system 10 can also communicate with the application 200, such as receiving images provided by the application 200 and sending notification messages to the application 200 (e.g., notifying the application 200 that the image storage was successful). For example, the application 200 can be a client on the user side or a process running on a server. During operation, the application 200 can send multiple images to the storage system 10 so that the storage system 10 can persistently store the images in the corresponding storage media.

[0069] Processor 100 can receive images sent by application 200. Typically, if processor 100 directly stores the image to the storage medium, the image's large data size will result in it occupying a significant amount of storage space. Even if processor 100 encodes multiple images one by one, the large differences between pixels in each image will still result in a large data size for each encoded image, thus requiring storage system 10 to consume substantial storage resources to store these multiple images.

[0070] Based on this, in the storage system 10 shown in Figure 1, the processor 100 can encode multiple images according to the similarity between the images, so as to minimize the amount of data of the encoded multiple images.

[0071] In practice, after receiving multiple images sent by application 200, processor 100 can group the images according to their pixel features. Specifically, similar images are grouped into the same group, resulting in multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups. For example, assuming the application sends images A, B, C, and D, processor 100 can group similar images A and B into group 1, and similar images C and D into group 2. Therefore, the similarity between image A and image B within the same group will be higher than the similarity between image A in group 1 and image C in group 2.

[0072] Then, for one of the groups, hereinafter referred to as the first group, the processor 100 uses the pixel library corresponding to the first group to encode the images within the first group, obtaining the image encoding result corresponding to the first group. This pixel library includes pixels from a portion of the images within the first group; that is, the pixel library can be created by the processor during the encoding process of the images within the first group using pixels from a portion of the images within the first group, and the created pixel library can be cached in memory 301 or memory 302. For example, for the aforementioned group 1, the processor 100 uses some or all of the pixels in image A to create the pixel library corresponding to group 1. Similarly, for the remaining groups, the processor 100 can encode the images within the remaining groups in a similar manner, obtaining the image encoding result corresponding to each group.

[0073] Finally, the processor 100 can send the image encoding results corresponding to each group to one or more storage media for persistent storage. Furthermore, the processor 100 can also send the pixel library corresponding to each group to the storage media for persistent storage.

[0074] Accordingly, when application 200 subsequently requests the multiple images from storage system 10, processor 100 can recover each image in the first group based on the pixel library corresponding to the first group and the image encoding result corresponding to the first group. Similarly, processor 100 can recover the images within each group, and thus processor 100 can feed back the recovered multiple images to application 200 to satisfy application 200's query requirements for multiple images. In this way, storage system 10 saves the image encoding results corresponding to each group, which is equivalent to saving the multiple images sent by application 200.

[0075] Because multiple images within the same group have a high degree of similarity, the processor 100 encodes multiple images within that group based on pixels from a subset of images in that group. This fully leverages the similarity between different images to improve the encoding effect for multiple images within that group. Specifically, the processor 100 can utilize the similarity between images to effectively compress the data volume of images within each group, resulting in a significantly smaller amount of encoded image data compared to the original image data. Consequently, during subsequent storage of multiple images, only the smaller encoded result needs to be stored, thus reducing the storage space required for storing multiple images and improving the image reduction rate. The reduction rate is the ratio of the reduced data volume to the original data volume.

[0076] For example, suppose an image A in the first group contains 172,800 pixels (resolution 480×360), and the value of each pixel is represented using 1 byte (B) (e.g., the image can be grayscale). Then the total number of pixel data in image A is 172,800B. Furthermore, suppose there exists an image B in the pixel library that has a high similarity to image A. Specifically, 170,000 consecutive pixels in image A have the same value as pixels in image B, while the remaining 2,800 consecutive pixels have different values ​​from pixels in image B. Then, processor 100 can encode image A using image B from the pixel library. Specifically, it can calculate the difference between the pixel values ​​in image A and the pixel values ​​in image B to obtain a corresponding difference sequence. The first 170,000 values ​​in this difference sequence are all 0, and the remaining 2,800 differences can be any values ​​between -255 and 255. Then, the processor 100 can encode the difference sequence into "(0, 170000), ..." (i.e., the image encoding result corresponding to image A). In this way, the processor 100 can encode the values ​​of the first 170,000 pixels in image A into "(0, 170000)". Assuming "(0, 170000)" occupies 10 bytes (each value occupies 4 bytes), the processor 100 can save at least 169,990 bytes of storage space after encoding image A, resulting in a reduction rate of 98.37% (i.e., 169,990 / 172,800). Similarly, multiple images within the first group can be encoded in the same way, thereby reducing the overall data size of the multiple images within the first group.

[0077] Furthermore, when the processor 100 uses a lossless encoding algorithm to encode the images in each group, the processor 100 can subsequently recover the complete image data in that group based on the encoding results corresponding to each group and the pixel library corresponding to that group. This reduces the storage space occupied by image storage while achieving lossless image storage, that is, it can preserve the complete image information.

[0078] In practical applications, the N storage media in the storage system 10 shown in Figure 1 can be deployed centrally or in a distributed manner.

[0079] In a centralized deployment, storage system 10 can be deployed on a single storage server, which is configured with N storage media, such as multiple SSDs (each SSD being a storage medium as shown in Figure 1) that can be inserted into the storage server. Furthermore, the processor 100 in storage system 10 can store the encoded results obtained after encoding multiple images into the configured storage media.

[0080] When a distributed deployment is adopted, the N storage media in storage system 10 can be distributed across multiple different storage servers. Each storage server can be configured with one or more storage media; for example, multiple SSDs (each SSD being a storage media as shown in Figure 1) can be inserted into each storage server. Furthermore, each storage server can be configured with a processor 100 to encode and store multiple images received by that storage server. Alternatively, the processor 100 can be deployed independently of multiple storage servers, allowing it to encode multiple images and then distribute the encoded results to different storage servers. Or, the processor 100 can be deployed to one of the storage servers, and after encoding multiple images received by that server, it can distribute the generated encoded results to multiple different storage servers. In the distributed storage system, different storage servers can communicate based on a communication protocol, such as Compute Express Link (CXL), Unified Bus (UB or Ubus), Remote Direct Memory Access (RDMA), or other types of communication protocols.

[0081] It is worth noting that the storage system 10 shown in Figure 1 is merely an illustrative example and is not intended to be limiting. For example, other possible storage systems may include general-purpose processors or heterogeneous processors, or other hardware. Alternatively, in other possible storage systems, the processor 100 may be integrated with the storage medium. In this case, the storage system 10 may specifically be a storage device capable of encoding and storing received image data, wherein the processor 100 may be a controller within the storage device.

[0082] For ease of understanding, embodiments of the image processing method provided in this application will be described below with reference to the accompanying drawings.

[0083] Referring to Figure 2, which is a flowchart illustrating an image processing method according to an embodiment of this application, this method can be applied to the storage system 10 shown in Figure 1, or to other applicable storage systems. For ease of explanation, this embodiment uses the storage system 10 shown in Figure 1 as an example for illustrative purposes.

[0084] The image processing method shown in Figure 2 may specifically include the following steps.

[0085] S201: Application 200 sends multiple images to be processed to storage system 10.

[0086] The image sent by application 200 can be any encoded image format. For example, the image sent by application 200 can be in Joint Photographic Experts Group (JPEG) format, Portable Network Graphics (PNG) format, Web Picture (WebP) format, Scalable Vector Graphics (SVG) format, Bitmap (BMP) format, Raw Image File (RAW) format, Run-Length Coding (RLE) format, or Tagged Image File (TIFF) format, or other formats, without limitation.

[0087] In one possible implementation, application 200 can generate multiple images during operation, such as generating multiple rendered images based on user rendering instructions. Then, application 200 can send image 1 to storage system 10. For example, when application 200 is specifically a user-side client, the client can generate a data storage request based on the user's storage operation instructions for the generated multiple images, and send the data storage request, which includes multiple images, to storage system 10. Accordingly, storage system 10 can parse the multiple images specified by the user from the received data storage request. In practical applications, storage system 10 can also obtain the multiple images generated by application 200 through other methods, and this is not limited.

[0088] In practical applications, the multiple images sent by application 200 to storage system 10 may include images of various formats, and the encoding algorithms and image data organization methods used by different image formats usually differ significantly. Therefore, processor 100 can convert the received multiple images into a standard format image, that is, convert the multiple images sent by application 200 into a unified format image, as shown in Figure 2. Subsequently, processor 100 can perform subsequent processing operations on the standard format image. For example, as shown in Figure 3, processor 100 can convert multiple images in formats such as JPEG, PNG, WebP, and SVG into a unified Huawei Universal Image Format (HUIF) image, which is the standard format. Alternatively, the standard format can also be other types of formats; there is no limitation on this.

[0089] In specific implementation, the processor 100 can decode each received image separately to obtain complete pixel data and metadata for each image. The pixel data may be, for example, the value of each pixel in the image, and the metadata may be data describing the image format, the encoding algorithm used, and the image color type (grayscale or color, etc.). The decoded pixel data can be standard format image data. Alternatively, after decoding the pixel data for each image, the processor 100 can encode the pixel data of each image using an encoding algorithm corresponding to the format, obtaining a standard format image. Furthermore, when encoding the pixel data of each image based on the standard format encoding algorithm, the processor 100 can also divide the image into multiple pixel blocks, thereby allowing the processor 100 to obtain the division information of multiple pixel blocks in each image. For example, the size of each pixel block can be, for example, 8 (pixels) × 8 (pixels), 16 × 16, or 32 × 32, etc.

[0090] Furthermore, during the format conversion of multiple images, the processor 100 can also extract the metadata of each image from its image file. Since the metadata in images of different formats differs in data organization and description methods, the processor 100 can also convert the metadata corresponding to each image into metadata of a unified format to facilitate subsequent processing and management of the metadata. For example, as shown in Figure 3, the processor 100 can convert the metadata corresponding to multiple images in formats such as JPEG, PNG, WebP, and SVG into a unified format of hybrid metadata.

[0091] Thus, the image file obtained after format conversion can include pixel data and mixed metadata (as well as pixel block division information). This enables the conversion of multiple images of different formats into an image of a unified format. It should be noted that in practical applications, application 200 can also send multiple images of the same format to storage system 10, such as sending multiple images of a standard format. In this case, processor 100 does not need to perform format conversion on the received multiple images. For ease of explanation, the multiple images processed by processor 100 from step S202 onwards will not be distinguished as either the multiple images obtained after format conversion or the multiple images received by processor 100.

[0092] In this embodiment, after acquiring multiple images (or after converting multiple images into a uniform format), the processor 100 can perform data reduction on the multiple images to improve the reduction rate of the multiple images, thereby reducing the storage space required to store the multiple images. The process of processing multiple images will now be described in detail with reference to Figure 2.

[0093] S202: Processor 100 extracts pixel features from each image.

[0094] For example, the pixel features of an image can be at least one of the brightness features and color features of pixels in the image, or other types of features, such as pixel features generated by weighted calculation of brightness and color features, etc., and there is no limitation on this. The following describes various non-limiting implementations for extracting brightness and color features from an image.

[0095] As examples, when the extracted pixel features are specifically the brightness features of pixels, the processor 100 can extract the brightness features from the image in various ways.

[0096] In one implementation method, the image can be a color image. In this case, the pixel values ​​in the image include values ​​from three channels: red (R), green (G), and blue (B). The processor 100 can then calculate the brightness value of each pixel based on its pixel values, such as by taking a weighted average of the R, G, and B channel values ​​for each pixel. The processor 100 can then extract the brightness features of the image based on the brightness values ​​of each pixel. For example, it can use the average, median, maximum, or minimum brightness values ​​of all pixels in the image as the brightness features, or it can extract the brightness features of the image using other methods based on the brightness values ​​of each pixel.

[0097] When the image is a grayscale image, the processor 100 can use the grayscale value of each pixel as the brightness value of that pixel, and extract the brightness features of the image based on the brightness values ​​of each pixel in the image.

[0098] In the second implementation method, the processor 100 can use an optical flow estimation algorithm (or gradient calculation method) to calculate the optical flow direction (or gradient descent direction) of multiple pixels in the image based on the pixels in the image, and then use the optical flow direction (or gradient descent direction) of the pixels in the image as the brightness feature of the image.

[0099] In the third implementation method, the processor 100 can perform edge detection on the image based on the brightness values ​​of each pixel in the image using an edge detection operator, and use the detected boundary information as the brightness feature of the image. The edge detection operator can be, for example, the Roberts Cross operator, Prewitt operator, Sobel operator, Kirsch operator, Canny operator, or compass operator, or other operators capable of detecting boundaries in an image.

[0100] In the fourth implementation method, the processor 100 can input an image into a pre-trained artificial intelligence (AI) model and use the AI ​​model to evaluate the brightness of the image. The processor 100 can then use the inference results output by the AI ​​model (such as the evaluated brightness value) as the brightness feature of the image. The AI ​​model can be, for example, a model built based on deep learning algorithms or other algorithms.

[0101] In practical applications, the processor 100 can also extract brightness features from the image in other ways, and there is no limitation on this.

[0102] As examples, when the extracted pixel features are specifically the color features of pixels, the processor 100 can extract color features from the image in various ways.

[0103] In the first implementation method, the image is a color image, meaning each pixel in the image can include values ​​from three channels: R, G, and B. Therefore, the processor 100 can calculate the average value RA of the R channel values ​​of all pixels in the image. average The average value of the G channel of all pixels in the image. average The average value of the B channel of all pixels in the image. average and the calculated R average G average B average Alternatively, the processor 100 can perform a weighted calculation on the values ​​of each pixel in the image and use the weighted calculation result as the color feature of the image.

[0104] In the second implementation method, the processor 100 can input the image into a pre-trained AI model and use the AI ​​model to output the color information of the image. This color information may be, for example, the color saturation of the image, and this color information is used as the color feature of the image. The AI ​​model can be, for example, a model built based on deep learning algorithms or other algorithms.

[0105] In practical applications, the processor 100 can also extract color features from images in other ways, without limitation.

[0106] Referring to the above method, the processor 100 can extract the color features or brightness features of each image and use the color features or brightness features as the pixel features of the image, or use the features obtained after performing corresponding calculations based on the color features and brightness features as the pixel features of the image, etc.

[0107] S203: The processor 100 groups the multiple images according to their pixel features, resulting in multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups.

[0108] In real-world applications, different images may exhibit a certain degree of similarity. For instance, suppose application 200 sends multiple images to storage system 10, specifically the four images shown in Figure 4. Image A and image B are similar in pixels within the dotted line, while image C and image D are similar in pixels within the background area (excluding the person). Therefore, processor 100 can group the images based on their similarity, such as grouping image A and image B together, and image C and image D together.

[0109] In a specific implementation, the processor 100 can calculate the similarity between pixel features of different images based on the pixel features of each image, and group images whose similarity between pixel features is greater than a threshold into the same group, thus obtaining multiple groups, namely group 1 to group N, where N is a positive integer greater than 1. For example, the processor 100 can use the K-means algorithm or an AI model to group the multiple images based on their pixel features.

[0110] Typically, each group can include multiple images. When an image is dissimilar to all other images, the processor 100 can group that image into a separate group. Furthermore, when the pixel features of an image include color features and brightness features, or are determined based on color features and brightness features, the processor 100 determines the similarity between different images based on the image's color information and illumination information, which can improve the accuracy of judging the similarity between different images and reduce false positives.

[0111] After dividing the image into multiple groups, the processor 100 can encode the images within each group to reduce the amount of data in that group. For ease of explanation, the following example illustrates the encoding of images within one of the multiple groups, which is group 1.

[0112] S204: The processor 100 uses the pixel library 1 corresponding to group 1 to encode the image in group 1 and obtain the image encoding result corresponding to group 1, wherein the pixel library 1 includes pixels in some images in group 1.

[0113] During the encoding of multiple images within group 1, processor 100 can create a pixel library 1 for group 1, so that pixels in pixel library 1 can be used to compress multiple images within group 1 later. Furthermore, processor 100 can compare pixels in the images within group 1 with pixels in pixel library 1 to obtain pixel difference information. This pixel difference information describes the differences between pixels in the images within group 1 and pixels in pixel library 1. Therefore, processor 100 can encode this pixel difference information to obtain the image encoding result corresponding to group 1, that is, the encoding result for multiple images within group 1.

[0114] In this embodiment, the following implementation examples of generating image encoding results corresponding to group 1 are provided.

[0115] As a first implementation example, processor 100 can add all pixels from a subset of images in group 1 to pixel library 1. Processor 100 can then use the pixels of the subset of images as a reference (hereinafter referred to as the reference image) to calculate the pixel differences between each image in group 1 and the reference image in pixel library 1, obtaining pixel difference information for each image in group 1. This pixel difference information describes the difference between pixels in the image and pixels in the reference image in pixel library 1. For example, this pixel difference information can be the residual (difference sequence) between the values ​​of all pixels in image 1 and the values ​​of all pixels in an image in pixel library 1. Then, processor 100 can encode the pixel difference information corresponding to each image to obtain the encoding result for that image. The encoding results for multiple images are thus the image encoding result for group 1.

[0116] For example, assuming group 1 includes image 1, image 2, and image 3, processor 100 can add all pixels from image 1 to pixel library 1. In this case, image 1 becomes the reference image in pixel library 1. Processor 100 can then calculate the pixel differences between images 1, 2, and 3 relative to the reference pixels in pixel library 1, obtaining pixel difference information for each image. Then, processor 100 encodes the pixel difference information for images 1, 2, and 3 respectively, thus obtaining the image encoding result corresponding to group 1. The method by which processor 100 selects images to add to pixel library 1 is not limited; for example, it can randomly select some images from multiple images to add to pixel library 1, or it can add images processed earlier to pixel library 1 according to the image processing order. This is not a limitation.

[0117] In practical applications, since pixel library 1 may include multiple reference images, the processor 100 can also record the mapping relationship between the encoding results of each image in group 1 and the reference images in pixel library 1 during the process of generating the image encoding results corresponding to group 1, so that the correct reference image and the encoding result can be selected to recover the image in group 1 according to the mapping relationship.

[0118] As a second implementation example, processor 100 encodes at the pixel block level. Specifically, processor 100 can process each image in group 1 sequentially. Assuming the currently processed image is image 1, processor 100 can divide image 1 into multiple pixel blocks, as shown in Figure 5. The size of each pixel block can be 8 (pixels) × 8 (pixels), 16 × 16, or 32 × 32, or other sizes smaller than the resolution of image 1. For ease of distinction, the pixel blocks in image 1 will be referred to as original pixel blocks. Processor 100 can traverse multiple original pixel blocks in image 1. For the first original pixel block traversed by processor 100, since pixel library 1 does not contain any pixels, processor 100 can directly add the pixels from the first original pixel block to pixel library 1. For ease of distinction, the original pixel block added to pixel library 1 will be referred to as a reference pixel block.

[0119] Then, the processor 100 traverses to the second original pixel block and determines whether there is a reference pixel block in the pixel library 1 that matches the second original pixel block. Specifically, the matching condition is that the similarity between the pixels of the two pixel blocks is greater than a threshold.

[0120] Specifically, the processor 100 can calculate the similarity between the pixels in the second original pixel block and the pixels in the reference pixel block in the pixel library 1. For example, the processor 100 can use discrete cosine transform (DCT) to calculate the similarity between the pixels in the second pixel block and the pixels in the reference pixel block, respectively, to obtain the calculation results corresponding to the second pixel block and the calculation results corresponding to the reference pixel block; and the processor 100 calculates the similarity between these two calculation results, which is the similarity between the values ​​of each pixel in the second pixel block and the values ​​of the pixels in the reference pixel block.

[0121] When the similarity is greater than a threshold, it indicates that the second original pixel block and the reference pixel block have a high similarity (i.e., there exists a reference pixel block in pixel library 1 that matches the original pixel block). Then, the processor 100 can calculate pixel difference information based on the second original pixel block and the reference pixel block. This pixel difference information describes the deviation between the pixel values ​​in the second original pixel block and the pixel values ​​in the reference pixel block, as shown in Figure 5. For example, the processor 100 can calculate the residual between the pixel values ​​in the second original pixel block and the pixel values ​​in the reference pixel block to obtain a difference sequence, and determine this difference sequence as the pixel difference information corresponding to the second original pixel block.

[0122] For example, assuming image 1 is a grayscale image, and both the second original pixel block and the reference pixel block are 8×8 pixel blocks, the values ​​of the 64 pixels in the second original pixel block and the reference pixel block are shown in Figure 6. Then, the processor 100 can calculate the residual between the pixel values ​​in the second original pixel block and the pixel values ​​in the reference pixel block, obtaining the difference sequence shown in Figure 6.

[0123] Generally, the higher the similarity between pixels in the original pixel block and pixels in the reference pixel block, the more regular the pixel difference information used to describe the differences between pixels in these two pixel blocks will be. For example, when the pixel difference information is a difference sequence, this difference sequence includes many identical or repeated differences, such as the difference sequence in Figure 6 which contains many values ​​of 0 and 5. In this case, the amount of data obtained by encoding this pixel difference information will be smaller, which means a better effect on reducing the storage space required for image storage.

[0124] When the similarity is less than the threshold, indicating a low similarity between the second original pixel block and the reference pixel block, the processor 100 can add the second original pixel block as a reference pixel block to the pixel library 1, as shown in Figure 6. At this time, the pixel library 1 includes two reference pixel blocks.

[0125] Next, the processor 100 iterates to the third original pixel block and calculates the similarity between the pixels of this third original pixel block and each reference pixel block in the pixel library 1. Furthermore, when there is a reference pixel block whose similarity to the pixels of the third original pixel block is greater than the threshold, the processor 100 can use that reference pixel block to calculate the pixel difference information corresponding to the third original pixel block. If there is no reference pixel block in the pixel library 1 whose similarity to the pixels of the third original pixel block is greater than the threshold, the processor 100 can add the third original pixel block as a new reference pixel block to the pixel library 1; at this time, the pixel library 1 includes 3 reference pixel blocks.

[0126] Similarly, for each original pixel block in image 1, the processor 100 can calculate the pixel difference information corresponding to each original pixel block in the manner described above.

[0127] For the original pixel blocks added to pixel library 1, their pixel difference information can describe that the difference between the pixels in the original pixel block and the reference pixel block in pixel library 1 is 0 (e.g., the residual is 0), meaning that the original pixel block is the same as a reference pixel block in pixel library 1. Thus, the processor 100 can process each original pixel block in image 1 based on a unified logic in the above manner. Therefore, after calculating the pixel difference information corresponding to each original pixel block in image 1 in the above manner, the processor 100 can obtain the pixel difference information of the entire image 1 relative to pixel library 1. Alternatively, the processor 100 may not need to encode the original pixel blocks added to pixel library 1. The final image encoding result generated by the processor 100 for the first group is the encoding result for multiple original pixel blocks not added to pixel library 1. Therefore, the processor 100 can store this image encoding result along with pixel library 1 in the storage medium. Alternatively, for the original pixel blocks added to pixel library 1, processor 100 can adjust or set the values ​​of the original pixel blocks (e.g., set all values ​​to 0) to minimize the amount of data in the subsequent encoding results for the original pixel blocks. In this embodiment, there are no limitations on whether the original pixel blocks added to pixel library 1 are encoded or on the specific encoding method.

[0128] Similarly, for each image within group 1, the processor 100 can calculate the pixel difference information corresponding to each image in the manner described above. Accordingly, during the calculation of the pixel difference information, the processor 100 completes the construction of pixel library 1.

[0129] Finally, the processor 100 can encode the pixel difference information corresponding to each original pixel block. After encoding the pixel difference information corresponding to all original pixel blocks in all images within group 1, the image encoding result corresponding to group 1 can be obtained, as shown in Figure 5. Since the similarity between the pixels of the original pixel blocks in each image and the reference pixel blocks in pixel library 1 is high, the pixel difference information used to describe the differences between the original pixel blocks and the reference pixel blocks is more regular. At this time, the amount of data in the encoding result obtained by the processor 100 for encoding this difference information will be smaller, which means a better effect on reducing the storage space required for image storage.

[0130] For example, taking the pixel difference information corresponding to a certain original pixel block in image 1 as the difference sequence shown in Figure 6, the data size of this difference sequence is 64 bytes, and there are multiple consecutive identical values ​​in the difference sequence, such as multiple consecutive values ​​of 0 and 5. Then, the processor 100 can use the RLE algorithm to encode the difference sequence, obtaining the encoded result (0, 24), 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, (5, 24). At this time, the data size of the encoded result is 24 bytes, which is less than the data size of the original pixel block itself (64 bytes), thereby increasing the reduction rate of the original pixel block to 62.5% (i.e., (64-24) / 64). Similarly, when the remaining pixel blocks in image 1 are encoded in a similar way, the reduction rate of image 1 can also be effectively improved.

[0131] In practical applications, since pixel library 1 may include multiple reference pixel blocks, the processor 100 can also record the mapping relationship between the encoding result of each original pixel block and the reference pixel block in pixel library 1 during the process of generating the image encoding result corresponding to group 1, so that the correct reference pixel block can be selected and the encoding result can be used to recover the original pixel block in the future based on the mapping relationship.

[0132] In practical applications, during the process of calculating the pixel difference information corresponding to multiple images in group 1, the number of reference pixel blocks included in pixel block 1 gradually increases. For example, pixel library 1 may include more than 1,000 reference pixel blocks. Therefore, when determining the reference pixel block that meets the matching conditions for an original pixel block in an image, a lot of computing resources are required.

[0133] Therefore, in a further possible implementation example, when the number of reference pixel blocks included in pixel library 1 is greater than a preset value (e.g., 1000), each reference pixel block in pixel library 1 can be configured with a priority, which indicates the priority of the reference pixel block in determining whether it meets the matching conditions with the original pixel block. Thus, when the processor 100 determines a reference pixel block that meets the matching conditions for each original pixel block, it can prioritize calculating the similarity between the original pixel block and a reference pixel block with higher priority in pixel library 1 based on the priority of each reference pixel block in pixel library 1. Furthermore, when the similarity between the reference pixel block and the original pixel block is greater than a threshold, the processor 100 can use the reference pixel block to calculate the pixel difference information corresponding to the original pixel block and continue to determine a reference pixel block with higher similarity for the next original pixel block. When the similarity between the reference pixel block and the original pixel block is less than a threshold, the processor 100 can continue to calculate the similarity between the original pixel block and the next reference pixel block in pixel library 1 based on priority, until a reference pixel block that meets the matching conditions for the original pixel block is determined. In this way, for each original pixel block, the processor 100 can quickly traverse to find the reference pixel blocks that meet the matching conditions of the original pixel block, so that it is not necessary to traverse all the reference pixel blocks in the pixel library 1 and perform calculations for the original pixel block. This can reduce the resource consumption required to determine the reference pixel block and improve the efficiency of determining the reference pixel block.

[0134] Furthermore, the processor 100 can be configured with multiple encoding algorithms, and during the process of encoding the pixel difference information corresponding to group 1, the processor 100 can use multiple encoding algorithms to encode the pixel difference information in sequence in order to achieve the highest possible reduction rate.

[0135] For example, taking the encoding of the difference sequence shown in Figure 6 as an example, the processor 100 can first use the RLE algorithm to encode the difference sequence into (0, 24), 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, (5, 24). Since the result obtained by the RLE algorithm contains consecutive repetitions of "1, 2, 3, 4", the processor 100 can continue to use the LZ (Lempel-Ziv) encoding algorithm to encode the current encoding result. The LZ encoding algorithm is a dictionary-based encoding algorithm. Assuming the dictionary includes the string "1, 2, 3, 4", and this string can be replaced with A, the processor 100 uses the LZ encoding algorithm to encode (0, 24), A, A, A, A, (5, 24). Finally, the processor 100 can continue to use the RLE algorithm to encode the current encoding result into (0, 24), (A, 4), (5, 24). In this way, the data size of the final encoded result can be reduced to 6 bytes, thereby increasing the reduction rate of the original pixel block to 90.6% (i.e. (64-6) / 64).

[0136] In practical applications, after acquiring pixel difference information, the processor 100 can extract data features from the pixel difference information. These data features can be, for example, features such as continuously repeating identical values ​​or cyclically repeating values, as shown in Figure 6. Since different encoding algorithms are suitable for encoding information with different data features (i.e., the information can achieve a high reduction rate after encoding), the processor 100 can determine one or more encoding algorithms that match the data feature from a variety of configured encoding algorithms, and then use the determined one or more encoding algorithms to encode the difference information sequentially. Specifically, using multiple encoding algorithms to encode the difference information sequentially means that after the current encoding algorithm encodes the difference information to obtain the encoding result, the next encoding algorithm continues to encode the encoding result, and so on, until the final encoding result is obtained.

[0137] In a further possible implementation, after encoding the difference information using at least one determined encoding algorithm, the processor 100 may further extract the data features of the encoding result and determine whether to further encode the encoding result based on the data features, so as to further reduce the data volume of the encoding result.

[0138] The encoding algorithm used to encode pixel difference information can be a lossless encoding algorithm, meaning that no information loss occurs after encoding the pixel difference information, such as the RLE algorithm. Alternatively, the encoding algorithm used to encode pixel difference information can also be a lossy encoding algorithm, meaning that a small amount of information loss occurs after encoding the pixel difference information, such as predictive coding algorithms.

[0139] The above explanation uses the encoding of images in group 1 as an example. In practical applications, for each of multiple groups, the processor 100 can encode the images within that group in the same manner to obtain the image encoding result corresponding to that group, thus obtaining the image encoding results corresponding to multiple groups, i.e., obtaining the image encoding results corresponding to multiple images. For example, for group 2, the processor 100 can create a pixel library 2 based on the pixels in some images within the group, and use the reference images or reference pixel blocks in pixel library 2 to encode the images within group 2 to obtain the image encoding result corresponding to group 2.

[0140] In addition, when the processor 100 converts multiple received images into standard format images, each image also carries metadata. At this time, the processor 100 can also save the metadata of each image.

[0141] When recording the metadata of each image, the processor 100 can directly save the metadata of each image in the storage medium.

[0142] Alternatively, the processor 100 can refer to the above method of saving the image encoding results corresponding to multiple images, and save the encoding results corresponding to the metadata of multiple images in the storage medium, hereinafter referred to as the metadata encoding results, thereby reducing the amount of metadata of the multiple images and further reducing the overall amount of data of the multiple images.

[0143] In a specific implementation, the pixel library corresponding to each group may also include reference metadata. As shown in Figure 7, the processor 100 can calculate the similarity between the metadata of the image within the group and each reference metadata in the pixel library corresponding to that group. Furthermore, when there is a reference metadata in the pixel library whose similarity to the image's metadata is greater than a threshold, the processor 100 can compare the differences between the image's metadata and the reference metadata to obtain metadata difference information. This metadata difference information describes the difference between the two metadata sets. For example, this metadata difference information could be the residual between two metadata sets. When there is no reference metadata in the pixel library whose similarity to the image's metadata is greater than a threshold, the processor 100 can add the image's metadata as reference metadata to the pixel library, as shown in Figure 7, so that the newly added reference metadata can be used subsequently to reduce the amount of metadata for other images.

[0144] Then, the processor 100 can encode the metadata difference information corresponding to each image in each group to obtain the metadata encoding result of each image, thereby obtaining the metadata encoding result corresponding to each group (including the encoding results corresponding to the metadata of multiple images in the group respectively).

[0145] Furthermore, during the metadata encoding process, when the similarity between reference metadata in the pixel library and the image's metadata exceeds a threshold, the processor 100 can also calculate the benefit 1 for encoding the image's metadata and the benefit 2 for encoding the image's metadata difference information based on a preset formula. When benefit 1 is less than benefit 2, the processor 100 can encode the metadata difference information to reduce the storage space required for subsequent metadata storage; while when benefit 1 is greater than or equal to benefit 2, the processor 100 can directly encode the image's metadata to minimize the encoding result of the image's metadata, thereby minimizing the storage space required for subsequent metadata storage.

[0146] Furthermore, after generating the image encoding results (and metadata encoding results) corresponding to each group, the processor 100 can persistently store them. For this purpose, the embodiment shown in FIG2 may also include the following step S205.

[0147] S205: The processor 100 sends the image encoding results corresponding to multiple images to the storage medium.

[0148] After generating the image encoding results corresponding to each group, the processor 100 can output the image encoding results corresponding to each group to one or more storage media for saving. Thus, saving the image encoding results corresponding to each group is equivalent to saving multiple images.

[0149] In practical applications, during the process of saving the image encoding results, the processor 100 may specifically save the image encoding results, the identifier of the group to which the image encoding results belong, the identifiers of the multiple images included in each group, and the mapping relationship in the storage medium. The mapping relationship may be, for example, the correspondence between the encoding results of each image in each group and the reference image in the pixel library corresponding to the group, or the correspondence between the encoding results of each original pixel block of each image in each group and the reference pixel block in the pixel library corresponding to the group.

[0150] Furthermore, when the processor 100 performs format conversion on multiple received images, it can also send the metadata encoding results corresponding to each group, as well as the mapping relationship between the metadata encoding results and the reference metadata in the pixel library, to the storage medium for persistent storage. Additionally, the processor 100 can also send the pixel library corresponding to each group to the storage medium for persistent storage.

[0151] Accordingly, when application 200 requests multiple images to be saved from storage system 10, processor 100 can reconstruct multiple images based on the image encoding results corresponding to the multiple images and the pixel library, and then feed the multiple images back to application 200.

[0152] Next, we will continue to describe the process by which the storage system 10 sends multiple images back to the application 200.

[0153] S206: Application 200 sends a data read request to storage system 10, which is used to request multiple images from storage system 10.

[0154] S207: The processor 100 responds to the data read request and recovers multiple images based on the image encoding results corresponding to the multiple images stored in the storage medium.

[0155] In one possible implementation, after receiving a data read request, the processor 100 can parse the identifiers of the multiple images to be read from the data read request, and based on the identifiers of the multiple images, read the image encoding results corresponding to the multiple images in the storage medium, the group identifier to which the image encoding results belong, the identifiers of the multiple images included in each group, and the mapping relationship. Then, the processor 100 can decode the image encoding results according to the group identifier to which the image encoding results belong and the identifiers of the multiple images included in each group to obtain the image difference information corresponding to the images in each group. Next, for each group, the processor 100 can access the pixel library corresponding to the group according to the image difference information and mapping relationship corresponding to the images in the group to obtain the reference image / reference pixel block corresponding to the images in the group. Thus, the processor 100 can calculate the pixel values ​​of each image in the group according to the image difference information and the reference image / reference pixel block, thereby recovering the complete pixel data of the multiple images in the group. Similarly, for each of the other groups, the processor 100 can also recover the complete pixel data of the multiple images in the group in the same manner as described above.

[0156] Furthermore, when the processor 100 records the metadata of multiple images during the process of saving multiple images, during the decoding of the image encoding results, the processor 100 can also read the metadata encoding results corresponding to the multiple images and the mapping relationship between the metadata encoding results and reference metadata from the storage medium. Based on this mapping relationship, the processor 100 uses the reference metadata in the pixel library to decode the metadata decoding results to obtain the metadata corresponding to each image. Specifically, the processor 100 can decode the metadata encoding results based on the reference metadata in the pixel library; the specific implementation method can be found in the description of the relevant parts of decoding the image encoding results above, and will not be repeated here. Thus, after recovering the complete pixel data from multiple images, the processor 100 can encode the recovered pixel data of the multiple images based on the decoded metadata to recover the multiple images sent by the application 200 to the storage system 10. The format of the recovered multiple images is consistent with the format of the multiple images previously sent by the application 200 to the storage system 10 for saving.

[0157] It is worth noting that in this embodiment, the example of simultaneously recording reference metadata and reference image / reference metadata in the pixel library is used for illustration. In actual applications, the processor 100 can be configured with two independent databases, one database for recording reference metadata and the other database for recording reference image / reference metadata.

[0158] S208: The processor 100 feeds back the recovered multiple images to the application 200.

[0159] For example, after recovering multiple images, the processor 100 can generate a response message including the multiple images and send the response message to the application 200 to satisfy the application 200's query requirements for the multiple images.

[0160] Thus, with a pixel library configured in the storage system 10, when saving multiple images 1, the storage system 10 only needs to save the image encoding results (and metadata encoding results) corresponding to the smaller data sizes of the multiple images, without needing to save the original data of the multiple images. This effectively reduces the storage space required to store multiple images, thereby improving the image reduction rate. Furthermore, when the application 200 requests multiple images from the storage system 10, the storage system 10 can reconstruct the complete multiple images based on the pixel library and the saved image encoding results (and metadata encoding results), thereby satisfying the application 200's query requirements for multiple images.

[0161] In addition, when the encoding algorithm used by the processor 100 is a lossless encoding algorithm, the storage system 10 can recover complete multiple images based on the image encoding results and the pixel library. This can ensure the integrity of information when storing images and achieve lossless storage of multiple images.

[0162] Furthermore, when the processor 100 encodes the images within each group at the pixel block granularity, during the process of restoring the images within each group, the processor 100 only needs to read the reference pixel blocks used to restore the original pixel blocks in the pixel library corresponding to that group, without having to read all the reference pixel blocks in the pixel library (and without having to read the reference pixel blocks in the pixel libraries corresponding to other groups), thereby avoiding the problem of read magnification.

[0163] The embodiment shown in Figure 2 above illustrates an example where application 200 sends multiple images to storage system 10 for storage. In other embodiments, application 200 may also send a single image to storage system 10 for storage. The process of processor 100 encoding and storing a received single image is described below.

[0164] Referring to Figure 8, a flowchart illustrating another image processing method provided in this embodiment is shown. This method can be applied to the storage system 10 shown in Figure 1, or to other applicable storage systems. For ease of explanation, this embodiment uses the storage system 10 shown in Figure 1 as an example for illustrative purposes.

[0165] The image processing method shown in Figure 8 may specifically include the following steps.

[0166] S801: Application 200 sends the image 1 to be processed to storage system 10.

[0167] The image 1 sent by application 200 can be an image in any encoded format. For example, the format of image 1 can be JPEG, PNG, WebP, SVG, BMP, RAW, RLE, or TIFF, or other formats, without limitation.

[0168] In one possible implementation, application 200 may generate image 1 during operation and send image 1 to storage system 10. For example, when application 200 is specifically a user-side client, the client can generate a data storage request based on the user's storage operation instruction for image 1 and send the data storage request, which includes image 1, to storage system 10. Accordingly, storage system 10 can parse the user-specified image 1 to be saved from the received data storage request. In practical applications, storage system 10 may also obtain image 1 through other means, and this is not limited.

[0169] In this embodiment, after the storage system 10 acquires image 1, the processor 100 can reduce the data of image 1 to improve the reduction rate of image 1 and reduce the storage space required to store image 1.

[0170] In specific implementation, the processor 100 compares image 1 with a reference pixel set in the pixel library to obtain difference information describing the pixel differences between image 1 and the reference pixel set. The processor 100 then encodes this difference information and sends the encoded result to the storage medium for storage. Subsequently, when application 200 needs to read the stored image 1, the processor 100 can reconstruct image 1 based on the encoded result and the reference pixel set. Thus, the storage system 10 stores the encoded result corresponding to image 1, which is equivalent to storing image 1. The process of processing image 1 will now be explained in detail with reference to Figure 8.

[0171] S802: Processor 100 extracts pixel features from image 1.

[0172] In practical applications, the images sent by application 200 to storage system 10 can be of any format, and the encoding algorithms and image data organization methods used by different image formats usually differ significantly. Therefore, processor 100 can convert the received images to obtain standard format images, that is, convert images of arbitrary formats into images of a unified format. Subsequently, processor 100 can perform subsequent processing operations on the standard format images.

[0173] In a specific implementation, the processor 100 can decode the received image 1 to obtain complete pixel data and metadata of image 1. The pixel data may include, for example, the values ​​of each pixel in image 1. The decoded image data can be in a standard format. Alternatively, after decoding the pixel data of image 1, the processor 100 can encode the pixel data using a preset encoding algorithm to obtain an image in a standard format.

[0174] Then, the processor 100 can extract the corresponding pixel features from the standard format image 1.

[0175] The pixel features of image 1 may be at least one of the brightness features and color features of the pixels in image 1, or may be other types of features, such as pixel features generated by weighted calculation of brightness features and color features, etc., and there is no limitation on this.

[0176] As examples, when the extracted pixel features are specifically the brightness features of pixels, the processor 100 can extract the brightness features from the image 1 in various ways.

[0177] In one implementation method, Image 1 can be a color image. In this case, the pixel values ​​in Image 1 include values ​​from three channels: R (red), G (green), and B (blue). The processor 100 can then calculate the brightness value of each pixel based on its pixel values, such as by taking a weighted average of the R, G, and B channel values ​​for each pixel. The processor 100 can then extract the brightness features of Image 1 based on the brightness values ​​of each pixel. For example, it can use the average, median, maximum, or minimum brightness values ​​of all pixels in Image 1 as the brightness features, or it can extract the brightness features of Image 1 using other methods based on the brightness values ​​of each pixel.

[0178] When image 1 is a grayscale image, processor 100 can use the grayscale value of each pixel as the brightness value of that pixel, and extract the brightness feature of image 1 based on the brightness value of each pixel in image 1.

[0179] In the second implementation method, the processor 100 can use an optical flow estimation algorithm (or gradient calculation method) to calculate the optical flow direction (or gradient descent direction) of multiple pixels in image 1 based on the pixels in image 1, and thus use the optical flow direction (or gradient descent direction) of the pixels in image 1 as the brightness feature of image 1.

[0180] In the third implementation method, the processor 100 can perform edge detection on image 1 based on the brightness values ​​of each pixel in image 1 using an edge detection operator, and use the detected boundary information as the brightness feature of image 1. The edge detection operator can be, for example, the Roberts Cross operator, Prewitt operator, Sobel operator, Kirsch operator, Canny operator, or compass operator, or other operators capable of detecting boundaries in an image.

[0181] In the fourth implementation method, the processor 100 can input image 1 into a pre-trained artificial intelligence (AI) model and use the AI ​​model to evaluate the brightness of image 1. The processor 100 can then use the inference results output by the AI ​​model (such as the evaluated brightness value) as the brightness feature of image 1. The AI ​​model can be, for example, a model built based on deep learning algorithms or other algorithms.

[0182] In practical applications, the processor 100 can also extract brightness features from image 1 in other ways, without limitation.

[0183] As examples, when the extracted pixel features are specifically the color features of pixels, the processor 100 can extract the color features from the image 1 in various ways.

[0184] In one implementation method, image 1 is a color image, meaning each pixel in image 1 can include values ​​from three channels: R, G, and B. Therefore, the image processing device 100 can calculate the average value RA of the R channel values ​​of all pixels in image 1. average The average value of the G channel of all pixels in image 1. average The average value of the B channel of all pixels in image 1. average and the calculated R average G average B average Alternatively, the image processing apparatus 100 may perform a weighted calculation on the values ​​of each pixel in image 1 and use the weighted calculation result as the color feature of image 1.

[0185] In the second implementation method, the processor 100 can input image 1 into a pre-trained AI model and use the AI ​​model to output the color information of image 1. This color information may be, for example, the color saturation of image 1, and this color information is used as the color feature of image 1. The AI ​​model can be, for example, a model built based on deep learning algorithms or other algorithms.

[0186] In practical applications, the processor 100 can also extract color features from the image 1 in other ways, without limitation.

[0187] S803: The processor 100 determines pixel library 1 from multiple pixel libraries based on the pixel features of image 1. The pixels in pixel library 1 satisfy the similarity condition with the pixels in image 1, while the pixels in different pixel libraries do not satisfy the similarity condition.

[0188] In this embodiment, the processor 100 may be pre-configured with multiple pixel libraries. These pixel libraries can be pre-configured by a technician, or the processor 100 can pre-acquire multiple images before acquiring image 1 and construct the multiple pixel libraries based on the pixels in those images. For example, the processor 100 can add pixels from some images to a pixel library, and different pixel libraries can include pixels from different images. In this case, each pixel library can include pixels from at least one of the multiple images, and pixels in different pixel libraries do not meet similarity conditions, such as the similarity between pixels in different pixel libraries being less than a threshold, meaning the pixels in different pixel libraries have significant differences. Each pixel library includes at least one pixel set; and when a pixel library includes multiple pixel sets, the pixels in different pixel sets included in the pixel library are similar, such as the similarity between pixel features in different pixel sets being greater than a threshold.

[0189] In this context, a pixel set can be all the pixels in an image, in which case each pixel set in the pixel library can specifically be an image. Alternatively, a pixel set can be a subset of pixels in an image, in which case each pixel set in the pixel library can specifically be a pixel block in an image. During the construction of the pixel library, multiple pixel blocks from an image (as multiple pixel sets) can be added to the pixel library.

[0190] Generally, the higher the similarity between pixel features in an image and pixel features in a pixel library, the more regular the difference information used to describe the pixel differences between the image and the pixel library. For example, when the difference information is a difference sequence, the difference sequence includes many identical or repeated differences. In this case, the amount of data obtained by encoding the difference information will be smaller, which means a better effect on reducing the storage space required for image storage. Based on this, for the image 1 to be stored, the processor 100 can determine a suitable pixel library for the image 1 in order to minimize the storage space required when storing the image 1. For ease of understanding, in this embodiment, the set of pixels in the pixel library 1 with a high similarity to the pixel features of the image 1 is referred to as the reference pixel set.

[0191] In one possible implementation, each pixel library in the processor 100 may be configured with a reference pixel feature, which may be, for example, the average value of pixel features of the various pixel sets in the pixel library.

[0192] Then, the processor 100 can sequentially calculate the similarity between the pixel features of the image 1 and the reference pixel features of each pixel library, and when there is a reference pixel feature whose similarity with the pixel features of the image 1 is greater than a threshold, the pixel library with the reference pixel feature is determined as the aforementioned pixel library 1.

[0193] S804: The processor 100 compares the image 1 with at least one reference pixel set in the pixel library 1 to obtain difference information, which is used to describe the pixel differences between the image 1 and at least one reference pixel set in the pixel library 1.

[0194] In a first possible implementation, the pixels in the pixel set belong to at least one image; that is, each pixel set may include one or more images. Then, the processor 100 can sequentially perform similarity calculations between image 1 and each image in the pixel library 1. Specifically, it can calculate the similarity between the values ​​of each pixel in image 1 and the values ​​of pixels in each image included in the pixel library 1. For example, the processor 100 can use DCT transform to calculate the similarity between image 1 and the images in the pixel library 1, respectively, to obtain the calculation result corresponding to image 1 and the calculation result corresponding to the images in the pixel library 1. Furthermore, the processor 100 can further calculate the similarity between these two calculation results, which is the similarity between the values ​​of each pixel in image 1 and the values ​​of pixels in the images included in the pixel library 1.

[0195] When it is determined that an image exists in pixel library 1, and the similarity between the pixel values ​​of that image and the pixel values ​​of image 1 is greater than a threshold, processor 100 can identify that image as a reference pixel set. In this case, the number of reference pixel sets participating in the pixel difference comparison is one. Then, processor 100 can calculate the residual between the pixel values ​​of image 1 and the pixel values ​​of the reference pixel set, obtaining a difference sequence corresponding to multiple pixels in image 1. This difference sequence is the aforementioned difference information, which can describe the deviation between the value of each pixel in image 1 and the value of the corresponding pixel in the reference pixel set.

[0196] In a second possible implementation, the pixel set includes pixels from at least one pixel block in an image. Therefore, image 1 can be divided into multiple pixel blocks, each with the same size as the pixel set in pixel library 1. For example, the size of each pixel block can be 8 (pixels) × 8 (pixels), 16 × 16, or 32 × 32, or other sizes smaller than the resolution of image 1. For ease of distinction, the pixel blocks in image 1 will be referred to as original pixel blocks. For each original pixel block in image 1, processor 100 can perform a similarity calculation between the original pixel block and each pixel block in pixel library 1. Specifically, this can involve calculating the similarity between the values ​​of the pixels in the original pixel block and the values ​​of the pixels in the pixel blocks in pixel library 1. When a pixel block exists in pixel library 1, and the similarity between the values ​​of the pixels in that pixel block and the values ​​of the pixels in the original pixel block is greater than a threshold, processor 100 can determine that the pixel block meets the matching condition with the original pixel block and designate that pixel block as a reference pixel block (i.e., the aforementioned reference pixel set). Thus, for each original pixel block in image 1, a reference pixel block that meets the matching conditions with the original pixel block can be determined from the pixel library 1 in the manner described above.

[0197] Then, for each original pixel block in image 1, the processor 100 can calculate the residual between the value of the pixel in the original pixel block and the value of the pixel in the reference pixel block corresponding to the original pixel block, and obtain the difference sequence corresponding to the original pixel block. Thus, the difference sequences corresponding to multiple original pixel blocks are the aforementioned difference information, which can describe the deviation between the value of the pixel in each original pixel block in image 1 and the value of the corresponding pixel in multiple reference pixel sets.

[0198] For example, suppose image 1 is a grayscale image, and both the original pixel block and the reference pixel block are 8×8 pixel blocks. The values ​​of the 64 pixels in the original pixel block and the reference pixel block are shown in Figure 6. The processor 100 can calculate the residual between the values ​​of the pixels in the original pixel block and the values ​​of the pixels in the corresponding reference pixel block, resulting in the difference sequence shown in Figure 6. Because the original pixel block and the reference pixel block have a certain similarity, the resulting difference sequence is prone to having many repeated values; for example, the difference sequence in Figure 6 contains many values ​​of 0 and 5.

[0199] In this embodiment, the example given is that each original pixel block in image 1 has a corresponding reference pixel block. When some original pixel blocks in image 1 do not match a reference pixel block, the processor 100 may not determine the difference information for that original pixel block; that is, the original pixel block can be directly stored in the storage medium. Furthermore, the processor 100 can add the original pixel block to pixel library 1 as a new reference pixel block (i.e., a new set of reference pixels) in pixel library 1, so that this new reference pixel block can be used subsequently to encode other original pixel blocks or pixel blocks in other images to be stored.

[0200] It is understood that the above-described methods for generating difference information are merely illustrative examples. In other embodiments, the processor 100 may also generate difference information in other ways. For example, the difference information may be other types of information besides residuals.

[0201] Thus, at least one set of reference pixels determined for image 1 based on the above method has a high degree of similarity with image 1 in terms of pixel features, that is, it satisfies the similarity condition.

[0202] Furthermore, when the pixel library 1 includes a large number of pixel sets (e.g., more than 1000 pixel sets), taking pixel sets specifically as pixel blocks as an example, each pixel block in the pixel library 1 can be configured with a priority. This priority indicates the degree to which a pixel block is used to determine whether it meets the matching conditions with the original pixel block. Thus, when the processor 100 determines a reference pixel block to match each original pixel block in image 1, it can calculate the similarity between the original pixel block and a higher-priority pixel block in the pixel library 1 based on the priority of each pixel block in the pixel library 1. Furthermore, when the similarity between the pixel block and the original pixel block is greater than a threshold, the pixel block is determined as a reference pixel block, and the processor continues to determine a reference pixel block with a higher similarity for the next original pixel block. When the similarity between the pixel block and the original pixel block is less than a threshold, the processor 100 can continue to calculate the similarity between the original pixel block and the next pixel block (with a higher priority) in the pixel library 1 based on the priority, until a reference pixel block that meets the matching conditions with the original pixel block is determined. In this way, the processor 100 can perform calculations on each original pixel block without traversing all pixel blocks in the pixel library 1, thereby reducing the resource consumption of determining the reference pixel block and improving the efficiency of determining the reference resource block.

[0203] S805: Processor 100 encodes the difference information to obtain the encoding result corresponding to image 1.

[0204] As an implementation example, the processor 100 can be configured with an encoding algorithm, and the processor 100 can use the configured encoding algorithm to encode the difference information to obtain the corresponding encoding result, which is also the encoding result of image 1. In practical applications, when the pixels in image 1 and the pixels in the reference pixel set meet the similarity condition, the generated difference information has a certain pattern. Therefore, the encoding result generated by the processor 100 for this difference information will have a smaller data volume than the data volume of image 1, thereby improving the reduction rate of image 1.

[0205] Taking the difference information corresponding to image 1 as shown in Figure 6 as an example, the data size of this difference sequence is 64 bytes, and there are multiple consecutive identical values ​​in the difference sequence, such as multiple consecutive values ​​of 0 and 5. Then, the processor 100 can use the RLE algorithm to encode the difference sequence, obtaining the encoded result (0, 24), 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, (5, 24). At this time, the data size of the encoded result is 24 bytes, which is less than the data size of image 1 itself, 64 bytes, thereby increasing the reduction rate of image 1 to 62.5% (i.e., (64-24) / 64).

[0206] Furthermore, the processor 100 can be configured with multiple encoding algorithms, and the processor 100 can use multiple encoding algorithms sequentially to achieve the highest possible reduction rate.

[0207] For example, for the difference sequence shown in Figure 6, processor 100 can first use the RLE algorithm to encode the difference sequence as (0, 24), 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, (5, 24). Since the result obtained by the RLE algorithm contains consecutive repetitions of "1, 2, 3, 4", processor 100 can further encode the current result using the LZ (Lempel-Ziv) encoding algorithm. The LZ encoding algorithm is a dictionary-based encoding algorithm. Assuming the dictionary includes the string "1, 2, 3, 4", and this string can be replaced with A, processor 100 will encode the result using the LZ algorithm as (0, 24), A, A, A, A, (5, 24). Finally, processor 100 can further use the RLE algorithm to encode the current result as (0, 24), (A, 4), (5, 24). Thus, the data size of the final encoded result can be reduced to 6 bytes, and the reduction rate of image 1 is increased to 90.6% (i.e. (64-6) / 64).

[0208] In practical applications, after acquiring the difference information, the processor 100 can extract data features from it. These data features can be, for example, features such as continuously repeating identical values ​​or cyclically repeating values, as shown in Figure 6. Since different encoding algorithms are suitable for encoding information with different data features (i.e., the information can achieve a high reduction rate after encoding), the processor 100 can determine one or more encoding algorithms that match the data feature from a variety of configured algorithms, and then use the determined one or more encoding algorithms to encode the difference information sequentially. Specifically, using multiple encoding algorithms to encode the difference information sequentially means that after the current encoding algorithm encodes the difference information to obtain the encoding result, the next encoding algorithm continues to encode the result, and so on, until the final encoding result is obtained.

[0209] In a further possible implementation, after encoding the difference information using at least one determined encoding algorithm, the processor 100 may further extract the data features of the encoding result and determine whether to further encode the encoding result based on the data features, so as to further reduce the data volume of the encoding result.

[0210] The encoding algorithm used to encode the difference information can be a lossless encoding algorithm, meaning that no information loss occurs after encoding the difference information, such as the RLE algorithm. Alternatively, the encoding algorithm used to encode the difference information can also be a lossy encoding algorithm, meaning that a small amount of information loss occurs after encoding the difference information, such as predictive coding algorithms.

[0211] S806: Processor 100 saves the encoded result to the storage medium.

[0212] After generating the encoding result corresponding to image 1, the processor 100 can output the encoding result to one or more storage media for storage. Thus, saving the encoding result is equivalent to saving image 1.

[0213] In practical applications, during the process of saving the encoding result, the processor 100 may specifically save the encoding result, the identifier of image 1, and the mapping between the encoding result and the reference pixel set in the pixel library 1 in the storage medium. This mapping may be, for example, the correspondence between the identifier of the encoding result and the identifier of the reference pixel set in the pixel library 1. Accordingly, when the application 200 requests image 1 from the storage system 10, the processor 100 can reconstruct image 1 based on the encoding result corresponding to image 1 and return image 1 to the application 200.

[0214] Next, we will continue to describe the process by which the storage system 10 feeds back image 1 to the application 200.

[0215] S807: Application 200 sends a data read request to storage system 10, the data read request being used to request image 1 from storage system 10.

[0216] S808: The processor 100 responds to the data read request and recovers image 1 based on the encoding result corresponding to image 1 stored in the storage medium.

[0217] In one possible implementation, after receiving a data read request, the processor 100 can parse the identifier of image 1 from the data read request, and based on the identifier of image 1, read the encoding result corresponding to image 1 in the storage medium, and the mapping between the encoding result and the reference pixel set in pixel library 1. Then, the processor 100 can decode the encoding result to obtain difference information. At the same time, the processor 100 can also access pixel library 1 according to the mapping between the encoding result and the reference pixel set to obtain the corresponding reference pixel set in pixel library 1, and calculate the value of each pixel in image 1 based on the decoded difference information and the reference pixel set, thereby recovering the complete pixel data in image 1.

[0218] Furthermore, during the process of saving the encoding result corresponding to image 1, processor 100 can also record metadata of image 1. This metadata may include, for example, data describing the format of image 1, the encoding algorithm used, and the image color type (grayscale image or color image, etc.). In this way, after recovering the complete pixel data in image 1, processor 100 can encode the recovered pixel data of image 1 according to the recorded metadata of image 1 to recover the image 1 sent by application 200 to storage system 10.

[0219] When recording the metadata of image 1, processor 100 can directly save the metadata of image 1 in the storage medium. Alternatively, processor 100 can save the encoding result corresponding to the metadata in the storage medium, referring to the method described above for saving the encoding result corresponding to the pixel data in image 1.

[0220] In specific implementation, the processor 100 may be configured with a metadata database, which includes one or more reference metadata. The processor 100 can compare the differences between the metadata of image 1 and the reference metadata in the metadata database to generate difference information for the metadata, hereinafter referred to as metadata difference information. This metadata difference information describes the differences between the metadata of image 1 and the reference metadata, and may be, for example, a residual. The specific implementation process by which the processor 100 determines the reference metadata in the metadata database for the metadata of image 1 can be found in the description of the relevant parts of determining the reference pixel set above, and will not be repeated here. Then, the processor 100 can encode the metadata difference information and establish a mapping relationship between the encoding result, the encoding result, and the identifier of the reference metadata in the metadata database. Furthermore, the processor 100 can also send the metadata database to a storage medium for persistent storage. At this time, while comparing the pixels in image 1 with the pixels in the reference pixel set in pixel library 1, the processor 100 can also compare the metadata of image 1 with the reference metadata in the metadata database. Therefore, the difference information obtained by the processor 100 can also be used to describe the differences between the metadata of image 1 and the reference metadata. For example, the difference information may specifically include pixel difference information and metadata difference information. The pixel difference information is used to describe the difference between the pixels in image 1 and the pixels in the reference pixel set in pixel library 1, and the metadata difference information is used to describe the difference between the metadata of image 1 and the reference metadata.

[0221] Accordingly, during the recovery of image 1, the processor 100 can read the encoding result and mapping relationship corresponding to the metadata difference information from the storage medium, and decode the encoding result according to the mapping relationship to obtain the metadata difference information. At the same time, it determines the reference metadata in the metadata database according to the mapping relationship. In this way, the processor 100 can recover the metadata of image 1 based on the metadata difference information and the reference metadata in the metadata database.

[0222] In practical applications, the metadata database and the pixel database can be two independent databases; or, the metadata database and the pixel database can be integrated into one database, such as the pixel database storing reference pixel sets and reference metadata at the same time.

[0223] S809: The processor 100 feeds back the recovered image 1 to the application 200.

[0224] For example, after recovering image 1, processor 100 can generate a response message including image 1 and send the response message to application 200 to satisfy application 200's query requirement for image 1.

[0225] Thus, with a pixel library configured in the storage system 10, when saving image 1, the storage system 10 only needs to save the encoding result corresponding to image 1, which has a smaller data volume, without needing to save the original image data. This effectively reduces the storage space required to store image 1, thereby improving the image reduction rate. Furthermore, when application 200 requests image 1 from the storage system 10, the storage system 10 can reconstruct the complete image 1 based on the pixel library and the saved encoding result, thereby meeting the application 200's requirements for image 1.

[0226] In addition, when the encoding algorithm used to encode the difference information is a lossless encoding algorithm, the storage system 10 can recover the complete pixel data of the image 1 based on the encoding result and the reference pixel result. This can ensure the integrity of the information when storing the image and realize lossless storage of the image.

[0227] Furthermore, when the difference information is obtained by comparing at the pixel block level, the processor 100 only needs to read the reference pixel blocks used to restore image 1 in the pixel library 1 during the process of restoring image 1. It does not need to read all the pixel blocks in the pixel library 1 (and does not need to read the pixel blocks in other pixel libraries), thereby avoiding the problem of read magnification.

[0228] It should be noted that this embodiment uses the process of processor 100 processing an image using pixel library 1 as an example. In actual applications, for other images provided by application 200, processor 100 can also use pixel library 1 or other pixel libraries to process other images accordingly, thereby improving the image reduction rate and reducing the storage space required for image storage. For example, when application 200 sends image 2 to storage system 10, processor 100 can determine pixel library 2 for image 2 from multiple pixel libraries. The pixels in pixel library 2 and the pixels in image 2 satisfy similarity conditions. Thus, processor 100 can compare image 2 with at least one reference pixel set in pixel library 2 to obtain difference information, and encode the difference information to obtain the encoding result corresponding to image 2. The encoding result corresponding to image 2 is then saved in the storage medium, thereby saving image 2. The multiple images stored sequentially by storage system 10 can be unrelated images, such as different images taken by the user at different times and locations.

[0229] The above, in conjunction with Figures 1 to 8, describes the image processing procedures during image storage. In practical applications, there may also be scenarios where audio needs to be stored. In this case, the storage space required for audio storage can be reduced by downsizing the audio data. The following examples, in conjunction with Figures 9 and 10, provide further illustration.

[0230] Referring to Figure 9, a schematic diagram of another storage system is shown. As shown in Figure 9, the storage system 90 includes a processor 900 and at least one storage medium, wherein each storage medium, such as an SSD, HDD, or disk, is capable of persistently storing data. The storage system 90 can also communicate with the application 200, such as receiving audio provided by the application 200 and sending notification messages to the application 200 (such as notifying the application 200 that the audio storage was successful).

[0231] Processor 900 is used to reduce the audio provided by application 200 and save the reduced audio data to a storage medium, thereby reducing the storage space required for audio storage. The specific implementation of processor 900 can be found in the relevant description of the implementation of processor 100 in Figure 1 above, and will not be repeated here.

[0232] Furthermore, the processor 900 may be configured with memory 1001, as shown in Figure 9, so that the processor 900 can use memory 1001 to store data and perform corresponding processing operations based on the data in memory 1001. Alternatively, the storage system 90 may also configure a separate memory 1002 for the processor 900, as shown in Figure 9, so that the data required or generated by the processor 900 to perform data processing operations can be saved through memory 1002.

[0233] Referring to Figure 10, a flowchart of an audio processing method is shown, which can be applied to the storage system 90 shown in Figure 9. As shown in Figure 10, the method may specifically include the following steps.

[0234] S1001: Application 200 sends the audio to be processed to storage system 90.

[0235] The audio sent by Application 200 can be in any format, such as Windows Media Audio (WMA), Waveform Audio File Format (WAV), Advanced Audio Coding (AAC), etc.

[0236] S1002: Processor 900 divides the audio into multiple raw audio segments.

[0237] In one possible implementation, since the audio sent by application 200 can be of any format, and different audio formats typically employ significantly different encoding algorithms and audio data organization methods, processor 900 can perform format conversion on the received audio to obtain a unified standard format audio. Subsequently, processor 900 can perform subsequent processing operations on the standard format audio. Specifically, processor 900 can decode the received audio to obtain audio data, and then encode the audio data into standard format audio using the encoding algorithm corresponding to that standard format.

[0238] Then, the processor 900 can divide the (standard format) audio into multiple segments according to a preset size. For ease of distinction and description, these segments are referred to below as original audio segments. For example, the processor 900 can divide the audio into multiple original audio segments containing the same number of audio frames according to the number of frames or duration, or divide the audio into multiple original audio segments corresponding to the same playback duration. In this embodiment, the method of dividing the audio is not limited.

[0239] S1003: The processor 900 compares each of the multiple original audio segments with at least one reference audio segment in the audio library to obtain the difference information corresponding to each of the multiple original audio segments. The difference information corresponding to each original audio segment is used to describe the audio difference between the original audio segment and a reference audio segment.

[0240] Correspondingly, the difference information corresponding to multiple original audio segments is used to describe the audio differences between the entire audio and the at least one reference audio segment.

[0241] In this embodiment, the processor 900 may be configured with an audio library, which may include one or more audio segments. For example, the audio library may be pre-configured in the processor 900 by a technician, or it may be created by the processor 900 based on audio received within a past time period. After dividing the audio into multiple original segments, for each original audio segment, the processor 900 may sequentially perform similarity calculations between the original audio segment and each audio segment in the audio library, such as calculating the similarity between two audio segments based on discrete cosine transform (DCT) technology. When an audio segment in the audio library is similar to the original audio segment (e.g., the similarity is greater than a threshold), the processor 900 may determine this audio segment as a reference audio segment so that it can be used subsequently to reduce the amount of audio data to be stored. Thus, the reference audio segment determined by the processor 900 satisfies the audio similarity condition with the original audio segment, such as the similarity between the two audio segments being greater than a threshold. When no audio segment in the audio library is similar to the original audio segment, the processor 900 can add the original audio segment to the audio library so that the added original audio segment can be used to retrieve data from other audio segments or other original audio segments later. In this embodiment, it is assumed that there is a reference audio segment corresponding to each original audio segment in the audio library.

[0242] Furthermore, when the audio library contains a large number of audio segments, each audio segment can be configured with a priority, which indicates the degree to which the audio segment is used in similarity calculation. Thus, when the processor 900 searches for audio segments (i.e., reference audio segments) similar to the original audio segment in the audio library, it can prioritize calculating the similarity between the original audio segment and the audio segment with the higher priority, based on the priority of each audio segment. In this way, by configuring priorities for the audio segments in the audio library, the efficiency of the processor 900 in determining the original audio segment and the reference audio segment can be improved, saving computational resources.

[0243] After determining the reference audio segment corresponding to the original audio segment, the processor 900 can compare the audio differences between the original audio segment and the reference audio segment to obtain difference information. This difference information may be, for example, the residual between the audio data included in the original audio segment and the audio data included in the reference audio segment. Thus, for each original audio segment, the processor 900 can calculate the difference information corresponding to each original audio segment in a similar manner.

[0244] S1004: The processor 900 encodes the difference information corresponding to each original audio segment to obtain the encoding results corresponding to multiple original audio segments respectively.

[0245] As an implementation example, the processor 900 may be configured with an encoding algorithm, and the processor 900 may use the configured encoding algorithm to encode the difference information corresponding to each original audio segment to obtain the corresponding encoding result. The encoding result corresponding to multiple original audio segments is also the encoding result of the audio.

[0246] In practical applications, since the original audio segments and the reference audio segments are similar, the difference information generated based on the original audio segments and the reference audio segments has a certain pattern. Therefore, the data volume of the encoding result generated by the processor 100 for the difference information corresponding to each original audio segment will be smaller than the data volume of the original audio segment. This can improve the audio reduction rate and reduce the storage space required for audio storage.

[0247] Furthermore, the processor 900 can be configured with multiple encoding algorithms. During the encoding of the difference information corresponding to the original audio segments, the processor 900 can first extract data features from the difference information. Based on these data features, it can determine one or more encoding algorithms from the configured set that match the data features, and then use these algorithms sequentially to encode the difference information. Sequentially encoding the difference information using multiple algorithms means that after the current encoding algorithm encodes the difference information to obtain an encoding result, the next encoding algorithm continues to encode that result, and so on, until the final encoding result is obtained. In this way, by matching the difference information with appropriate encoding algorithms, the audio reduction rate can be improved.

[0248] S1005: The processor 900 sends the encoding results corresponding to the multiple original audio segments to the storage medium for storage.

[0249] In practical applications, during the process of saving the encoding results, the processor 900 can store the encoding results, the identifier of the corresponding audio, and the mapping between the encoding results and reference audio segments in the audio library in the storage medium. Furthermore, the processor 900 can also send the audio library to the storage medium for persistent storage. Thus, when the application 200 requests audio from the storage system 100, the processor 900 can determine the encoding results corresponding to all the original audio segments included in the audio based on the identifier of the corresponding audio, and determine the reference audio segments in the audio library corresponding to each original audio segment in the audio based on the mapping. Therefore, the processor 900 can recover the original audio segments included in the audio based on the encoding results corresponding to each original audio segment and the reference audio segments corresponding to each original audio segment, thereby recovering the complete audio data. Furthermore, when saving the encoded data, the processor 900 can also save the audio metadata, which may include, for example, the encoding format of the audio sent by the application 200. In this way, after recovering the complete audio data, the processor 900 can encode the audio data into an audio file in the format required by the application 200 based on the saved metadata, so as to meet the audio requirements of the application 200.

[0250] In this embodiment, the processor 900 processes a single audio file as an example. When the application 200 provides the storage system 100 with a larger number of other audio files, the processor 900 can use an audio library to reduce the data size of the other audio files before storing them, in a similar manner as described above.

[0251] Furthermore, the processor 900 can be configured with multiple audio libraries. For different audio files sent by application 200, the processor 900 can first determine a suitable audio library for that audio file and then use the determined audio library to perform data reduction on the audio. Specifically, the processor 900 can extract audio features from the audio, such as the average energy or power spectrum of the audio, without limitation. Simultaneously, each audio library is configured with reference audio features, such as the average value of the audio features of each audio segment included in the audio library. Then, the processor 900 can calculate the similarity between the audio features of the audio file and the corresponding reference audio features of each audio library, and determine the audio library with the highest similarity as the audio library used for data reduction of the audio to be processed.

[0252] In this embodiment, the processor 900 uses a pre-created audio library to reduce the data of the currently received audio as an example. In other embodiments, the processor 900 may also create an audio library for the audio during the audio processing process. Specifically, when the playback duration of the audio is large (e.g., the playback duration reaches 1 hour), the processor 900 can first divide the audio to obtain multiple original audio segments (e.g., 3600 original audio segments), and construct an audio library based on a portion of these original audio segments (e.g., 600 original audio segments). The processor 900 can then use the original audio segments in this audio library to reduce the remaining original audio segments (e.g., the remaining 3000 original audio segments), thereby improving the overall reduction rate of the audio and reducing the storage space required for audio storage.

[0253] It is worth noting that other reasonable combinations of steps that can be conceived by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0254] The storage system, image processing method, and audio processing method provided in the embodiments of this application have been described above with reference to Figures 1 to 10. Next, the structure of the apparatus and computing device provided in the embodiments of this application will be described with reference to the accompanying drawings.

[0255] Referring to Figure 11, a schematic diagram of an image processing apparatus is shown. The image processing apparatus 1100 includes:

[0256] The acquisition module 1101 is used to acquire multiple images to be processed;

[0257] Grouping module 1102 is used to group multiple images according to the pixel features of multiple images to obtain multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups. The multiple groups include a first group.

[0258] The encoding module 1103 is used to encode the image in the first group using the first pixel library corresponding to the first group, and obtain the image encoding result corresponding to the first group. The first pixel library includes pixels in some images in the first group. The image encoding result corresponding to the first group is used to recover the image in the first group with the first pixel library.

[0259] In one possible implementation, the multiple images to be processed include images of different formats, and the image processing device 1100 further includes:

[0260] The conversion module 1104 is used to convert the format of multiple images to obtain multiple images in a standard format;

[0261] The grouping module 1102 is specifically used to group multiple images in a standard format according to the pixel features of the multiple images in a standard format.

[0262] In one possible implementation, the pixel library corresponding to the first group also includes reference metadata;

[0263] The encoding module 1103 is also used to encode the metadata of the standard format image in the first group using the reference metadata, so as to obtain the metadata encoding result corresponding to the first group. The metadata encoding result corresponding to the first group is used to recover the metadata of the standard format image in the first group with the reference metadata.

[0264] In one possible implementation, the encoding module 1103 is used for:

[0265] The pixels in the image within the first group are compared with the pixels in the first pixel library to obtain pixel difference information. The pixel difference information is used to describe the difference between the pixels in the image within the first group and the pixels in the first pixel library.

[0266] The pixel difference information is encoded to obtain the image encoding result corresponding to the first group.

[0267] In one possible implementation, the first pixel library includes pixels in a plurality of reference pixel blocks, wherein the reference pixel blocks are pixel blocks in an image, and the pixel difference information is residual.

[0268] Encoding module 1103 is used for:

[0269] The image within the first group is divided into multiple original pixel blocks;

[0270] Based on the first pixel library, a reference pixel block is determined that satisfies the matching conditions with each of the original pixel blocks in the multiple original pixel blocks. The matching conditions include that the pixel similarity between the two pixel blocks is greater than a threshold.

[0271] Calculate the residual between the original pixel block that meets the matching conditions and the reference pixel block.

[0272] In one possible implementation, each of the plurality of reference pixel blocks is configured with a priority, the priority being used to indicate the degree to which a reference pixel block is used to determine whether it meets the matching conditions with the original pixel block;

[0273] The encoding module 1103 is used to determine, from the first pixel library, reference pixel blocks that satisfy the matching conditions of each of the multiple original pixel blocks according to the priority of each reference pixel block.

[0274] In one possible implementation, the plurality of original pixel blocks includes a first original pixel block;

[0275] Encoding module 1103 is used for:

[0276] Check if there exists a reference pixel block in the first pixel library that meets the matching conditions of the first original pixel block;

[0277] If there is no reference pixel block in the first pixel library that meets the matching conditions with the first original pixel block, the first original pixel block is added to the first pixel library as a reference pixel block.

[0278] In one possible implementation, the encoding module 1103 is used for:

[0279] Data features for extracting pixel difference information;

[0280] Based on the data characteristics, at least one encoding algorithm that matches the data characteristics is determined from a variety of encoding algorithms;

[0281] Encode the pixel difference information sequentially using at least one encoding algorithm.

[0282] Since the image processing device 1100 shown in FIG11 corresponds to the processor 100 in the embodiment shown in FIG2 above, and is used to execute the method executed by the processor 100 in the embodiment shown in FIG2 above, the specific implementation of the image processing device 1100 shown in FIG11 and its technical effects can be found in the relevant descriptions in the embodiment shown in FIG2 above, and will not be repeated here.

[0283] Referring to Figure 12, a schematic diagram of another image processing apparatus is shown. This image processing apparatus 1200 includes:

[0284] The acquisition module 1201 is used to acquire the first image to be processed;

[0285] The determining module 1202 is used to determine a first pixel library from multiple pixel libraries, wherein the pixels in the first pixel library and the pixels in the first image satisfy the similarity condition, and the pixels in different pixel libraries do not satisfy the similarity condition;

[0286] The comparison module 1203 is used to compare the first image with at least one set of reference pixels in the first pixel library to obtain difference information, which is used to describe the pixel differences between the first image and at least one set of reference pixels.

[0287] The encoding module 1204 is used to encode the difference information to obtain the encoding result corresponding to the first image. The encoding result is used to recover the first image with at least one set of reference pixels.

[0288] In one possible implementation, the pixels in the reference pixel set belong to at least one image, and the difference information is used to describe the pixel differences between the first image and an image in the first pixel library.

[0289] Alternatively, the reference pixel set includes pixels in a reference pixel block, which is a pixel block in an image, and the difference information is used to describe the pixel differences between each pixel block in the first image and a reference pixel block in the first pixel library.

[0290] In one possible implementation, the reference pixel set includes pixels in a reference pixel block, and the difference information is a residual.

[0291] Comparison module 1203 is used for:

[0292] The first image is divided into multiple original pixel blocks;

[0293] Determine reference pixel blocks from the first pixel library that satisfy matching conditions with each of the multiple original pixel blocks, the matching conditions including that the similarity between pixels in the two pixel blocks is greater than a threshold;

[0294] Calculate the residual between the pixels in the original pixel block that meet the matching conditions and the pixels in the reference pixel block.

[0295] In one possible implementation, each reference pixel block in the first pixel library is configured with a priority, which indicates the degree of priority of the reference pixel block in determining whether it meets the matching conditions with the original pixel block.

[0296] The determining module 1202 is used to determine, from the first pixel library, a reference pixel block that satisfies the matching condition with each of the multiple original pixel blocks, based on the priority of each reference pixel block.

[0297] In one possible implementation, the determining module 1202 is configured to:

[0298] Extract pixel features from the first image;

[0299] Based on the pixel features of the first image, a first pixel library is determined from multiple pixel libraries, wherein the reference pixel features of the first pixel library have the greatest similarity to the pixel features of the first image among the multiple pixel libraries.

[0300] In one possible implementation, the encoding module 1204 is used for:

[0301] Data features from which differences are extracted;

[0302] Based on the data characteristics, at least one encoding algorithm that matches the data characteristics is determined from a variety of encoding algorithms;

[0303] The difference information is encoded using at least one encoding algorithm.

[0304] In one possible implementation, the determining module 1202 is configured to:

[0305] The first image is converted to a standard format to obtain the first image.

[0306] For a first image in a standard format, a first pixel library is determined from multiple pixel libraries.

[0307] In one possible implementation, the difference information includes metadata difference information and pixel difference information, and the first pixel library also includes reference metadata;

[0308] Comparison module 1203 is used for:

[0309] The metadata of the first image in standard format is compared with the reference metadata in the first pixel library to obtain metadata difference information;

[0310] The first image in standard format is compared with at least one set of reference pixels in the first pixel library to obtain pixel difference information.

[0311] Since the image processing device 1200 shown in FIG12 corresponds to the processor 100 in the embodiment shown in FIG8 above, and is used to execute the method executed by the processor 100 in the embodiment shown in FIG8 above, the specific implementation of the image processing device 1200 shown in FIG12 and its technical effects can be found in the relevant descriptions in the embodiment shown in FIG8 above, and will not be repeated here.

[0312] Referring to Figure 13, a schematic diagram of another audio processing device is shown. This audio processing device 1300 includes:

[0313] Acquisition module 1301 is used to acquire the audio to be processed;

[0314] The segmentation module 1302 is used to segment the audio to obtain multiple original audio segments;

[0315] The comparison module 1303 is used to compare each audio segment in a plurality of original audio segments with at least one reference audio segment in an audio library to obtain difference information corresponding to the plurality of original audio segments. The at least one reference audio segment in the audio library includes a portion of the plurality of original audio segments. The difference information is used to describe the audio differences between the audio and the at least one reference audio segment.

[0316] The encoding module 1304 is used to encode the difference information to obtain the encoding result corresponding to the audio. The encoding result is used to reconstruct the audio segment with at least one reference audio.

[0317] Since the audio processing device 1300 shown in FIG13 corresponds to the processor 900 in the embodiment shown in FIG10 and is used to execute the method executed by the processor 900 in the embodiment shown in FIG10, the specific implementation of the audio processing device 1300 shown in FIG13 and its technical effects can be found in the relevant descriptions in the embodiment shown in FIG10, and will not be repeated here.

[0318] Figure 14 is a schematic diagram of the hardware structure of a computing device 1400 provided in this application. The computing device 1400 may, for example, implement the processor 100 in the embodiment shown in Figure 2 or Figure 8, or implement the processor 900 in the embodiment shown in Figure 10.

[0319] As shown in Figure 14, the computing device 1400 includes a processor 1401, a memory 1402, and a communication interface 1403. The processor 1401, memory 1402, and communication interface 1403 communicate via a bus 1404, or via wireless transmission or other means. The memory 1402 stores instructions, and the processor 1401 executes the instructions stored in the memory 1402. Furthermore, the computing device 1400 may also include a memory unit 1405, which is connected to the processor 1401, the storage medium 1402, and the communication interface 1403 via the bus 1404. The memory 1402 stores program code, and the processor 1401 can call the program code stored in the memory 1402 to perform the following operations:

[0320] Acquire multiple images to be processed;

[0321] Based on the pixel features of multiple images, the images are grouped to obtain multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups. The multiple groups include the first group.

[0322] Using the first pixel library corresponding to the first group, the images within the first group are encoded to obtain the image encoding result corresponding to the first group. The first pixel library includes pixels from some images within the first group. The image encoding result corresponding to the first group is used to recover the images within the first group with the first pixel library.

[0323] Alternatively, processor 1401 can call program code stored in memory 1402 to perform the following operations:

[0324] Obtain the first image to be processed;

[0325] A first pixel library is determined from multiple pixel libraries. Pixels in the first pixel library and pixels in the first image satisfy the similarity condition, while pixels in different pixel libraries do not satisfy the similarity condition.

[0326] The first image is compared with at least one set of reference pixels in the first pixel library to obtain difference information, which is used to describe the pixel differences between the first image and at least one set of reference pixels.

[0327] The difference information is encoded to obtain the encoding result corresponding to the first image. The encoding result is used to recover the first image with at least one set of reference pixels.

[0328] Alternatively, processor 1401 can call program code stored in memory 1402 to perform the following operations:

[0329] Get the audio to be processed;

[0330] The audio is divided into multiple original audio segments;

[0331] Each audio segment in the multiple original audio segments is compared with at least one reference audio segment in the audio library to obtain the difference information corresponding to the multiple original audio segments. The at least one reference audio segment in the audio library includes some of the original audio segments in the multiple original audio segments. The difference information is used to describe the audio differences between the audio and the at least one reference audio segment.

[0332] The difference information is encoded to obtain the corresponding encoding result of the audio. The encoding result is used to reconstruct the audio segment with at least one reference audio segment.

[0333] It should be understood that in this embodiment, the processor 1401 can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete device assemblies, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0334] The memory 1402 may include read-only memory and random access memory, and provides instructions and data to the processor 1401. The memory 1402 may also include non-volatile random access memory.

[0335] The memory 1402 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0336] The communication interface 1403 is used to communicate with other devices connected to the computing device 1400. The bus 1404 may include a data bus, as well as a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus 1404 in the figure.

[0337] It should be understood that the computing device 1400 according to the embodiments of this application may correspond to the processor 100 or processor 900 in the embodiments of this application, and may correspond to the method executed by the processor 100 or processor 900 in the methods shown in the above embodiments. The above and other operations and / or functions implemented by the computing device 1400 are respectively to implement the flow of the corresponding method executed by the processor 100 or processor 900 in the above method embodiments. For the sake of brevity, they will not be described in detail here.

[0338] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the aforementioned image processing method or audio processing method.

[0339] This application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application are generated.

[0340] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0341] The computer program product can be a software installation package. When it is necessary to use either of the aforementioned image processing method or audio processing method, the computer program product can be downloaded and executed on a computing device.

[0342] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0343] The terminology used in the above embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, “one or more” refers to one, two, or more; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship. In the embodiments of this application, “simultaneously” means within the same time period, including situations where they are at the same moment. The terms “first,” “second,” etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate, and this is merely a way of distinguishing objects with the same attributes in the embodiments of this application.

[0344] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0345] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

An image processing method, characterized in that, The method includes: Acquire multiple images to be processed; Based on the pixel features of the multiple images, the multiple images are grouped to obtain multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups. The multiple groups include a first group. Using the first pixel library corresponding to the first group, the images within the first group are encoded to obtain the image encoding result corresponding to the first group. The first pixel library includes pixels from some images within the first group. The image encoding result corresponding to the first group is used to recover the images within the first group with the first pixel library. The method according to claim 1, characterized in that, The multiple images to be processed include images of different formats, and the method further includes: The multiple images are converted to a standard format to obtain multiple images. The step of grouping the multiple images based on their pixel features includes: Based on the pixel features of the multiple images in the standard format, the multiple images in the standard format are grouped. The method according to claim 2, characterized in that, The pixel library corresponding to the first group also includes reference metadata, and the method further includes: Using the reference metadata, the metadata of the standard format images within the first group is encoded to obtain the metadata encoding result corresponding to the first group. The metadata encoding result corresponding to the first group is used to recover the metadata of the standard format images within the first group with the reference metadata. The method according to any one of claims 1 to 3, characterized in that, The step of encoding the image within the first group using the first pixel library corresponding to the first group includes: Pixels in the image within the first group are compared with pixels in the first pixel library to obtain pixel difference information, which is used to describe the difference between pixels in the image within the first group and pixels in the first pixel library; The pixel difference information is encoded to obtain the image encoding result corresponding to the first group. The method according to claim 4, characterized in that, The first pixel library includes pixels from multiple reference pixel blocks, where each reference pixel block is a pixel block in an image, and the pixel difference information is a residual. The step of comparing pixels in the image within the first group with pixels in the first pixel library to obtain pixel difference information includes: The image within the first group is divided to obtain multiple original pixel blocks; Based on the first pixel library, a reference pixel block is determined that satisfies the matching condition with each of the multiple original pixel blocks, wherein the matching condition includes that the pixel similarity between the two pixel blocks is greater than a threshold. Calculate the residual between the original pixel block and the reference pixel block that satisfy the matching condition. The method according to claim 5, characterized in that, Each of the plurality of reference pixel blocks is configured with a priority, which indicates the degree of priority of the reference pixel block in determining whether it meets the matching conditions with the original pixel block; The step of determining reference pixel blocks that satisfy matching conditions with each of the plurality of original pixel blocks includes: Based on the priority of each reference pixel block, a reference pixel block that satisfies the matching condition with each of the plurality of original pixel blocks is determined from the first pixel library. The method according to claim 5 or 6, characterized in that, The plurality of original pixel blocks includes a first original pixel block; The step of determining, based on the first pixel library, reference pixel blocks that satisfy matching conditions with each of the plurality of original pixel blocks includes: Check if there exists a reference pixel block in the first pixel library that satisfies the matching condition with the first original pixel block; If there is no reference pixel block in the first pixel library that satisfies the matching condition with the first original pixel block, the first original pixel block is added to the first pixel library as a reference pixel block. The method according to any one of claims 4 to 7, characterized in that, The encoding of the pixel difference information includes: Extract the data features of the pixel difference information; Based on the data characteristics, at least one encoding algorithm that matches the data characteristics is determined from a variety of encoding algorithms; The encoding operation for the pixel difference information is executed sequentially using the at least one encoding algorithm. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire multiple images to be processed; A grouping module is used to group the multiple images according to the pixel features of the multiple images to obtain multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups. The multiple groups include a first group. The encoding module is used to encode the image within the first group using the first pixel library corresponding to the first group, thereby obtaining the image encoding result corresponding to the first group. The first pixel library includes pixels from some images within the first group, and the image encoding result corresponding to the first group is used to reconstruct the image within the first group using the first pixel library. The apparatus according to claim 9 is characterized in that, The multiple images to be processed include images of different formats, and the device further includes: The conversion module is used to convert the multiple images into a format to obtain multiple images in a standard format; The grouping module is specifically used to group the multiple images in the standard format according to the pixel features of the multiple images in the standard format. The apparatus according to claim 10, characterized in that, The pixel library corresponding to the first group also includes reference metadata; The encoding module is further configured to use the reference metadata to encode the metadata of the standard format image within the first group, thereby obtaining the metadata encoding result corresponding to the first group. The metadata encoding result corresponding to the first group is used to recover the metadata of the standard format image within the first group with the reference metadata. The apparatus according to any one of claims 9 to 11, characterized in that, The encoding module is used for: Pixels in the image within the first group are compared with pixels in the first pixel library to obtain pixel difference information, which is used to describe the difference between pixels in the image within the first group and pixels in the first pixel library; The pixel difference information is encoded to obtain the image encoding result corresponding to the first group. The apparatus according to claim 12 is characterized in that, The first pixel library includes pixels from multiple reference pixel blocks, where each reference pixel block is a pixel block in an image, and the pixel difference information is a residual. The encoding module is used for: The image within the first group is divided to obtain multiple original pixel blocks; Based on the first pixel library, a reference pixel block is determined that satisfies the matching condition with each of the multiple original pixel blocks, wherein the matching condition includes that the pixel similarity between the two pixel blocks is greater than a threshold. Calculate the residual between the original pixel block and the reference pixel block that satisfy the matching condition. The apparatus according to claim 13 is characterized in that, Each of the plurality of reference pixel blocks is configured with a priority, which indicates the degree of priority of the reference pixel block in determining whether it meets the matching conditions with the original pixel block; The encoding module is used to determine, from the first pixel library, reference pixel blocks that satisfy matching conditions with each of the plurality of original pixel blocks according to the priority of each reference pixel block. The apparatus according to claim 13 or 14 is characterized in that, The plurality of original pixel blocks includes a first original pixel block; The encoding module is used for: Check if there exists a reference pixel block in the first pixel library that satisfies the matching condition with the first original pixel block; If there is no reference pixel block in the first pixel library that satisfies the matching condition with the first original pixel block, the first original pixel block is added to the first pixel library as a reference pixel block. The apparatus according to any one of claims 12 to 15 is characterized in that, The encoding module is used for: Extract the data features of the pixel difference information; Based on the data characteristics, at least one encoding algorithm that matches the data characteristics is determined from a variety of encoding algorithms; The encoding operation for the pixel difference information is executed sequentially using the at least one encoding algorithm. A computing device, characterized in that, Including the processor and memory; The processor is configured to execute instructions stored in the memory to cause the computing device to perform the steps of the method as described in any one of claims 1 to 8. A storage system, characterized in that, The storage system includes a processor and at least one storage medium, the processor being configured to perform the method as described in any one of claims 1 to 8, and the at least one storage medium being configured to persistently store image encoding results. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computing device, cause the computing device to perform the steps of the method as described in any one of claims 1 to 8. A computer program product containing instructions, characterized in that, When it is run on at least one computing device, it causes the at least one computing device to perform the steps of the method as described in any one of claims 1 to 8.

Images