Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides an image processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: cutting an image to be transmitted into a plurality of image blocks, and performing compression processing on each image block to obtain a plurality of compressed image blocks; determining a target image block in the plurality of compressed image blocks based on the feature similarity between each compressed image block and the to-be-transmitted image; quantizing the target image block, and sending the quantized target image block and other compressed image blocks except the target image block in the plurality of image blocks to a receiving end, the target image block subjected to quantization processing and other compressed image blocks except the target image block are used for indicating a receiving end to perform inverse quantization processing and inverse compression processing on the target image block subjected to quantization processing and other compressed image blocks except the target image block to obtain an image to be transmitted. According to the method, the image transmission speed can be improved, and the image quality loss is reduced.

Description

Technical Field

[0001] The present application relates to the field of computers, and particularly to an image processing method, apparatus, electronic device, storage medium, and program product. Background Art

[0002] A server is a huge and complex system, and there is a certain upper limit on the transmission rate of the internal data path of the server. In scenarios with high speed requirements, the transmission speed of the path in the server will be insufficient. At this time, other acceleration technologies are needed to accelerate the data transmission.

[0003] Especially in the field of image transmission, slow transmission speed will lead to slow image refresh speed. Therefore, the image can be processed before data transmission. However, only focusing on image processing to improve the transmission speed of image data may result in more loss of image quality, leading to lower clarity and restoration degree of the image. Therefore, image processing before image transmission needs to balance between the transmission speed of image data and the loss of image quality to ensure that the transmission speed of the image can be improved and the loss of image quality can be reduced. Summary of the Invention

[0004] The present application provides an image processing method, apparatus, electronic device, storage medium, and program product to solve the problems of low transmission speed and quality of images.

[0005] In a first aspect, the present application provides an image processing method, including:

[0006] Cutting a to-be-transmitted image into multiple image blocks, and respectively performing compression processing on each image block to obtain multiple compressed image blocks;

[0007] Based on the feature similarity between each compressed image block and the to-be-transmitted image, determining a target image block among the multiple compressed image blocks;

[0008] Performing quantization processing on the target image block, and sending the quantized target image block and the other compressed image blocks except the target image block among the multiple image blocks to a receiving end. The quantized target image block and the other compressed image blocks except the target image block are used to instruct the receiving end to perform inverse quantization processing and inverse compression processing on the quantized target image block and the other compressed image blocks except the target image block to obtain the to-be-transmitted image.

[0009] In an implementable manner, the respectively performing compression processing on each image block to obtain multiple compressed image blocks includes:

[0010] Determine the summation coefficients corresponding to each image block based on the image position of each image block in the image to be transmitted, and the summation coefficients corresponding to image blocks located at different image positions are different;

[0011] Determine the compressed image block corresponding to each image block based on the summation coefficient of each image block and the feature data in each image block.

[0012] In an implementable manner, the summation coefficients of each image block include sub - coefficients of different image blocks, and the multiple image blocks include a first image block; the determining the compressed image block corresponding to each image block based on the summation coefficient of each image block and the feature data in each image block includes:

[0013] Obtain the coefficient feature data of the first image block at different pixel positions, where the coefficient feature data of the first image block at different pixel positions includes the product of the feature data of all image blocks at different pixel positions and the sub - coefficient of the corresponding image block indicated by the first image block;

[0014] Perform a summation process on the coefficient feature data of the first image block at different pixel positions to obtain the summation feature data at different pixel positions;

[0015] Determine the ratio between the summation feature data at different pixel positions and a preset value as the compressed feature data of the first image block at different pixel positions, and determine the compressed feature data at different pixel positions as the feature data of the corresponding compressed image block of the first image block at different pixel positions, so as to determine the compressed image block of the first image block.

[0016] In an implementable manner, the determining the target image block among the multiple compressed image blocks based on the feature similarity between each compressed image block and the image block to be transmitted includes:

[0017] Extract the first feature data in the image to be transmitted, and extract the second feature data of each compressed image block respectively;

[0018] Calculate the feature similarity between the first feature data and the second feature data of each compressed image block;

[0019] Based on the numerical magnitude of the feature similarity of each compressed image block, determine the compressed image block corresponding to the feature similarity less than the preset threshold as the target image block.

[0020] In an implementable manner, the quantizing the target image block includes:

[0021] Extract the target pixel values of each pixel point of the target image block;

[0022] Reduce the target pixel value of each pixel point to obtain the mapped pixel value of each pixel point;

[0023] Determine the pixel value of the corresponding pixel point in the target image block for quantization processing as the mapped pixel value of each pixel point.

[0024] In one implementable manner, the reducing the target pixel value of each pixel point to obtain the mapped pixel value of each pixel point includes:

[0025] Determine the maximum pixel value and the minimum pixel value in the target pixel value;

[0026] Determine the first difference between the maximum pixel value and the minimum pixel value, and determine the second difference between the target pixel value of each pixel point and the minimum pixel value, and determine the ratio between the second difference and the first difference as the mapped pixel value of each pixel point.

[0027] In a second aspect, the present application provides an image processing apparatus, including:

[0028] A compression module, configured to cut the image to be transmitted into a plurality of image blocks, and perform compression processing on each image block respectively to obtain a plurality of compressed image blocks;

[0029] A target image acquisition module, configured to determine a target image block among the plurality of compressed image blocks based on the feature similarity between each compressed image block and the image to be transmitted;

[0030] A quantization module, configured to perform quantization processing on the target image block, and send the target image block after quantization processing and other compressed image blocks except the target image block among the plurality of image blocks to a receiving end, where the target image block after quantization processing and the other compressed image blocks except the target image block are used to instruct the receiving end to perform inverse quantization processing and inverse compression processing on the target image block after quantization processing and the other compressed image blocks except the target image block to obtain the image to be transmitted.

[0031] In a third aspect, the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0032] The memory stores computer-executable instructions;

[0033] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in the first aspect.

[0035] In a fifth aspect, the present application provides a computer program product, including a computer program which, when executed by a processor, implements the method described in the first aspect.

[0036] The image processing method, apparatus, electronic device, storage medium and program product provided by the present application cut the image to be transmitted, compress the obtained image blocks to reduce the volume of the image blocks, and then perform further quantization processing on the compressed image to further reduce the volume of the image, thereby improving the transmission speed of the image. At the same time, during quantization, by compressing the feature similarity between the image blocks and the image to be transmitted, the target image blocks are determined. The target image blocks can be regarded as the parts with less content in the image to be transmitted, and the other compressed image blocks except the target image blocks are the parts with more or more important content in the image to be transmitted. Quantization processing is performed on the target image blocks, and quantization processing is not performed on the other compressed image blocks except the target image blocks, so as to ensure that there is no data quality loss caused by quantization processing in the parts with more or more important content in the image to be transmitted, and reduce the image quality loss on the premise of improving the transmission speed of the image data. Description of the Drawings

[0037] The drawings here are incorporated into the specification and constitute a part of this specification, showing the embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0038] Figure 1 is a diagram of an implementation scenario shown in an exemplary embodiment;

[0039] Figure 2 is a flowchart of an image processing method shown in an exemplary embodiment;

[0040] Figure 3 is a flowchart of the processing of an image to be transmitted shown in an exemplary embodiment;

[0041] Figure 4 is a flowchart of an image processing method shown in another exemplary embodiment;

[0042] Figure 5 is a flowchart of an image processing method shown in another exemplary embodiment;

[0043] Figure 6 is a structural diagram of an image processing apparatus shown in an exemplary embodiment;

[0044] Figure 7 is a block diagram of an electronic device shown in an exemplary embodiment.

[0045] Through the above-mentioned accompanying drawings, specific embodiments of the present application have been shown, and will be described in more detail hereinafter. These drawings and the textual description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by reference to specific embodiments. Detailed Description of Specific Embodiments

[0046] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0047] There are various communication protocols for data transmission in a server system, such as I2C (Inter-Integrated Circuit) and SPI (Serial Peripheral Interface). Among them, the SPI bus protocol is a high-speed, full-duplex, synchronous communication protocol, which has wide applications in scenarios such as communication between a microcontroller and a sensor, memory interface, data transmission, and network communication. However, due to protocol limitations during data transmission, there is a certain upper limit to the transmission rate of the SPI data path inside the server. In scenarios with high speed requirements, the transmission speed of the SPI path in the server is insufficient, and other acceleration technologies are needed to accelerate data transmission at this time.

[0048] Image data can be transmitted through the SPI path. There is often a trade-off between the transmission speed of image data and the loss of image quality. Slow transmission speed will lead to slow image refresh speed, while more loss of image quality will result in lower clarity and restoration degree of the image, both of which will bring a poor experience to users.

[0049] For SPI FLASH (flash memory), full-duplex is not commonly used, and data transmission is often one-way. Therefore, many of the data returned by the slave are invalid data. Thus, the usage of MOSI (Master Out Slave In) I is extended, and it can work in half-duplex to double data transmission.

[0050] For a DUAL (dual-line) SPI FLASH, a command byte can be sent to enter DUAL MODE, so that MOSI becomes SIO0 (SERIAL IO 0, serial input / output line 0), MOSO becomes SIO1 (SERIAL IO 1, serial input / output line 1), and two bits of data are transmitted in one cycle, doubling the data transmission.

[0051] The above method of improving the SPI image transmission speed only improves the performance from the hardware configuration, and does not combine with other software-level acceleration technologies to produce a better acceleration effect, and cannot meet the increasingly rich image application scenarios. For example, when designing a video projection screen or an image conversion interface, if the image data transmission speed is too slow, it may bring a poor experience when the picture is displayed.

[0052] The image processing method, device, electronic device, storage medium and program product provided by this application aim to solve the above technical problems in the prior art, can transmit image data for the SPI bus protocol, and greatly improve the data transmission speed on the basis of losing a small part of the image quality, and can improve the transmission speed and stability of the data path of the server.

[0053] The following uses specific embodiments to detail the technical solutions of this application and how the technical solutions of this application solve the above technical problems. These several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the drawings.

[0054] It should be understood that this application does not restrictively divide which parts of the image processing system are specifically deployed in what environment. In actual application, it can be adaptively deployed according to the computing power of the electronic device, the resource occupancy of the edge environment and the cloud environment, or specific application requirements.

[0055] As Figure 1 is an implementation scenario diagram shown in an exemplary embodiment. The corresponding image processing system of this implementation scenario includes a memory, a processor, a controller, and a receiving end, and the memory, the processor, the controller, and the receiving end are all connected through a communication bus.

[0056] The execution subject of the method in the embodiment of this application is the processor. The memory stores the image to be transmitted and sends the image to be transmitted to the processor, and the processor performs image processing.

[0057] The controller sends the data processed by the processor to the receiving end, and the receiving end processes the received image to restore the image to be transmitted.

[0058] In some embodiments, the memory may be structured as SRAM (Static Random-Access Memory), which stores data of the image to be transmitted and transmits the image to be transmitted to the processor on it.

[0059] The processor may be a device with data processing capabilities such as a CPU (Central Processing Unit). In some embodiments, the processor cuts the image to be transmitted into multiple image blocks, and performs compression processing on each image block respectively to obtain multiple compressed image blocks; based on the feature similarity between each compressed image block and the image to be transmitted, a target image block is determined from the multiple compressed image blocks; the target image block is quantized, and the quantized target image block and the other compressed image blocks except the target image block among the multiple image blocks are sent to the controller, and the controller sends the other compressed image blocks except the target image block to the receiving end.

[0060] In some embodiments, the image processing system may further include a flash memory. The controller sends the quantized target image block and the other compressed image blocks except the target image block among the multiple image blocks to the flash memory, and the flash memory sends them to the receiving end.

[0061] In some embodiments, the controller may be a device such as DMA (Direct Memory Access). Since there is a certain interval in each byte during SPI data transmission, SPI cannot achieve the theoretical speed performance. The quantized target image block and the other compressed image blocks except the target image block among the multiple image blocks are transmitted by the controller in cooperation with the SPI protocol bus. The configuration process of the controller is to set the destination address, set the flash memory address, set the transmission data length, set the configuration of the transmission channel, and enable the controller. When sending, a DMA request is issued each time the enabled TX (transmission) is set to "1", and the controller writes data to the flash memory. When the data transmission is completed, the request flag is cleared and becomes 0, thus completing the interruption, improving the stability during data transmission.

[0062] In some embodiments, the flash memory receives the quantized target image block and the other compressed image blocks except the target image block among the multiple image blocks, and transmits the data to the receiving end. For Dual SPI Flash, a command byte can be sent to enter the Dual Mode, so that MOSI becomes SIO0 and MOSI becomes SIO1. Although the data transfer becomes unidirectional in this way, 2 bits of data can be transmitted within one clock cycle, doubling the data transfer, thereby improving the transmission efficiency of the quantized target image block and the other compressed image blocks except the target image block among the multiple image blocks.

[0063] In some embodiments, the receiving end is a device in the image processor system for restoring the image to be transmitted. The receiving end can be the device corresponding to the target address or a device only for restoring the image to be transmitted. When the receiving end is a device only for restoring the image to be transmitted, it can also send the restored image to be transmitted to the device corresponding to the destination address, that is, the destination device where the image to be transmitted needs to arrive during transmission.

[0064] In some embodiments, the receiving end receives the quantized target image block and other compressed image blocks except the target image block among the multiple image blocks, and performs inverse quantization processing and inverse compression processing on the quantized target image block and other compressed image blocks except the target image block to obtain the image to be transmitted.

[0065] In some other embodiments, the receiving end can send the image to be transmitted to the destination device. The destination device can be within the same device as the receiving end, memory, processor, and controller, or can be a peripheral device not within the same device as the memory, processor, controller, and receiving end.

[0066] The receiving end can be a CPLD (Complex Programmable Logic Device), or a device provided with a CPLD, and specific limitations are not made here.

[0067] Figure 2 is a flowchart of an image processing method shown in an exemplary embodiment, applied to Figure 1 the processor in, such as Figure 2 shown, the method includes steps S201 to S203, which are introduced in detail as follows:

[0068] S201. Cut the image to be transmitted into multiple image blocks, and perform compression processing on each image block respectively to obtain multiple compressed image blocks.

[0069] In some embodiments, the processor receives the image to be transmitted and performs cutting processing on the image to be transmitted to obtain multiple image blocks.

[0070] The sizes of the multiple cut image blocks are the same.

[0071] In some embodiments, the processor evenly divides the image to be transmitted. As Figure 3 shown, the size of the image to be transmitted is 8*8, and the processor evenly divides it into 4 image blocks of 4*4 according to the position, that is, Figure 3 the a, b, c, and d image blocks in.

[0072] In some other embodiments, if the size of the image to be transmitted is too large, the image to be transmitted can also be evenly divided, and the image blocks obtained by the even division are evenly divided again until image blocks of a certain size are obtained.

[0073] In some embodiments, after obtaining a plurality of image blocks, the plurality of image blocks are compressed, so that each image block can obtain a corresponding compressed image block.

[0074] As Figure 3 shown, the image blocks a, b, c, and d are compressed to obtain the compressed image blocks A, B, C, and D.

[0075] In some embodiments, the compression process can be to compress the image blocks with any compression algorithm, such as DWT (Discrete Wavelet Transform), etc., and no specific limitation is made here.

[0076] In some embodiments, based on the image positions of the respective image blocks in the image to be transmitted, the summation coefficients corresponding to the respective image blocks are determined, and the summation coefficients corresponding to the image blocks located at different image positions are different; based on the summation coefficients of the respective image blocks and the feature data in the respective image blocks, the compressed image blocks corresponding to the respective image blocks are determined.

[0077] S202. Based on the feature similarity between each compressed image block and the image to be transmitted, a target image block is determined from among the plurality of compressed image blocks.

[0078] In some embodiments, a target image block can be determined, and this target image block is a compressed image block for subsequent quantization processing to further reduce the data volume.

[0079] In some embodiments, the feature similarity between each compressed image block and the image to be transmitted can characterize the richness of the feature data corresponding to the compressed image block. The higher the feature similarity, the higher the richness of the image content corresponding to the compressed image. To reduce the loss of the image quality of the image to be transmitted, for the other image blocks except the target image block among the plurality of compressed image blocks, no quantization processing is performed, and for the target image block, quantization processing is performed, so as to reduce the loss of the image quality on the premise of reducing the transmission volume of the image data and improving the transmission speed.

[0080] In some embodiments, the first feature data in the image to be transmitted is extracted, and the second feature data of each compressed image block is respectively extracted; the feature similarity between the first feature data and the second feature data of each compressed image block is calculated; based on the numerical magnitudes of the feature similarities of the respective compressed image blocks, the compressed image blocks corresponding to the feature similarities less than a preset threshold are determined as the target image blocks.

[0081] S203. Quantize the target image block, and send the quantized target image block and the other compressed image blocks in the multiple image blocks except the target image block to the receiving end. The quantized target image block and the other compressed image blocks except the target image block are used to instruct the receiving end to perform inverse quantization processing and inverse compression processing on the quantized target image block and the other compressed image blocks except the target image block to obtain the image to be transmitted.

[0082] In some embodiments, perform quantization processing on the target image block. For example, Figure 3 as shown in the figure, if the target image blocks are B, C, and D, then perform quantization processing on B, C, and D, and do not perform quantization processing on the compressed image block A in the gallery. At this time, the quantized B, C, and D and the unquantized A can be obtained.

[0083] In some embodiments, the quantization processing can be MinMax quantization (maximum-minimum quantization), or it can be uniform quantization, and specific limitations are not made here.

[0084] In some embodiments, extract the target pixel values of each pixel point of the target image block; map the target pixel values of each pixel point to a preset numerical range to obtain the mapped pixel values of each pixel point; determine the mapped pixel values of each pixel point as the pixel values of the corresponding pixel points in the quantized target image block, so as to obtain the quantized target image block.

[0085] In some embodiments, after obtaining the quantized target image block and the other compressed image blocks in the multiple image blocks except the target image block, send the quantized target image block and the other compressed image blocks in the multiple image blocks except the target image block to the receiving end. The receiving end performs inverse quantization processing on the quantized target image block to obtain the target image block, and performs inverse compression processing on the target image block obtained by the inverse quantization processing and the other compressed image blocks except the target image block, so as to restore the image to be transmitted.

[0086] It can be understood that the image to be transmitted obtained by the receiving end performing inverse quantization processing and inverse compression processing on the quantized target image block and the other compressed image blocks except the target image block is not the same as the original image to be transmitted before the processor cuts it into multiple image blocks. There may be image quality loss during the compression processing and quantization processing. The image to be transmitted obtained by the receiving end performing inverse quantization processing and inverse compression processing on the quantized target image block and the other compressed image blocks except the target image block is the image to be transmitted with image quality loss. Since the above compression and quantization methods can reduce the loss of image quality, the image to be transmitted obtained by the receiving end is close to the original image to be transmitted, and here it is regarded as the original image to be transmitted.

[0087] In the embodiments of the present application, the image to be transmitted is cut, and the obtained image blocks are compressed to reduce the volume of the image blocks. Then, the compressed image is further quantized to further reduce the volume of the image, thereby improving the transmission speed of the image. At the same time, during quantization, by compressing the feature similarity between the image blocks and the image to be transmitted, the target image blocks are determined. The target image blocks can be regarded as the parts with less content in the image to be transmitted. The other compressed image blocks except the target image blocks are the parts with more or more important content in the image to be transmitted. The target image blocks are quantized, and the other compressed image blocks except the target image blocks are not quantized, so as to ensure that there is no data quality loss caused by quantization in the parts with more or more important content in the image to be transmitted, and reduce the image quality loss on the premise of improving the transmission speed of the image data.

[0088] Figure 4 is a flowchart of an image processing method shown in another exemplary embodiment, which is applied to Figure 1 the processor in, such as Figure 4 shown, and this method includes step S401 to step S402. Figure 4 The steps in are Figure 2 a realizable manner of step S201 in, and the details are introduced as follows:

[0089] S401. Based on the image positions of the respective image blocks in the image to be transmitted, determine the summation coefficients corresponding to the respective image blocks. The summation coefficients corresponding to the image blocks located at different image positions are different.

[0090] In some embodiments, according to the image positions of the image blocks in the image to be transmitted, the summation coefficients corresponding to the corresponding image blocks can be determined, and the summation coefficients are used to calculate the compressed image blocks of the corresponding image blocks.

[0091] Such as Figure 3 the a, b, c, d image blocks in. Taking the image to be transmitted as a reference, it can be divided into 4 image positions of the image to be transmitted, namely the upper left corner, the upper right corner, the lower left corner, and the lower right corner. Among them, the image block a is located in the upper left corner and corresponds to its summation coefficient; the image block b is located in the upper right corner and corresponds to its summation coefficient; the image block c is located in the lower left corner and corresponds to its summation coefficient; the image block d is located in the lower right corner and corresponds to its summation coefficient.

[0092] In some embodiments, the summation coefficients of the respective image blocks include sub-coefficients of different image blocks. For example, for an image block, its summation coefficient includes the sub-coefficients of all image blocks. The sub-coefficients on a corresponding different image block can be the same or different.

[0093] For example, for image block a, its summation coefficients include the sub - coefficients of image blocks a, b, c, and d. For image block b, its summation coefficients include the sub - coefficients of image blocks a, b, c, and d. The sub - coefficient of image block a in the summation coefficients of image block a may be the same as or different from the sub - coefficient of image block a in the summation coefficients of image block b.

[0094] In some embodiments, the sub - coefficient is positive one or negative one. By setting the summation coefficients of each image block, the compressed image block corresponding to the corresponding image block is determined through the sub - coefficients in the summation coefficients.

[0095] In some embodiments, the summation coefficients are set by empirical parameters or determined according to a preset variation method, and no specific limitation is made here.

[0096] S402: Determine the compressed image block corresponding to each image block based on the summation coefficients of each image block and the feature data in each image block.

[0097] In some embodiments, through the summation coefficients and the feature data, the feature data of the compressed image block corresponding to the corresponding image block can be obtained, thereby determining the compressed image block.

[0098] In some embodiments, a method for determining a compressed image block from an image block is also proposed. Taking the first image block among multiple image blocks as an example, obtain the coefficient feature data of the first image block at different pixel positions. The coefficient feature data of the first image block at different pixel positions includes the product of the feature data of all image blocks at different pixel positions and the sub - coefficient of the corresponding image block indicated by the first image block; perform a summation process on the coefficient feature data of the first image block at different pixel positions to obtain the summation feature data at different pixel positions; determine the ratio between the summation feature data at different pixel positions and a preset value as the compressed feature data of the first image block at different pixel positions, and determine the compressed feature data at different pixel positions as the feature data of each pixel position of the compressed image block corresponding to the first image block, so as to determine the compressed image block of the first image block.

[0099] In one embodiment, the first image block is any image block, and the first image block corresponds to summation coefficients. The summation coefficients of the first image block include the sub - coefficients of all image blocks.

[0100] In some embodiments, determine the coefficient feature data of the first image block at different pixel positions. The coefficient feature data may be data such as pixel values, and no specific limitation is made here.

[0101] In some embodiments, for each image block, a corresponding coordinate system is constructed based on each image block. Generally speaking, the origin of this coordinate system is located at the same position in different image blocks, such as the vertex at the lower left corner or the upper left corner of different image blocks. Since each image block is a uniformly divided image block, that is, each image block has the same size and can be overlapped, the pixel positions can be determined in the same coordinate system with these completely overlapping multiple image blocks.

[0102] The pixel position is the position corresponding to the horizontal and vertical coordinates of each image block in the coordinate system.

[0103] For the first image block, the coefficient feature data at a certain pixel position includes the product of the feature data and the corresponding coefficients of all image blocks at the corresponding pixel position. For example, for the first image block a, the coefficient feature data of the first image block a at the pixel position (i, j) includes the product of the feature data of all image blocks at the pixel position (i, j) and the sub - coefficient of the corresponding image block indicated by the summation coefficient of the first image block a. As Figure 3 shown in the figure, for image blocks a, b, c, and d, the coefficient feature data of the first image block a at the pixel position (i, j) includes the product of the feature data of the first image block a at the pixel position (i, j) and the sub - coefficient of the first image block a in the summation coefficient of the first image block a, the product of the feature data of image block b at the pixel position (i, j) and the sub - coefficient of image block b in the summation coefficient of the first image block a, the product of the feature data of image block c at the pixel position (i, j) and the sub - coefficient of image block c in the summation coefficient of the first image block a, and the product of the feature data of image block d at the pixel position (i, j) and the sub - coefficient of image block d in the summation coefficient of the first image block a, where i is the abscissa of each image block in the coordinate system and j is the ordinate of each image block in the coordinate system.

[0104] In one embodiment, the sub - coefficient of image block a in the summation coefficient of image block a is “+1”, the sub - coefficient of image block b in the summation coefficient of image block a is “+1”, the sub - coefficient of image block c in the summation coefficient of image block a is “+1”, the sub - coefficient of image block d in the summation coefficient of image block a is “+1”, the sub - coefficient of image block a in the summation coefficient of image block b is “+1”, the sub - coefficient of image block b in the summation coefficient of image block b is “−1”, the sub - coefficient of image block c in the summation coefficient of image block b is “+1”, the sub - coefficient of image block d in the summation coefficient of image block b is “−1”, the sub - coefficient of image block a in the summation coefficient of image block c is “+1”, the sub - coefficient of image block b in the summation coefficient of image block c is “+1”, the sub - coefficient of image block c in the summation coefficient of image block c is “−1”, the sub - coefficient of image block d in the summation coefficient of image block c is “−1”, the sub - coefficient of image block a in the summation coefficient of image block d is “+1”, the sub - coefficient of image block b in the summation coefficient of image block d is “−1”, the sub - coefficient of image block c in the summation coefficient of image block d is “−1”, and the sub - coefficient of image block d in the summation coefficient of image block d is “+1”.

[0105] The characteristic coefficients of image block a include: aij, b ij , c ij , d ij , and the characteristic coefficients of image block b include: aij, -b ij , c ij , -d ij , the characteristic coefficients of image block c include: aij, b ij , -c ij , -d ij , and the characteristic coefficients of image block d include: aij, -b ij , c ij , d ij , where aij, b ij , c ij , d ij are the characteristic data of image blocks a, b, c, and d at pixel position (i, j) respectively.

[0106] In some embodiments, the compressed characteristic data at different pixel positions is determined as the characteristic data of the corresponding compressed image block of the first image block at different pixel positions to determine the compressed image block of the first image block.

[0107] In some embodiments, the preset value can be set by empirical parameters, such as values 2, 3, 4, etc.

[0108] In some embodiments, taking the preset value as 2 as an example, the compressed characteristic data of image blocks a, b, c, and d at pixel position (i, j) can be calculated as:

[0109]

[0110] Among them, A ij 、B ij 、C ij 、D ij are respectively the compression feature data of image blocks a, b, c, d at the pixel position (i, j), or are respectively the feature data of compressed image blocks A, B, C, D at the pixel position (i, j).

[0111] In the embodiments of the present application, by setting the summation coefficients of different image blocks, the compressed image blocks can be compressed through the summation coefficients, thereby reducing the data volume of the image blocks.

[0112] In some embodiments, the inverse compression process can also be performed on other compressed image blocks except the target image block through the sub - coefficients of different image blocks in the summation coefficients. For example, through the above - mentioned A ij 、B ij 、C ij 、D ij formula for derivation to perform the inverse compression process.

[0113] Figure 5 is a flowchart of an image processing method shown in another exemplary embodiment, which is applied to the processor in Figure 1 As shown in Figure 5 , this method includes steps S501 to S503. Figure 4 The steps in Figure 2 are a feasible implementation manner of step S202 in

[0114] S501. Extract the first feature data in the image to be transmitted, and respectively extract the second feature data of each compressed image block.

[0115] In some embodiments, the feature data of the image to be transmitted and the compressed image blocks can be respectively extracted through a pre - set feature extraction model (such as a neural network model, etc.), and the feature data is the feature data of the image.

[0116] S502. Calculate the feature similarity between the first feature data and the second feature data of each compressed image block.

[0117] In some embodiments, the feature similarity can be represented by the feature distance, that is, calculate the feature distance between the first feature data and the second feature data of each compressed image block. The feature distance can be calculated by means such as Euclidean distance, cosine similarity, Jaccard similarity, etc., and then the feature distance value is determined as the feature similarity. The closer the feature distance is, the higher the feature similarity is.

[0118] S503. Based on the numerical magnitudes of the feature similarities of the compressed image blocks, determine the compressed image blocks corresponding to the feature similarities less than a preset threshold as target image blocks.

[0119] In some embodiments, the greater the feature similarity, the more content or the more important part in the image block. If quantization is performed, it may cause quality loss of the image block and cannot be restored to the image to be transmitted. Therefore, determine the compressed image blocks corresponding to the feature similarities less than the preset threshold as target image blocks, and perform quantization processing on the target image blocks.

[0120] In the embodiments of the present application, through the feature similarity, only the target image blocks are quantized, reducing the image quality loss on the premise of improving the transmission speed of the image data.

[0121] In some embodiments, a quantization method is also proposed: extract the target pixel values of each pixel point of the target image block; perform a reduction process on the target pixel values of each pixel point to obtain the mapped pixel values of each pixel point; determine the mapped pixel values of each pixel point as the pixel values of the corresponding pixel points in the target image block after quantization processing.

[0122] In some embodiments, the target pixel values can be reduced by the maximum pixel value and the minimum pixel value.

[0123] In some embodiments, determine the maximum pixel value and the minimum pixel value in the target pixel values; determine the first difference between the maximum pixel value and the minimum pixel value, and determine the second difference between the target pixel value of each pixel point and the minimum pixel value, and determine the ratio between the second difference and the first difference as the mapped pixel value of each pixel point.

[0124] In some embodiments, by the maximum pixel value and the minimum pixel value, the target pixel values are reduced, thereby further reducing the data volume of the image data and improving the transmission rate of the image data.

[0125] Figure 6 It is a structural diagram of an image processing device shown in an exemplary embodiment. The image processing device 600 may include:

[0126] A compression module 610, configured to cut the image to be transmitted into multiple image blocks, and perform compression processing on each image block respectively to obtain multiple compressed image blocks;

[0127] A target image acquisition module 630, configured to determine target image blocks among the multiple compressed image blocks based on the feature similarities between the compressed image blocks and the image to be transmitted;

[0128] A quantization module 650 is configured to perform quantization processing on a target image block, and send the quantized target image block and other compressed image blocks in the multiple image blocks except the target image block to a receiving end. The quantized target image block and other compressed image blocks except the target image block are used to instruct the receiving end to perform inverse quantization processing and inverse compression processing on the quantized target image block and other compressed image blocks except the target image block, so as to obtain an image to be transmitted.

[0129] In an implementable manner, the compression module includes:

[0130] A summation coefficient determination unit is configured to determine summation coefficients corresponding to respective image blocks based on the image positions of the respective image blocks in the image to be transmitted, and the summation coefficients corresponding to the image blocks located at different image positions are different;

[0131] A compression unit is configured to determine compressed image blocks corresponding to respective image blocks based on the summation coefficients of the respective image blocks and the feature data in the respective image blocks.

[0132] In an implementable manner, the summation coefficients of the respective image blocks include sub - coefficients of different image blocks, and the multiple image blocks include a first image block; the compression unit includes:

[0133] A coefficient feature data acquisition section is configured to acquire coefficient feature data of the first image block at different pixel positions, and the coefficient feature data of the first image block at different pixel positions includes the product of the feature data of all image blocks at different pixel positions and the sub - coefficients of the corresponding image blocks indicated by the first image block;

[0134] A summation section is configured to perform summation processing on the coefficient feature data of the first image block at different pixel positions to obtain summation feature data at different pixel positions;

[0135] A compression section is configured to determine the ratio between the summation feature data at different pixel positions and a preset value as the compression feature data of the first image block at different pixel positions, and determine the compression feature data at different pixel positions as the feature data of the corresponding compressed image block of the first image block at different pixel positions, so as to determine the compressed image block of the first image block.

[0136] In an implementable manner, the target image acquisition module includes:

[0137] A feature extraction unit is configured to extract first feature data in the image to be transmitted, and extract second feature data of each compressed image block respectively;

[0138] A similarity calculation unit is configured to calculate the feature similarity between the first feature data and the second feature data of each compressed image block;

[0139] The target image acquisition unit is used to determine the compressed image block corresponding to the feature similarity less than a preset threshold as the target image block based on the numerical value of the feature similarity of each compressed image block.

[0140] In one possible implementation, the quantization module includes:

[0141] A pixel value acquisition unit, used to extract a target pixel value of each pixel point of a target image block;

[0142] A reduction unit, used for reducing the target pixel value of each pixel point to obtain a mapping pixel value of each pixel point;

[0143] The quantization unit is used to determine the mapped pixel value of each pixel point as the pixel value of the corresponding pixel point in the target image block of the quantization processing.

[0144] In one achievable manner, reducing the unit includes:

[0145] The maximum value determination section is used to determine the maximum pixel value and the minimum pixel value in the target pixel value;

[0146] The mapping plate is used to determine the first difference between the maximum pixel value and the minimum pixel value, and determine the second difference between the target pixel value and the minimum pixel value of each pixel point, and determine the ratio between the second difference and the first difference as the mapping pixel value of each pixel point.

[0147] The image processing device provided in this embodiment can be used to execute the above-mentioned image processing method. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.

[0148] Figure 7 is a block diagram of an electronic device shown in an exemplary embodiment, see Figure 7 The electronic device 700 may include: a processor 71 and a memory 72, wherein the processor 71 and the memory 72 may communicate; illustratively, the processor 71 and the memory 72 communicate via a communication bus 73, the memory 72 is used to store computer-executable instructions, and the processor 71 is used to call the computer-executable instructions in the memory to execute the image processing method shown in any of the above method embodiments.

[0149] The above-mentioned processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the present application may be directly embodied as being executed and completed by a hardware processor, or may be executed and completed by a combination of hardware and software modules in the processor.

[0150] The present application provides a computer-readable storage medium, on which computer-executable instructions are stored; when the computer-executable instructions are executed by a processor, they are used to implement the image processing method of any of the above embodiments.

[0151] An embodiment of the present application provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the above-mentioned image processing method is implemented.

[0152] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0153] Furthermore, it should be noted that although the various steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential either, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0154] It should be understood that the above device embodiments are merely illustrative, and the devices of the present application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0155] In addition, without special instructions, in each embodiment of the present application, each functional unit / module can be integrated in one unit / module, or each unit / module can exist physically alone, or two or more units / modules can be integrated together. The above integrated unit / module can be implemented in the form of hardware or in the form of a software program module.

[0156] When the integrated unit / module is implemented in the form of hardware, the hardware can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Without special instructions, the processor can be any suitable hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC, etc. Without special instructions, the storage unit can be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM (Resistive Random Access Memory), dynamic random access memory DRAM (Dynamic Random Access Memory), static random access memory SRAM (Static Random-Access Memory), enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), high-bandwidth memory HBM (High-Bandwidth Memory), hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0157] When an integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. And the aforementioned memory includes: various media such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0158] In the above embodiments, the descriptions of the various embodiments each have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope described in this specification.

[0159] Those skilled in the art will readily think of other implementation manners of this application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptive changes of this application, and these variations, uses, or adaptive changes follow the general principles of this application and include common general knowledge or conventional technical means in the technical field not disclosed in this application. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of this application are pointed out by the following claims.

[0160] It should be understood that this application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is only limited by the appended claims.

Claims

1. An image processing method, characterized in that: include: Cutting the image to be transmitted into a plurality of image blocks, and compressing each image block respectively to obtain a plurality of compressed image blocks; Determining a target image block from the plurality of compressed image blocks based on feature similarities between each compressed image block and the image to be transmitted; The target image block is quantized, and the quantized target image block and other compressed image blocks except the target image block among the multiple image blocks are sent to a receiving end, wherein the quantized target image block and the other compressed image blocks except the target image block are used to instruct the receiving end to perform inverse quantization and inverse compression on the quantized target image block and the other compressed image blocks except the target image block to obtain the image to be transmitted.

2. The method according to claim 1, characterized in that The step of compressing each image block to obtain a plurality of compressed image blocks comprises: Determining a summation coefficient corresponding to each image block based on an image position of each image block in the image to be transmitted, where image blocks at different image positions have different summation coefficients; Based on the sum coefficients of each image block and the characteristic data in each image block, the compressed image block corresponding to each image block is determined.

3. The method according to claim 2, characterized in that The summed coefficient of each image block includes sub-coefficients of different image blocks, and the multiple image blocks include a first image block; and determining the compressed image block corresponding to each image block based on the summed coefficient of each image block and feature data in each image block includes: Acquire coefficient feature data of the first image block at different pixel positions, where the coefficient feature data of the first image block at different pixel positions includes the product of feature data of all image blocks at different pixel positions and a sub-coefficient of a corresponding image block indicated by the first image block; Adding the coefficient feature data of the first image block at different pixel positions to obtain sum feature data at different pixel positions; The ratio between the summed feature data at different pixel positions and a preset value is determined as the compressed feature data at different pixel positions of the first image block, and the compressed feature data at different pixel positions are determined as the feature data of the compressed image blocks corresponding to the first image block at different pixel positions, so as to determine the compressed image block of the first image block.

4. The method according to claim 1, characterized in that: The step of determining a target image block from the plurality of compressed image blocks based on the feature similarity between each compressed image block and the image block to be transmitted comprises: Extracting first characteristic data from the image to be transmitted, and respectively extracting second characteristic data from each compressed image block; Calculating feature similarity between the first feature data and the second feature data of each compressed image block; Based on the numerical values ​​of the feature similarities of the compressed image blocks, the compressed image blocks corresponding to the feature similarities less than a preset threshold are determined as the target image blocks.

5. The method according to claim 1, characterized in that The quantizing process of the target image block comprises: Extracting target pixel values ​​of each pixel point of the target image block; The target pixel value of each pixel point is reduced to obtain the mapping pixel value of each pixel point; The mapped pixel value of each pixel point is determined as the pixel value of the corresponding pixel point in the target image block for quantization processing.

6. The method according to claim 5, characterized in that The step of reducing the target pixel value of each pixel point to obtain the mapped pixel value of each pixel point includes: Determining a maximum pixel value and a minimum pixel value among the target pixel values; A first difference between the maximum pixel value and the minimum pixel value is determined, and a second difference between the target pixel value of each pixel point and the minimum pixel value is determined, and a ratio between the second difference and the first difference is determined as a mapping pixel value of each pixel point.

7. An image processing device, characterized in that: include: A compression module, used for cutting the image to be transmitted into a plurality of image blocks, and performing compression processing on each image block respectively to obtain a plurality of compressed image blocks; A target image acquisition module, configured to determine a target image block from among the plurality of compressed image blocks based on feature similarities between each compressed image block and the image to be transmitted; A quantization module is used to perform quantization processing on the target image block, and send the quantized target image block and other compressed image blocks except the target image block among the multiple image blocks to a receiving end, wherein the quantized target image block and the other compressed image blocks except the target image block are used to instruct the receiving end to perform inverse quantization processing and inverse compression processing on the quantized target image block and the other compressed image blocks except the target image block to obtain the image to be transmitted.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images