Image processing method, apparatus, device, and medium

Abstract

The embodiment of the present disclosure provides an image processing method, device, equipment and medium, which relates to the technical field of image processing, and the method comprises the following steps: acquiring to-be-transmitted image data and reference image data; calculating the inter-frame deviation data of the to-be-transmitted image data and the reference image data; acquiring the first accumulated deviation data between the to-be-transmitted image data and the reference image data; in the case that the first accumulated deviation data is greater than or equal to a preset accumulated deviation threshold, updating the reference image data and resetting the first accumulated deviation data, and the preset accumulated deviation threshold is determined based on the inter-frame deviation data; in the case that the first accumulated deviation data is less than the preset accumulated deviation threshold, determining the difference data of the to-be-transmitted image data relative to the reference image data as the target image data to be transmitted. In this way, not only the amount of data to be transmitted is effectively reduced, but also the problems of accumulated deviation, picture drift and distortion accumulation caused by long-term use of fixed reference image data for inter-frame comparison are effectively solved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and medium. Background Technology

[0002] In the process of the Baseboard Management Controller (BMC) managing a remote server, the BMC often needs to transmit the screen images from the remote server back to the local machine for display. However, transmitting all screen images frame by frame would result in a large amount of data to be transmitted, leading to a rapid consumption of system resources. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, device, and medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this disclosure provide an image processing method, the method comprising: Acquire the image data to be transmitted and the reference image data; Calculate the inter-frame offset data between the image data to be transmitted and the reference image data; Obtain the first cumulative deviation data between the image data to be transmitted and the reference image data; If the first cumulative deviation data is greater than or equal to the preset cumulative deviation threshold, the reference image data is updated and the first cumulative deviation data is reset. The preset cumulative deviation threshold is determined based on the inter-frame deviation data. If the first cumulative deviation data is less than the preset cumulative deviation threshold, the difference data between the image data to be transmitted and the reference image data is determined, which is the target image data to be transmitted.

[0005] Secondly, embodiments of this disclosure provide an image processing method apparatus, the apparatus comprising: The acquisition module is used to acquire the image data to be transmitted and the reference image data; The calculation module is used to calculate the inter-frame deviation data between the image data to be transmitted and the reference image data; The acquisition module is also used to acquire the first cumulative deviation data between the image data to be transmitted and the reference image data; The update module is used to update the reference image data and reset the first cumulative deviation data when the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold. The preset cumulative deviation threshold is determined based on the inter-frame deviation data. The determination module is used to determine the difference data between the image data to be transmitted and the reference image data when the first cumulative deviation data is less than the preset cumulative deviation threshold, and to determine the target image data to be transmitted.

[0006] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image processing method of the first aspect.

[0007] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the image processing method according to the first aspect.

[0008] The image processing method provided in this disclosure can calculate the inter-frame deviation data between the acquired image data to be transmitted and the reference image data, obtain the first cumulative deviation data between the image data to be transmitted and the reference image data, and promptly update the reference image data and reset the corresponding first cumulative deviation data when the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold determined based on the inter-frame deviation data. This effectively solves the problems of cumulative deviation, image drift, and distortion accumulation caused by long-term use of fixed reference image data for inter-frame comparison in the prior art. Furthermore, when the first cumulative deviation data is less than the preset cumulative deviation threshold determined based on the inter-frame deviation data, the difference data between the image data to be transmitted and the reference image data can be determined as the target image data to be transmitted, effectively reducing the amount of data to be transmitted. This not only reduces the compression pressure on the compression engine but also reduces the number of DDR controller accesses, effectively mitigating frame loss during image transmission.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this disclosure; Figure 2 This is an example diagram of image block division provided in an embodiment of this disclosure; Figure 3 This is an example diagram of pixels in an image block provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of an image processing system provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure; Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0011] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0012] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0013] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0015] As described in the background section, during the process of a BMC transmitting images from a remote server back to the local machine for display, transmitting all images frame by frame would result in a large amount of data to be transmitted, potentially leading to rapid consumption of system resources. Specifically, a BMC often needs to compress the images from the remote server before sending them to the local server for display over the network. In this case, transmitting all images frame by frame would not only significantly increase the performance pressure on the compression engine, but also cause heavy access to the Double Data Rate Synchronous Dynamic Random Access Memory (DDR), potentially leading to frame drops during actual transmission.

[0016] Based on this, the present disclosure provides an image processing method to at least solve the technical problem in the prior art where the large amount of data to be transmitted leads to rapid consumption of system resources.

[0017] The image processing method provided in this disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this disclosure.

[0019] like Figure 1 As shown, the execution entity of this method can be a BMC. Based on this, the image processing method provided in this embodiment of the disclosure may specifically include the following steps: S110, acquire the image data to be transmitted and the reference image data.

[0020] The image data to be transmitted can be either full image data or partial image data extracted from the image frame to be transmitted. Similarly, the reference image data can be either full image data or partial image data extracted from the reference image frame. It should be noted that the reference image frame can be a historical image frame that has already been transmitted, while the image frame to be transmitted can be the current image frame to be transmitted. This will not be elaborated further here.

[0021] In addition, the reference image data is used to compare with the image data to be transmitted in order to obtain the difference data in which the image data to be transmitted changes significantly relative to the reference image data before transmission.

[0022] S120, calculate the inter-frame offset data between the image data to be transmitted and the reference image data.

[0023] The inter-frame deviation data here can be determined based on the comparison between the current image data to be transmitted and the reference image data. It is used to characterize the degree of change of the image data to be transmitted relative to the reference image data. That is, if the inter-frame deviation data is larger, it indicates that the degree of change of the image data to be transmitted relative to the reference image data is greater, and if the inter-frame deviation data is smaller, it indicates that the degree of change of the image data to be transmitted relative to the reference image data is smaller. No specific limitation is made here.

[0024] S130, acquire the first cumulative deviation data between the image data to be transmitted and the reference image data.

[0025] The first cumulative deviation data corresponding to the reference image data can be used to characterize the cumulative deviation data generated by the reference image frame in the historical comparison process and the current comparison process, without specific limitations here.

[0026] S140, if the first cumulative deviation data is greater than or equal to the preset cumulative deviation threshold, update the reference image data and reset the first cumulative deviation data.

[0027] The preset cumulative deviation threshold can be determined according to the actual situation. In some embodiments, the preset cumulative deviation threshold can be determined based on inter-frame deviation data, which is not specifically limited here.

[0028] Alternatively, updating the reference image data can refer to directly updating the current image data to be transmitted to the reference image data. Correspondingly, the first cumulative deviation data needs to be reset to 0. Furthermore, updating the reference image data can also refer to updating any interval image data between the image data to be transmitted and the reference image data to the new reference image data. Correspondingly, the first cumulative deviation data needs to be reset to the corresponding target value, which is obtained by sequentially comparing and accumulating the inter-frame deviation data between the new reference image data and the current image data to be transmitted for each interval.

[0029] S150, if the first cumulative deviation data is less than the preset cumulative deviation threshold, the difference data between the image data to be transmitted and the reference image data is determined as the target image data to be transmitted.

[0030] The aforementioned difference data may be relevant data showing image changes in the image data to be transmitted relative to the reference image data, and no specific limitation is made here.

[0031] Specifically, the system can acquire image data to be transmitted and reference image data, calculate the inter-frame deviation data between the two data, and acquire a first cumulative deviation data between them. Based on this, it can determine whether the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold determined based on the inter-frame deviation data. If the first cumulative deviation data is greater than or equal to the preset cumulative deviation threshold, it indicates that the image data to be transmitted has a significant change relative to the reference image data. In this case, the reference image data can be updated, and the corresponding first cumulative deviation data can be reset. Conversely, if the first cumulative deviation data is less than the preset cumulative deviation threshold, it indicates that the image data to be transmitted has a small change relative to the reference image data. In this case, the difference data between the image data to be transmitted and the reference image data can be determined as the target image data to be transmitted.

[0032] More specifically, after initializing the reference image data, the system iteratively acquires the image data to be transmitted. In each iteration, the inter-frame deviation data between the current image data to be transmitted and the current reference image data is calculated, and the first cumulative deviation data between the image data to be transmitted and the reference image data is acquired. If the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold, the current reference image data is updated to the current image data to be transmitted, and the first cumulative deviation data is reset, then the next iteration begins. If the first cumulative deviation data is less than the preset cumulative deviation threshold, the difference data between the current image data to be transmitted and the current reference image data is determined as the target image data to be transmitted, then the next iteration begins.

[0033] The image processing method provided in this disclosure can calculate the inter-frame deviation data between the acquired image data to be transmitted and the reference image data, and obtain the first cumulative deviation data between the image data to be transmitted and the reference image data. When the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold determined based on the inter-frame deviation data, the reference image data can be updated in a timely manner, and the corresponding first cumulative deviation data can be reset. This effectively solves the problems of cumulative deviation, image drift, and distortion accumulation caused by long-term use of fixed reference image data for inter-frame comparison in the prior art. Furthermore, when the first cumulative deviation data is less than the preset cumulative deviation threshold determined based on the inter-frame deviation data, the difference data between the image data to be transmitted and the reference image data can be determined as the target image data to be transmitted. This effectively reduces the amount of data to be transmitted, thereby reducing not only the compression pressure on the compression engine but also the number of DDR controller accesses, effectively mitigating frame loss during image transmission.

[0034] In order to accurately calculate the inter-frame deviation data between the image data to be transmitted and the reference image number, in one embodiment, the above-mentioned S120 may specifically include the following steps: Calculate the difference between the pixel value of each first pixel and the pixel value of the corresponding second pixel to obtain multiple pixel differences; Calculate inter-frame deviation data based on multiple pixel differences.

[0035] In some embodiments, the image data to be transmitted may include pixel values ​​corresponding to multiple first pixels, and correspondingly, the reference image data may include pixel values ​​corresponding to multiple second pixels, without specific limitations.

[0036] It should be noted here that the second pixel corresponding to the first pixel can refer to the two pixels having the same position in their respective image frames. That is, the pixel position of the first pixel in the image frame to be transmitted corresponding to the image data to be transmitted is the same as the pixel position of the second pixel in the reference image frame corresponding to the reference image data. No specific limitation is made here.

[0037] Specifically, since the image data to be transmitted may include pixel values ​​corresponding to multiple first pixels, and the reference image data may include pixel values ​​corresponding to multiple second pixels, each of the first pixels can be used as a calculation unit to calculate the pixel value of each first pixel and the pixel value of the corresponding second pixel, calculate the pixel difference between the two, and obtain multiple pixel differences. Based on these multiple pixel differences, the corresponding inter-frame deviation data can then be calculated.

[0038] In this embodiment, since the image data to be transmitted may include pixel values ​​corresponding to multiple first pixels, and the reference image data may include pixel values ​​corresponding to multiple second pixels, the pixel difference between each first pixel and its corresponding second pixel can be calculated on a pixel-by-pixel basis, and then inter-frame deviation data can be calculated based on multiple pixel differences. Thus, inter-frame deviation calculation can be performed at the pixel level, effectively improving the accuracy of the inter-frame deviation data.

[0039] Therefore, in order to accurately calculate the inter-frame deviation data between the image data to be transmitted and the reference image data based on the multiple pixel differences, in one embodiment, the step of calculating the inter-frame deviation data based on the multiple pixel differences may specifically include: The sum of multiple pixel differences is determined as the inter-frame deviation data; or, The mean square error of multiple pixel differences is determined as the inter-frame deviation data.

[0040] In one example, after calculating the pixel difference between each first pixel and its corresponding second pixel, it is possible to calculate the sum of multiple pixel differences and determine the sum of these multiple pixel differences as the inter-frame offset data between the image data to be transmitted and the reference image data.

[0041] In another example, after calculating the pixel difference between each first pixel and its corresponding second pixel, it is possible to calculate the mean square error between multiple pixel differences and determine the mean square error between these multiple pixel differences as the inter-frame deviation data between the image data to be transmitted and the reference image data.

[0042] In this embodiment, after calculating the pixel difference between each first pixel and its corresponding second pixel, the sum of these pixel differences or the mean square error of these pixel differences is determined as the inter-frame deviation data. This allows for inter-frame deviation calculation at the pixel level, effectively improving the accuracy of the inter-frame deviation data.

[0043] In another embodiment, the image data to be transmitted may include image data of multiple first image blocks, and the reference image data may include image data of second image blocks corresponding to the multiple first image blocks. It should be noted that the second image block corresponding to the first image block refers to the image block in the corresponding image frame being at the same position; that is, the image block position of the first image block in the image frame to be transmitted is the same as the image block position of the second image block in the reference image frame. Further details are omitted here.

[0044] Based on this, the above-mentioned S120 may specifically include the following steps: Based on the image data of each first image block and the image data of the corresponding second image block, calculate the image block deviation data between the first image block and the second image block; Select third image blocks from multiple first image blocks whose corresponding image block deviation data is greater than a preset image block deviation, and determine the number of third image blocks; The ratio of the number of image blocks in the third image block to the number of image blocks in multiple first image blocks is determined as the inter-frame offset data.

[0045] The image patch deviation data is used to characterize the degree of change of the first image patch relative to the second image patch corresponding to the first image patch. Specifically, the larger the image patch deviation data, the greater the degree of change between the two; the smaller the image patch deviation data, the smaller the degree of change between the two. This will not be elaborated further here.

[0046] In addition, the aforementioned preset image block deviation can be preset based on actual experience or circumstances. It is used to determine whether the degree of change between the first image block and its corresponding second image block is too large. No specific limitation is made here.

[0047] Specifically, since the image data to be transmitted may include image data of multiple first image blocks, and the reference image data may include image data of second image blocks corresponding to each of the multiple first image blocks, image block deviation data can be calculated based on the image data of each first image block and the image data of its corresponding second image block. Furthermore, from the multiple first image blocks, a third image block whose corresponding image block deviation data is greater than a preset image block deviation can be selected. This third image block is the first image block among the multiple first image blocks that exhibits a greater degree of change compared to its corresponding second image block. Then, the number of images in this third image block can be determined. Finally, the ratio of the number of images in this third image block to the number of images in the multiple first image blocks can be determined as the inter-frame deviation data.

[0048] It should be noted here that when calculating the image block deviation data between each first image block and its corresponding second image block, the pixel difference between the first pixel in the first image block and the second pixel in the corresponding second image block can be calculated using pixels as the unit. Then, the image block deviation data between the two can be determined based on the sum of multiple pixel differences. Alternatively, the image block deviation data between the two can be determined based on the mean square error of multiple pixel differences. This will not be elaborated on further here.

[0049] In this embodiment, since the image data to be transmitted may include image data of multiple first image blocks, and the reference image data may include image data of second image blocks corresponding to the multiple first image blocks, comparison can be performed on an image block-by-image basis. Third image blocks (i.e., blocks with significant variations) whose image block deviation data is greater than a preset image block deviation threshold can be selected, and the ratio of their number to the total number of first image blocks can be calculated to obtain the block mismatch rate as inter-frame deviation data. In this way, fine differences at the pixel level can be aggregated into block-level statistical features. Even if there are some minor noise points or slight changes in local pixels in the image frame, misjudgment will not occur, thereby effectively improving the accuracy of the inter-frame deviation data.

[0050] In order to provide a comprehensive and detailed description of the image processing method provided in the embodiments of this disclosure, in one embodiment, the step of obtaining the first cumulative deviation data between the image data to be transmitted and the reference image data may specifically include the following steps: Count the number of image frames between the image frame to be transmitted and the reference image frame; The number of image frames at intervals is determined as the first cumulative deviation data.

[0051] The image data to be transmitted can be determined based on the image frame to be transmitted, and the reference image data can be determined based on the reference image frame. The interval image frame can refer to the image frame between the transmitted image frame and the reference image frame, without specific limitation here.

[0052] Specifically, it is possible to count the number of image frames in the interval image frames between the image frames to be transmitted corresponding to the image data to be transmitted and the reference image frames corresponding to the reference image data, and use the number of image frames in the interval image frames as the first cumulative deviation data between the image data to be transmitted and the reference image data, thereby accurately determining the first cumulative deviation data between the image data to be transmitted and the reference image data.

[0053] In this embodiment, it is possible to consider that as the number of image frames in the interval between the image frame to be transmitted and the reference image frame gradually increases, the first cumulative deviation data between the two will also increase. Thus, by counting the number of image frames in the interval between the image frame to be transmitted and the reference image frame, the first cumulative deviation data between the image data to be transmitted and the number of reference images can be determined. This not only effectively reduces the computational load but also accurately determines the first cumulative deviation data between the image data to be transmitted and the number of reference images.

[0054] In another embodiment, the steps of acquiring the image data to be transmitted and the reference image data described above may specifically include the following steps: Obtain the second cumulative deviation data corresponding to the baseline image data; The inter-frame offset data is accumulated into the second cumulative offset data to obtain the first cumulative offset data after accumulation.

[0055] The second cumulative deviation data corresponding to the reference image data can be accumulated based on the inter-frame deviation data generated during the historical comparison process of the reference image.

[0056] Specifically, it is possible to obtain the first cumulative deviation data generated by the reference image data in the historical comparison process, and add the inter-frame deviation data generated in the current comparison process to the first cumulative deviation data, so as to accurately obtain the final first cumulative deviation data.

[0057] It should be noted that the inter-frame deviation data can be determined based on at least one of the following methods: the sum of pixel differences between each pixel in the two images (i.e., the image frame to be transmitted and the reference image frame), the mean square error of the pixel differences, or the block mismatch rate between the two images. Accordingly, the first cumulative deviation data obtained by accumulating the inter-frame deviation data can also be characterized using any of the above methods.

[0058] Since the foregoing embodiments require determining a preset cumulative deviation threshold based on inter-frame deviation data, in order to accurately determine the preset cumulative deviation threshold and facilitate timely updates of the reference image data or determination of difference data for transmission, in one embodiment, the image processing method provided by this disclosure may further include the following steps: If the inter-frame deviation data is less than the first deviation threshold, the preset cumulative deviation threshold is determined as the first cumulative deviation threshold. If the inter-frame deviation data is greater than or equal to the first deviation threshold and the inter-frame deviation data is less than the second deviation threshold, the preset cumulative deviation threshold is determined as the second cumulative deviation threshold. If the inter-frame deviation data is greater than the second deviation threshold, the preset cumulative deviation threshold is determined as the third cumulative deviation threshold.

[0059] In some embodiments, the first deviation threshold is less than the second deviation threshold. Additionally, the first cumulative deviation threshold is greater than the second cumulative deviation threshold, and the second cumulative deviation threshold is greater than the third cumulative deviation threshold. It should be noted that the specific values ​​of the first deviation threshold, the second deviation threshold, the first cumulative deviation threshold, the second cumulative deviation threshold, and the third cumulative deviation threshold can be determined according to the actual situation, and are not specifically limited here.

[0060] Thus, if the inter-frame deviation data is less than the first deviation threshold, it means that the change of the image data to be transmitted relative to the reference image data is very small, almost static. Here, the preset cumulative deviation threshold can be determined as a larger first cumulative deviation threshold.

[0061] If the inter-frame deviation data is greater than or equal to the first deviation threshold and less than the second deviation threshold, it indicates that the image data to be transmitted has a small change relative to the reference image data, such as cursor movement, text scrolling, and interface gradation. Here, the preset cumulative deviation threshold can be determined as the second cumulative deviation threshold, which is less than the first cumulative deviation threshold.

[0062] If the inter-frame deviation data is greater than the second deviation threshold, it indicates that the transmitted image data has changed significantly relative to the reference image data. For example, changes such as interface switching, full-screen filtering, and window pop-ups may occur. Here, the preset cumulative deviation threshold can be determined to be a third cumulative deviation threshold that is less than the second cumulative deviation threshold.

[0063] To accurately and comprehensively describe the image processing method provided in this embodiment, in one example, if the calculated inter-frame deviation data is less than a first deviation threshold, it indicates that the change in the image data to be transmitted relative to the reference image data is minimal. In this case, the number of image frames in the interval between the image frame to be processed and the reference image frame can be counted, and it can be determined whether the number of image frames in the interval is greater than or equal to a first cumulative deviation threshold. The first cumulative deviation threshold can be preset to 60 frames depending on the actual situation, and is not specifically limited here. If the number of image frames is greater than or equal to the first cumulative deviation threshold, the reference image data is updated and the first cumulative deviation data is reset. If the number of image frames is less than the first cumulative deviation threshold, the difference data between the image data to be processed and the reference image data is determined as the target image data to be transmitted.

[0064] If the calculated inter-frame deviation data is greater than or equal to the first deviation threshold and less than the second deviation threshold, it indicates that the image data to be transmitted has a small variation relative to the reference image data. In this case, the second cumulative deviation data corresponding to the reference image data can be obtained, and the current inter-frame deviation data is added to the second cumulative deviation data to obtain the accumulated first cumulative deviation data. Next, it is determined whether the accumulated first cumulative deviation data is greater than or equal to the second cumulative deviation threshold. If the accumulated first cumulative deviation data is greater than or equal to the second cumulative deviation threshold, the reference image data is updated and the first cumulative deviation data is reset. If the accumulated first cumulative deviation data is less than the second cumulative deviation threshold, the difference data between the image data to be processed and the reference image data is determined as the target image data to be transmitted.

[0065] If the inter-frame deviation data is greater than or equal to the second deviation threshold, it indicates that the image data to be transmitted has a significant change relative to the reference image data. In this case, the reference image data can be directly updated and the first cumulative deviation data can be reset without judging the cumulative deviation. It can be regarded as the third cumulative deviation threshold being set to 0.

[0066] In this embodiment, a progressively decreasing cumulative deviation threshold can be determined based on the gradual increase of inter-frame deviation data. This allows for accurate assessment of whether the degree of change between the image data to be transmitted and the reference image data is excessive, even when inter-frame deviation data exists. This enables timely updates to the reference image data and reset of the corresponding first cumulative deviation data when the difference between the two is significant. This solves the problems of cumulative deviation, image drift, and distortion accumulation caused by long-term use of fixed reference image data for inter-frame comparison in existing technologies. Furthermore, when the difference between the two is small, the difference between the image data to be transmitted and the reference image data can be determined as the target image data to be transmitted, effectively reducing the amount of data to be transmitted. This not only reduces the compression pressure on the compression engine but also reduces the number of DDR controller accesses, effectively mitigating frame loss during image transmission.

[0067] To comprehensively and thoroughly describe the image processing method provided in the embodiments of this disclosure, in one embodiment, the image data to be transmitted may include image data of multiple first image data blocks, and the reference image data may include image data of second image blocks corresponding to the multiple first image blocks. Based on this, the step of determining the difference data between the image data to be transmitted and the reference image data as the target image data to be transmitted includes: Based on the image data of each first image block and the image data of the corresponding second image block, calculate the image block correlation between the first image block and the second image block; From multiple first image blocks, select target image blocks whose image block correlation between the first image block and the second image block is less than a preset correlation threshold; The image data of the target image block is determined to be the target image data to be transmitted.

[0068] The image patch correlation between the first image patch and the second image patch can be used to characterize the correlation or similarity between the two image patches. Specifically, a higher correlation indicates a greater similarity between the first and second image patches, and a smaller degree of variation between them; conversely, a lower correlation indicates a smaller similarity between the first and second image patches, and a greater degree of variation between them. No further limitations are imposed here. The aforementioned preset image patch deviation threshold can be determined according to the actual situation; it is used to determine whether the image correlation between the first and second image patches is too high, and no specific limitations are imposed here.

[0069] Furthermore, when calculating the image correlation between the first image block and the second image block, a preset correlation algorithm can be used to further process the image data of the first and second image blocks to obtain their image block correlation. Alternatively, image block deviation data between the first and second image blocks can be calculated. The specific calculation method can be referred to the aforementioned embodiments and is not specifically limited here. Since image block deviation data and image block correlation are inversely correlated, the image block correlation between the two can be determined based on their image block deviation data, which is also not specifically limited here.

[0070] Specifically, since the image data to be transmitted may include image data of each of the multiple first image blocks, and the reference image data may include image data of the second image blocks corresponding to the multiple first image blocks, the image block correlation between the first image block and the second image block can be calculated based on the image data of each first image block and the image data of the corresponding second image block. Then, from the multiple first image blocks, target image blocks with an image block correlation between the first image block and the second image block less than a preset correlation threshold can be selected. That is, the image data of the target image block is the difference data between the image data to be transmitted and the reference image data, and thus the image data of the target image block can be determined as the target image data to be transmitted.

[0071] In this embodiment, the correlation between the image data of each first image block and the image data of its corresponding second image block in the image data to be transmitted can be calculated. Then, the image data of the target image block in which the correlation between the image blocks of the multiple first image blocks is less than a preset correlation threshold can be used as the target image data to be transmitted, which effectively reduces the amount of data to be transmitted. This not only reduces the compression pressure of the compression engine, but also reduces the number of accesses of the DDR controller, effectively alleviating the frame loss during image transmission.

[0072] In order to provide a comprehensive and detailed description of the image processing method provided in the embodiments of this disclosure, in one embodiment, the image processing method provided in the embodiments of this disclosure may further include the following steps: Acquire the image frame to be transmitted, the reference image frame, and block partitioning information, including row partitioning information and / or column partitioning information; The image frames to be transmitted are divided according to the block division information to obtain multiple first image blocks, and the image data of the first image blocks are extracted from the multiple first image blocks respectively; The baseline image frame is divided according to the block division information to obtain multiple second image blocks, and the image data of the second image blocks is extracted from the multiple second image blocks respectively.

[0073] In some embodiments, the block partitioning information described above may be user-preconfigured information used to partition the image frame to be transmitted and the reference image frame. This block partitioning information may include row partitioning information and / or column partitioning information, which are not specifically limited here. Furthermore, the block partitioning information may also include image resolution, which is not specifically limited here.

[0074] The row partitioning information can represent the number of image blocks in the row direction of the image frame, and correspondingly, the column partitioning information represents the number of image blocks in the column direction of the image frame. Alternatively, the row partitioning information can also represent the number of pixels in the row direction of the image block, and correspondingly, the column partitioning information represents the number of pixels in the column direction of the image block. This disclosure does not specifically limit this.

[0075] Specifically, it is possible to acquire the image frame to be transmitted, the reference image frame, and block partitioning information. Thus, the image frame to be transmitted can be partitioned according to the block partitioning information to obtain multiple first image blocks, and image data of each first image block can be extracted from these multiple first image blocks. Similarly, the reference image frame can be partitioned according to the block partitioning information to obtain multiple second image blocks, and image block data of each second image block can be extracted from these multiple second image blocks.

[0076] In one example, if the resolution of the acquired image frame (the image frame to be transmitted or the reference image frame) is 640*480, and the acquired block partitioning information can include row partitioning information and column partitioning information, where the row partitioning information indicates that the image frame has 48 image blocks in the row direction, and the column partitioning information indicates that the image frame has 64 image blocks in the column direction, then the image frame is partitioned according to this block partitioning information. A detailed partitioning diagram can be found in the following example... Figure 2 As shown, this allows for the generation of multiple image patches. Furthermore, the position of each image patch can be represented by at least one of the four vertex pixels of that image patch, for example, Figure 2 The pixel positions of the top-left pixel P1 and the bottom-right pixel P2 of the image block Dx,y can be used to represent the image block position of the image block Dx,y.

[0077] It should be noted here that when dividing an image frame into multiple image blocks, the image processing method provided in this disclosure does not specifically limit the number of pixels in each image block. For example, as Figure 3 As shown, the image block may include 4 pixels, which are represented by A, B, C and D respectively. Further details will not be provided here.

[0078] In this embodiment, the image frame to be transmitted, the reference image frame, and block partitioning information can be acquired. The image frame to be transmitted and the reference image frame are divided according to the block partitioning information to obtain multiple first image blocks and multiple second image blocks, and the image data of each image block is extracted. This facilitates subsequent comparison on an image block-by-block basis, thereby accurately determining the differences between the image data to be transmitted and the reference image data, and using these differences as the target image data for transmission.

[0079] In order to provide a comprehensive and detailed description of the image processing method provided in the embodiments of this disclosure, in one embodiment, the image processing method provided in the embodiments of this disclosure may further include the following steps: Obtain the position of the target image block in the image frame to be transmitted, corresponding to the image data to be transmitted; Compress the image data and image block location of the target image block to obtain compressed image data; Compressed image data is sent to the receiving end for decompression. Based on the image data of the target image block, the image block position, and the reference image data pre-transmitted to the receiving end, image restoration processing is performed to obtain the image data to be transmitted.

[0080] Specifically, after determining the image data of the target image block as the target image data to be transmitted, the image block position of the target image block in the image frame to be transmitted can be obtained, and the image data and image block position of the target image block can be compressed to obtain compressed image data. Then, the compressed image data can be sent to the receiving end for the receiving end to decompress the compressed image data, and image restoration processing can be performed based on the image data of the target image block, the image block position, and the reference image data pre-transmitted to the receiving end to obtain the image data to be transmitted.

[0081] In this embodiment, after determining the target image data to be transmitted, the image data and image block position of the target image block can be bound and compressed. The compressed image data obtained by combining the two is sent to the receiving end for decompression. Based on the image data, image block position, and reference image data pre-transmitted to the receiving end, image restoration processing is performed to obtain the image data to be transmitted. This process effectively reduces the amount of data to be transmitted, thereby reducing not only the compression pressure on the compression engine but also the number of DDR controller accesses, effectively mitigating frame loss during image transmission.

[0082] To provide a comprehensive and detailed description of the image processing method provided in the embodiments of this disclosure, the embodiments of this disclosure also provide an image processing system, which can be specifically combined with... Figure 4 The image processing system provided in the embodiments of this disclosure will be described in detail.

[0083] Figure 4 This is a schematic diagram of the structure of an image processing system provided in an embodiment of this disclosure.

[0084] like Figure 4 As shown, the image processing system 40 includes the following parts: (1) Image acquisition module 401 is used to acquire the image frame to be transmitted and extract the corresponding image data to be transmitted based on the frame header and frame tail of the image frame.

[0085] (2) The first cache module 402 is connected to the image acquisition module 401 and is used to cache the image data to be transmitted acquired by the image acquisition module 401 to smooth the speed difference between data inflow and outflow and avoid processing lag. Its storage format is: {column[M-1:m], row[N-1:n], rgb_data[r]}, where M-1 is the total number of columns of the image frame, N-1 is the total number of rows of the image frame, column[M-1:m] represents the m-th column, row[N-1:n] represents the n-th row, and rgb_data[r] represents the pixel value of the pixel at that position.

[0086] (3) The image calibration control module 403, connected to the image acquisition module 401, is used to calculate the inter-frame deviation data between the image data to be transmitted and the reference image data. Based on the deviation range of the inter-frame deviation data, and combined with the first cumulative deviation data corresponding to the reference image data, it decides whether to update the reference image data. The reference image frame corresponding to the reference image data can be the image frame to be transmitted in the historical time period. This module is also used to periodically refresh the reference image data to avoid excessive cumulative deviation due to the long-term use of the same reference. This solves the problems of cumulative deviation, image drift, and distortion accumulation caused by long-term use of a fixed reference image for inter-frame correlation comparison, and ensures that the BMC remote image is stably and accurately restored in the long term under low bandwidth.

[0087] (4) The second cache module 404 and the third cache module 405 are connected to the image calibration control module 403 respectively, forming a dual cache mechanism. One cache is used to store the current reference image data, and the other cache is used to pre-store the new reference image data to be switched (i.e., the image data to be transmitted). When the reference update condition is met (i.e., the first cumulative deviation data is greater than or equal to the preset cumulative deviation threshold), the cache is directly switched seamlessly to realize the smooth update of the reference image.

[0088] (5) Block configuration module 406 is used to configure block partitioning information. The block partitioning information includes image resolution, row partitioning information (how many rows of blocks an image is divided into, and the number of pixels in each row block) and column partitioning information (how many columns of blocks an image is divided into, and the number of pixels in each column block), so that correlation calculation can be performed on image blocks as units in the future.

[0089] (6) Block partitioning module 407, connected to block configuration module 406, is used to receive block partitioning information sent by block configuration module 406, and to partition the image frame to be transmitted and the reference image frame based on the block partitioning information to obtain multiple first image blocks and multiple second image blocks respectively.

[0090] (7) Block correlation calculation control module 408, which is connected to image calibration control module 403 and block partitioning module 407 respectively, is used for: Receive block partitioning information (including image resolution, number of blocks, and coordinates of each block); If the received data is reference image data, it is sent directly to the compression engine module 413; If the received data is not the reference image data, the reference image block and the image block to be transmitted are encapsulated in the format {block_num[n-1:0], tail_data, head_data, rgb_data_corr, rgb_data_base}, where block_num represents the image block number, head_data and tail_data identify the start and end positions of the image block, rgb_data_base is the image data of the reference image block (the second image block), and rgb_data_corr is the image data of the image block to be transmitted (i.e., the first image block). The encapsulated data is stored in the fifth cache module 412 and then sent to the correlation calculation module 409 for processing. The status information (including image resolution, number of blocks, coordinates of each block, and the image block being processed) is sent to the frame rate control module 415. After the last image block of the reference image frame and the image frame to be transmitted is processed, the block transmission information recording module 410 is requested to transmit the recording information, and the image data is sent together with the recording information to the compression engine module 413.

[0091] (8) Correlation calculation module 409, connected to block correlation calculation control module 408, is used for: Receive pixel data from the reference image block and the image block to be transmitted, and extract rgb_data_base, rgb_data_corr, head_data, tail_data, and block_num; The image block deviation data between two image blocks is calculated based on a preset correlation algorithm (such as the absolute difference algorithm, the mean square error algorithm, etc.). The image patch deviation data is compared with a preset threshold: if the image patch deviation data is less than the threshold, the correlation between the two image patches is marked as 0 (indicating a high correlation, and the image data of the reference image patch can be used instead); if the image patch deviation data is greater than or equal to the threshold, the correlation between the two image patches is marked as 1 (indicating a low correlation, and the image data of the image patch to be transmitted needs to be transmitted). The image block correlation and pixel coordinates (i.e., the image block position that determines the target image block to be transmitted) are sent to the block transmission information recording module 410.

[0092] (9) Block transmission information recording module 410, connected to block related calculation control module 408, is used for: Receive image patch correlation and pixel coordinates, and store the image patch correlation into the fourth cache module 411 according to the pixel coordinates; In response to the request from the block-related calculation control module 408, the image block transmission record information is read from the fourth cache module 411 and returned; The image block transmission information is used to record the transmission status of each image block in the entire image (i.e., which blocks need to be actually transmitted and which blocks can be replaced by the reference image block) and their storage address in the fourth cache module 411.

[0093] (10) The fourth cache module 411 is connected to the block correlation calculation control module 408 and is used to store the image block correlation generated by the block transmission information recording module 410, that is, the "transmit / not transmit" flag of each image block.

[0094] (11) The fifth cache module 412 is connected to the block correlation calculation control module 408 and is used to cache the image data of the encapsulated image blocks to be transmitted for the correlation calculation module 409 to read and process block by block.

[0095] (12) Compression engine module 413, connected to block-related calculation and control module 408, is used for: Receive the image data to be transmitted (including baseline image data and image block data with a correlation calculation result of 1); The received image data is compressed to reduce the data volume before being transmitted over the network, thereby reducing the pressure on network bandwidth and DDR controller access.

[0096] (13) DDR controller 414, connected to compression engine module 413, is used to control DDR read and write access. Since the DDR controller is used by multiple service modules simultaneously, its processing capacity is limited. When the access requests are too dense, image data may not be processed in time, resulting in frame dropping. This system monitors the busy status of the DDR controller through frame rate control module 415 and adjusts the frame dropping strategy accordingly.

[0097] (14) Frame rate control module 415, which is connected to compression engine module 413 and block correlation calculation control module 408 respectively, is used for: Receive status information (including image resolution, number of blocks, coordinates of each block, and image block being processed) sent by the block-related calculation control module 408; Monitor the busy / idle status of the compression engine module 413 and the DDR controller 414; Based on a comprehensive assessment of the processing pressure of the current image frames, when the system is busy, a frame dropping request is sent to the image acquisition module 401 to discard image data of some frames; The discarded frames still need to participate in the reference image update of the image calibration control module 403 (that is, only the reference is updated, and image data is not transmitted to the next frame) to prevent the image from suddenly changing. When the system is idle, more image frames are allowed to enter the subsequent processing flow.

[0098] (15) The first network module 416 is connected to the DDR controller 414 through the processor and is used to send the difference data between the image data to be transmitted and the reference image data (i.e. the image data processed by the compression engine module 413) to the remote recovery engine 41.

[0099] (16) Remote recovery engine 41, used to receive and recover image data sent from a remote end, including: The second network module 417 is connected to the first network module 416 and is used to receive image data sent by the first network module 416. The decompression engine module 418 is connected to the second network module 417 and is used to decompress the received compressed image data and restore the original image data. The image recovery engine 419, connected to the decompression engine module 418, is used to recover the complete image frame to be transmitted based on the received difference data and the locally cached reference image data.

[0100] In addition, the DDR controller 414 is connected to the DDR (Double Data Rate Synchronous Dynamic Random Access Memory) and is used to manage DDR read and write operations. The DDR serves as the system's main memory, storing various types of operational data, including image data to be processed, compressed image data, and data from other service modules. Since the DDR controller processes access requests from multiple services serially, excessive image data transmission can lead to DDR controller overload, resulting in dropped frames. This system monitors the DDR controller's status through the frame rate control module 415 and dynamically adjusts the frame dropping strategy to transmit as much valid image data as possible while ensuring overall system performance.

[0101] Based on the same inventive concept, this disclosure provides an image processing apparatus, which can be specifically described in conjunction with the appendix. Figure 5 The image processing apparatus provided in the embodiments of this disclosure will be described in detail.

[0102] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure.

[0103] like Figure 5 As shown, the image processing apparatus 500 may include: The acquisition module 510 is used to acquire the image data to be transmitted and the reference image data; Calculation module 520 is used to calculate the inter-frame deviation data between the image data to be transmitted and the reference image data; The acquisition module 510 is further configured to acquire first cumulative deviation data between the image data to be transmitted and the reference image data; The update module 530 is used to update the reference image data and reset the first cumulative deviation data when the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold. The preset cumulative deviation threshold is determined based on the inter-frame deviation data. The determining module 540 is used to determine the difference data between the image data to be transmitted and the reference image data when the first cumulative deviation data is less than the preset cumulative deviation threshold, and thus determine the target image data to be transmitted.

[0104] In one embodiment, the image processing apparatus provided in this disclosure may include: The calculation module is specifically used to calculate the difference between the pixel value of each first pixel and the pixel value of the second pixel corresponding to the first pixel, thereby obtaining multiple pixel differences; The calculation module is specifically used to calculate the inter-frame deviation data based on the multiple pixel differences; The image data to be transmitted includes pixel values ​​corresponding to multiple first pixel points, and the reference image data includes pixel values ​​corresponding to multiple second pixel points.

[0105] In one embodiment, the image processing apparatus provided in this disclosure may include: The determining module is further configured to determine the sum of the multiple pixel differences as the inter-frame deviation data; or, The determining module is further configured to determine the mean square error of the multiple pixel differences, which is the inter-frame deviation data.

[0106] In one embodiment, the image data to be transmitted includes image data of a plurality of first image blocks, and the reference image data includes image data of second image blocks corresponding to the plurality of first image blocks respectively. The image processing apparatus provided in this disclosure embodiment may include: The calculation module is specifically used to calculate the image block deviation data between the first image block and the second image block based on the image data of each first image block and the image data of the second image block corresponding to the first image block; The filtering module is used to filter out third image blocks from the plurality of first image blocks whose corresponding image block deviation data is greater than a preset image block deviation, and to determine the number of image blocks in the third image block; The determining module is further configured to determine the ratio of the number of image blocks in the third image block to the number of image blocks in the plurality of first image blocks, which is the inter-frame offset data.

[0107] In one embodiment, the image processing apparatus provided in this disclosure may include: The statistics module is used to count the number of image frames between the image frame to be transmitted and the reference image frame. The determining module is further configured to determine the number of image frames of the interval image frames, which is the first cumulative deviation data; or, The acquisition module is specifically used to acquire the second cumulative deviation data corresponding to the reference image data; The accumulation module is used to accumulate the inter-frame offset data to the second cumulative offset data to obtain the accumulated first cumulative offset data.

[0108] In one embodiment, the image processing apparatus provided in this disclosure may include: The determining device is further configured to determine the preset cumulative deviation threshold as the first cumulative deviation threshold when the inter-frame deviation data is less than the first deviation threshold. The determining device is further configured to determine the preset cumulative deviation threshold as the second cumulative deviation threshold when the inter-frame deviation data is greater than or equal to the first deviation threshold and the inter-frame deviation data is less than the second deviation threshold. The determining device is further configured to determine the preset cumulative deviation threshold as a third cumulative deviation threshold when the inter-frame deviation data is greater than the second deviation threshold. Wherein, the first deviation threshold is less than the second deviation threshold; the first cumulative deviation threshold is greater than the second cumulative deviation threshold, and the second cumulative deviation threshold is greater than the third cumulative deviation threshold.

[0109] In one embodiment, the image data to be transmitted includes image data of a plurality of first image blocks, and the reference image data includes image data of second image blocks corresponding to the plurality of first image blocks respectively; the image processing apparatus provided in this disclosure embodiment may include: The calculation module is also used to calculate the image block correlation between the first image block and the second image block based on the image data of each first image block and the image data of the second image block corresponding to the first image block; The filtering module is used to filter target image blocks from multiple first image blocks, where the image block correlation between the first image block and the second image block is less than a preset correlation threshold; The determining module is specifically used to determine that the image data of the target image block is the target image data to be transmitted.

[0110] In one embodiment, the image processing apparatus provided in this disclosure may include: The acquisition module is also used to acquire the image frame to be transmitted, the reference image frame, and block partitioning information, wherein the block partitioning information includes row partitioning information and / or column partitioning information; The segmentation module is used to segment the image frame to be transmitted according to the block segmentation information to obtain multiple first image blocks, and extract the image data of the first image blocks from the multiple first image blocks respectively; The segmentation module is further configured to segment the reference image frame according to the block segmentation information to obtain multiple second image blocks, and extract the image data of the second image blocks from the multiple second image blocks respectively.

[0111] In one embodiment, the image processing apparatus provided in this disclosure may include: The acquisition module is further configured to acquire the position of the target image block in the image frame to be transmitted corresponding to the image data to be transmitted; The compression module is used to compress the image data and image block position of the target image block to obtain compressed image data; The transmitting module is used to transmit the compressed image data to the receiving end so that the receiving end can decompress the compressed image data and perform image restoration processing based on the image data of the target image block, the position of the image block, and the reference image data pre-transmitted to the receiving end to obtain the image data to be transmitted.

[0112] It is understood that, when implementing the corresponding image processing method apparatus provided in the above embodiments, the processing can be distributed to different program modules as needed to complete all or part of the processing described above. Furthermore, the apparatus and the corresponding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0113] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform an image processing method.

[0114] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored and when executed by a processor, the processor will execute the image processing method provided in this application.

[0115] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0116] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0117] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0118] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0119] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure; as shown below. Figure 6 As shown, the electronic device 60 includes: a processor 601, and a memory 602 communicatively connected to the processor 601; the memory 602 stores instructions executable by the processor 601. The instructions are executed by the processor 601 to enable the processor 601 to perform the following: Acquire the image data to be transmitted and the reference image data; Calculate the inter-frame offset data between the image data to be transmitted and the reference image data; Obtain the first cumulative deviation data between the image data to be transmitted and the reference image data; If the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold, the reference image data is updated and the first cumulative deviation data is reset. The preset cumulative deviation threshold is determined based on the inter-frame deviation data. If the first cumulative deviation data is less than the preset cumulative deviation threshold, the difference data between the image data to be transmitted and the reference image data is determined as the target image data to be transmitted.

[0120] The electronic devices and corresponding image processing methods provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0121] In practical applications, the electronic device 60 may further include at least one network interface 603. The various components of the electronic device 60 are coupled together via a bus system 604. It is understood that the bus system 604 is used to implement communication between these components. In addition to a data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 All buses are labeled as bus system 604. The number of processors 601 and the number of memories 602 can be at least one. The network interface 603 is used for wired or wireless communication between the electronic device 60 and other devices.

[0122] The memory 602 in this embodiment is used to store various types of data to support the operation of the electronic device 60.

[0123] The methods disclosed in the above embodiments of this disclosure can be applied to or implemented by processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 601 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 602. Processor 601 reads the information in memory 602 and, in conjunction with its hardware, completes the steps of the aforementioned image processing method.

[0124] In some embodiments, the electronic device 60 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0125] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0126] In the above description, the term "some embodiments" refers to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0127] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in this disclosure is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0128] It should be understood that in the various embodiments of this disclosure, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0129] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0130] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: Acquire the image data to be transmitted and the reference image data; Calculate the inter-frame offset data between the image data to be transmitted and the reference image data; Obtain the first cumulative deviation data between the image data to be transmitted and the reference image data; If the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold, the reference image data is updated and the first cumulative deviation data is reset. The preset cumulative deviation threshold is determined based on the inter-frame deviation data. If the first cumulative deviation data is less than the preset cumulative deviation threshold, the difference data between the image data to be transmitted and the reference image data is determined as the target image data to be transmitted.

2. The method according to claim 1, characterized in that, The calculation of the inter-frame deviation data between the image data to be transmitted and the reference image data includes: Calculate the difference between the pixel value of each first pixel and the pixel value of the corresponding second pixel to obtain multiple pixel differences; The inter-frame deviation data is calculated based on the multiple pixel differences; The image data to be transmitted includes pixel values ​​corresponding to multiple first pixel points, and the reference image data includes pixel values ​​corresponding to multiple second pixel points.

3. The method according to claim 2, characterized in that, The step of calculating the inter-frame deviation data based on the plurality of pixel differences includes: The sum of the multiple pixel differences is determined as the inter-frame deviation data; or, The mean square error of the multiple pixel differences is determined as the inter-frame deviation data.

4. The method according to claim 1, characterized in that, The image data to be transmitted includes image data of multiple first image blocks, and the reference image data includes image data of second image blocks corresponding to the multiple first image blocks respectively; The calculation of the inter-frame deviation data between the image data to be transmitted and the reference image data includes: Based on the image data of each first image block and the image data of the second image block corresponding to the first image block, calculate the image block deviation data between the first image block and the second image block; Filter out third image blocks from the plurality of first image blocks whose corresponding image block deviation data is greater than a preset image block deviation, and determine the number of image blocks in the third image block; The ratio of the number of image blocks in the third image block to the number of image blocks in the plurality of first image blocks is determined as the inter-frame offset data.

5. The method according to any one of claims 1 to 4, characterized in that, The step of acquiring the first cumulative deviation data between the image data to be transmitted and the reference image data includes: Count the number of image frames between the image frame to be transmitted and the reference image frame; The number of image frames in the interval image frame is determined as the first cumulative deviation data; or, Obtain the second cumulative deviation data corresponding to the reference image data; The inter-frame offset data is added to the second cumulative offset data to obtain the first cumulative offset data after addition.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: If the inter-frame deviation data is less than the first deviation threshold, the preset cumulative deviation threshold is determined to be the first cumulative deviation threshold. If the inter-frame deviation data is greater than or equal to the first deviation threshold and the inter-frame deviation data is less than the second deviation threshold, the preset cumulative deviation threshold is determined to be the second cumulative deviation threshold. If the inter-frame deviation data is greater than the second deviation threshold, the preset cumulative deviation threshold is determined to be the third cumulative deviation threshold. Wherein, the first deviation threshold is less than the second deviation threshold; the first cumulative deviation threshold is greater than the second cumulative deviation threshold, and the second cumulative deviation threshold is greater than the third cumulative deviation threshold.

7. The method according to any one of claims 1 to 4, characterized in that, The image data to be transmitted includes image data of multiple first image blocks, and the reference image data includes image data of second image blocks corresponding to the multiple first image blocks respectively; Determining the difference data between the image data to be transmitted and the reference image data as the target image data to be transmitted includes: Based on the image data of each first image block and the image data of the second image block corresponding to the first image block, the image block correlation between the first image block and the second image block is calculated; From a plurality of first image blocks, target image blocks in which the image block correlation between the first image block and the second image block is less than a preset correlation threshold are selected; The image data of the target image block is determined to be the target image data to be transmitted.

8. The method according to claim 7, characterized in that, The image data to be transmitted includes image data of multiple first image blocks, and the reference image data includes image data of a second image block corresponding to each first image block; The acquisition of the image data to be transmitted and the reference image data includes: Acquire the image frame to be transmitted, the reference image frame, and block partitioning information, wherein the block partitioning information includes row partitioning information and / or column partitioning information; The image frame to be transmitted is divided according to the block division information to obtain multiple first image blocks, and the image data of the first image block is extracted from the multiple first image blocks respectively; The reference image frame is divided according to the block division information to obtain multiple second image blocks, and the image data of the second image blocks is extracted from the multiple second image blocks respectively.

9. The method according to claim 7, characterized in that, After determining that the image data of the target image block is the target image data to be transmitted, the method further includes: Obtain the position of the target image block in the image frame to be transmitted corresponding to the image data to be transmitted; Compress the image data and image block position of the target image block to obtain compressed image data; The compressed image data is sent to the receiving end for the receiving end to decompress the compressed image data and perform image restoration processing based on the image data of the target image block, the position of the image block, and the reference image data pre-transmitted to the receiving end to obtain the image data to be transmitted.

10. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the image data to be transmitted and the reference image data; The calculation module is used to calculate the inter-frame deviation data between the image data to be transmitted and the reference image data; The acquisition module is further configured to acquire first cumulative deviation data between the image data to be transmitted and the reference image data; An update module is used to update the reference image data and reset the first cumulative deviation data when the first cumulative deviation data is greater than or equal to a preset cumulative deviation threshold, wherein the preset cumulative deviation threshold is determined based on the inter-frame deviation data. The determination module is used to determine the difference data between the image data to be transmitted and the reference image data when the first cumulative deviation data is less than the preset cumulative deviation threshold, and thus determine the target image data to be transmitted.

11. The apparatus according to claim 10, characterized in that, The device includes: The calculation module is specifically used to calculate the difference between the pixel value of each first pixel and the pixel value of the second pixel corresponding to the first pixel, and obtain multiple pixel differences; The calculation module is specifically used to calculate the inter-frame deviation data based on the multiple pixel differences; The image data to be transmitted includes pixel values ​​corresponding to multiple first pixel points, and the reference image data includes pixel values ​​corresponding to multiple second pixel points.

12. The apparatus according to claim 11, characterized in that, The device includes: The determining module is further configured to determine the sum of the multiple pixel differences as the inter-frame deviation data; or, The determining module is further configured to determine the mean square error of the multiple pixel differences, which is the inter-frame deviation data.

13. The apparatus according to claim 10, characterized in that, The image data to be transmitted includes image data of multiple first image blocks, and the reference image data includes image data of second image blocks corresponding to the multiple first image blocks; the device includes: The calculation module is specifically used to calculate the image block deviation data between the first image block and the second image block based on the image data of each first image block and the image data of the second image block corresponding to the first image block; The filtering module is used to filter out third image blocks from the plurality of first image blocks whose corresponding image block deviation data is greater than a preset image block deviation, and to determine the number of image blocks in the third image block; The determining module is further configured to determine the ratio of the number of image blocks in the third image block to the number of image blocks in the plurality of first image blocks, which is the inter-frame offset data.

14. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 9.

Images