Image differential processing method based on data serialization format

Abstract

The embodiment of the invention provides an image differential processing method based on a data serialization format, and the method comprises the steps: 1, obtaining an original image and a target image, and carrying out the image differential calculation, and generating a differential image; 2, performing data compression and optimization based on the differential image to generate differential image data; 3, designing a message format of specific data serialization; 4, serializing the generated differential image data by adopting a setting method, and converting the differential image data into a binary format; and step 5, performing differential image decoding and reconstruction so as to perform display or processing. The transmission and processing efficiency of an image difference result can be improved while redundant data and bandwidth occupation can be effectively reduced, so that the requirements of a large-scale image processing system are met.

Description

Technical Field

[0001] The invention relates to the technical field of image processing and data serialization and transmission, and in particular to an image difference processing method based on a data serialization format. Background Art

[0002] In the field of image processing, image differential processing is a common method that aims to detect changes or highlight certain features by comparing the pixel differences between two or more images. This technology is widely used in many scenarios such as video surveillance, image change detection, motion tracking, and target detection. For example, in security monitoring systems, image differential processing is used to identify changes in the camera's field of view, thereby quickly detecting moving objects or abnormal situations.

[0003] Current image difference processing methods mostly focus on direct pixel-level difference, edge difference or advanced methods (such as structural similarity SSIM). These methods can effectively detect changes in images, but face some technical challenges, especially in the transmission and processing of large-scale image data, where the problems are particularly prominent:

[0004] 1. Data redundancy: When processing adjacent frames in a video stream or image sequence, only a small part of the area changes, but the traditional image difference method generates the entire difference image, resulting in a large amount of redundant data. These redundant data occupy a large bandwidth in network transmission, reducing system efficiency.

[0005] 2. Bandwidth and storage pressure: In distributed systems or real-time monitoring scenarios, image differential processing requires frequent transmission of differential image data. Due to the large amount of differential image data, it is easy to cause excessive network bandwidth occupancy and increase transmission delay. At the same time, storing this data also requires a large storage space, which puts pressure on the system's storage devices.

[0006] 3. Existing serialization methods are inefficient: Existing serialization methods, such as JSON and XML, can be used for serialization and transmission of image differential data, but they are inefficient, especially when processing large-scale, real-time image differential data, where performance bottlenecks are obvious. These formats often introduce unnecessary overhead and cannot meet the needs of efficient transmission.

[0007] In the above context, how to reduce bandwidth burden and improve system real-time response capability through more efficient data serialization and transmission technology has become an important issue in image differential processing. Protocol Buffers (Protobuf), as an efficient binary data serialization format, provides a potential solution to optimize data transmission problems in image differential processing due to its high efficiency, cross-platform and low overhead in data structure transmission. Summary of the invention

[0008] In view of the above-mentioned problems existing in the prior art, an embodiment of the present invention provides an image difference processing method based on a data serialization format, which can effectively reduce redundant data and bandwidth occupancy while improving the transmission and processing efficiency of image difference results to meet the needs of large-scale image processing systems.

[0009] The embodiment of the present invention provides an image difference processing method based on a data serialization format, comprising:

[0010] Step 1: Obtain the original image and the target image, perform image difference calculation, and generate a difference image;

[0011] Step 2: Based on the differential image, data compression and optimization are performed to generate differential image data;

[0012] Step 3: Design a message format for specific data serialization;

[0013] Step 4: Using the setting method, the generated differential image data is serialized and converted into a binary format;

[0014] Step 5: Decode and reconstruct the differential image for display or processing.

[0015] In some embodiments of the present invention, in step 1, the original image and the target image may be images of two consecutive video frames, or images of the same scene at different time points.

[0016] In some embodiments of the present invention, in step 1, the method comprises:

[0017] Set the original image to Image A and the target image to Image B;

[0018] Use the pixel difference method to compare the RGB value difference of each pixel in Image A and Image B;

[0019] For each pixel location (x,y), calculate the difference:

[0020] D t (x,y)=|I t (x,y)-I t-1 (x,y)|

[0021]

[0022] Among them, D t (x, y) represents the difference image at time t;

[0023] I t(x, y) represents the image at time t, which represents the target image Image B;

[0024] I t-1 (x, y) indicates that the image at time t-1 represents the original image Image A;

[0025] (x,y) is the pixel coordinate of the image;

[0026] If D t If (x, y) is greater than the set threshold T, it is considered that there is a moving target at the pixel;

[0027] Generate a difference image to show the changed area between Image A and Image B.

[0028] In some embodiments of the present invention, the differential image is a grayscale image or a color image containing RGB difference information.

[0029] In some embodiments of the present invention, in step 2, the method comprises:

[0030] For the generated differential image, a sparse matrix representation or other compression technology is used to reduce the storage and transmission overhead of unchanged areas;

[0031] If the pixel position does not exist, it will be skipped and not stored, and only the changed pixels will be retained;

[0032] Through sparse storage, only the pixel difference information of the changed part of the image is saved.

[0033] In some embodiments of the present invention, in step 3, the method comprises:

[0034] Use Protobuf for data serialization and design the Protobuf message format to represent the information of the differential image;

[0035] The defined Protobuf message includes image size information, pixel position, and difference value;

[0036] ImageDiff is used to represent the differential image, which contains the basic information of the image and the list of pixel differences;

[0037] PixelDiff is used to store the difference value of each pixel, including the pixel position (x, y) and the corresponding difference value.

[0038] In some embodiments of the present invention, in step 4, the method comprises:

[0039] Use the API provided by Protobuf to serialize the generated differential image data and convert it into a compact binary format.

[0040] In some embodiments of the present invention, in step 4, the method comprises:

[0041] The serialized differential image data is transmitted to a remote client via a network or stored in a file, wherein the differential image data can be decoded and reconstructed in a variety of programming languages ​​and operating systems.

[0042] In some embodiments of the present invention, in step 5, the method comprises:

[0043] On the receiving end, the received binary data is decoded into an ImageDiff object through the Protobuf deserialization API;

[0044] Reconstruct the differential image according to the pixel difference information recorded in ImageDiff;

[0045] For each changed pixel position (x, y), restore it to the target image Image B according to the difference value:

[0046] B(x,y)=A(x,y)+diff(x,y)

[0047] If necessary, the target image is reconstructed using the original image and the difference image.

[0048] Compared with the prior art, the image difference processing method based on the data serialization format provided by the embodiment of the present invention has the beneficial effect that it significantly improves the data transmission efficiency and the real-time performance of the system by combining image difference with Protocol Buffers. Its performance in reducing data volume, adapting to distributed environments, enhancing real-time performance and reliability, etc., shows strong practicality and innovation. This system can be widely used in scenarios that require efficient image transmission and meet application requirements in low-bandwidth, high-latency environments.

[0049] 1. Significantly reduce the amount of data transmission: By using image difference technology, only the changed parts between images are transmitted instead of the entire image. This method effectively reduces the amount of data that needs to be transmitted, especially in video streaming or high-resolution image transmission, where the differential data is far less than the complete image data, thereby significantly reducing bandwidth consumption. Furthermore, by combining with Protocol Buffers, the differential data is serialized, and the compressed data format is more compact, efficient and fast. This serialization is not only applicable to image data, but also can transmit data with complex structures, significantly improving the efficiency of the system.

[0050] 2. Improve transmission efficiency: Protocol Buffers is a lightweight and efficient serialization format. Its binary encoding structure has faster transmission speed and lower overhead than traditional formats such as JSON and XML, and is particularly suitable for real-time data transmission scenarios. By transmitting the image difference results in Protocol Buffers format, the network bandwidth usage is reduced, which meets the application requirements of low latency and high throughput.

[0051] 3. Enhance the real-time and reliability of the system: Compared with traditional image transmission methods, the use of image differentiation and Protocol Buffers can greatly improve real-time performance. In network transmission, the size of data packets is smaller and the transmission is faster, thereby reducing latency. In addition, differential technology can reduce the time it takes for the system to process each frame of an image in the scenario of continuous video frame transmission. Even if there is packet loss or instability in network transmission, the differential data has a strong ability to recover, especially through efficient differentiation and serialization methods, which enhances the reliability of the transmission system.

[0052] 4. Adapt to distributed and multi-device environments: This technical solution is particularly suitable for distributed systems or IoT scenarios. The cross-platform characteristics of Protocol Buffers make it easy to transmit and parse data between different devices. Whether it is a low-power device, server or mobile device, it can efficiently decode and process image differential data.

[0053] In a large-scale distributed environment, the data transmission burden between system nodes is reduced, which is suitable for scenarios that require efficient data exchange, such as smart cities, unmanned driving, and telemedicine.

[0054] 5. Flexibility and scalability: By customizing the data structure of Protocol Buffers, the data content can be flexibly adjusted according to specific application requirements to adapt to different application scenarios. At the same time, the differential calculation method can be dynamically adjusted according to the image content, transmission conditions and real-time requirements, and has strong scalability. The system performs well in image compression and transmission scenarios, but can also be extended to other application scenarios, such as video surveillance, remote transmission and streaming media services. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flowchart of an image difference processing method based on a data serialization format provided by an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of the protobuffer message definition in the image difference processing method based on the data serialization format provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0058] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0059] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0060] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0061] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0062] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to determine the true intent based on the user's historical operations and to avoid unnecessary or redundant details that make the present application unclear. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but are merely used as the basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0063] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0064] The embodiment of the present invention provides an image difference processing method based on a data serialization format, such as Figure 1 and Figure 2 As shown, the method includes:

[0065] Step 1: Obtain the original image and the target image, perform image difference calculation, and generate a difference image;

[0066] In this embodiment, the original image and the target image may be images of two consecutive video frames, or images of the same scene at different time points.

[0067] Set the original image to Image A and the target image to Image B;

[0068] Use the pixel difference method to compare the RGB value difference of each pixel in Image A and Image B;

[0069] For each pixel location (x,y), calculate the difference:

[0070] D t (x,y)=|I t (x,y)-I t-1 (x,y)|

[0071]

[0072] Among them, D t (x, y) represents the difference image at time t;

[0073] I t (x, y) represents the image at time t, which represents the target image Image B;

[0074] I t-1 (x, y) indicates that the image at time t-1 represents the original image Image A;

[0075] (x,y) is the pixel coordinate of the image;

[0076] If D t If (x, y) is greater than the set threshold T, it is considered that there is a moving target at the pixel;

[0077] Generate a difference image to show the changed area between Image A and Image B.

[0078] The difference image is a grayscale image or a color image containing RGB difference information.

[0079] Step 2: Based on the differential image, data compression and optimization are performed to generate differential image data;

[0080] In this embodiment, for the generated differential image, a sparse matrix representation method or other compression technology is used to reduce the storage and transmission overhead of the unchanged area;

[0081] If the pixel position does not exist, it will be skipped and not stored, and only the changed pixels will be retained;

[0082] Through sparse storage, only the pixel difference information of the changed part of the image is saved.

[0083] Step 3: Design a message format for specific data serialization;

[0084] In this embodiment, Protobuf is used for data serialization, and the Protobuf message format is designed to represent the information of the differential image;

[0085] The defined Protobuf message includes image size information, pixel position, and difference value;

[0086] ImageDiff is used to represent the differential image, which contains the basic information of the image and the list of pixel differences;

[0087] PixelDiff is used to store the difference value of each pixel, including the pixel position (x, y) and the corresponding difference value.

[0088] Step 4: Using the setting method, the generated differential image data is serialized and converted into a binary format;

[0089] In this embodiment, the API provided by Protobuf is specifically used to serialize the generated differential image data and convert it into a compact binary format.

[0090] The serialized differential image data is transmitted to a remote client via a network or stored in a file, wherein the differential image data can be decoded and reconstructed in a variety of programming languages ​​and operating systems.

[0091] Step 5: Decode and reconstruct the differential image for display or processing.

[0092] In this embodiment, in step 5, the method includes:

[0093] On the receiving end, the received binary data is decoded into an ImageDiff object through the Protobuf deserialization API;

[0094] Reconstruct the differential image according to the pixel difference information recorded in ImageDiff;

[0095] For each changed pixel position (x, y), restore it to the target image Image B according to the difference value:

[0096] B(x,y)=A(x,y)+diff(x,y)

[0097] If necessary, the target image is reconstructed using the original image and the difference image.

[0098] In order to facilitate the understanding of the above technical solution, the following is explained in conjunction with specific examples, as follows:

[0099] The embodiment of the present invention specifically provides an efficient data transmission and processing method based on Protocol Buffers and image difference technology, which is suitable for efficient transmission and real-time processing scenarios of image and video data.

[0100] The following are the specific implementation steps of this technology:

[0101] 1. Image difference processing

[0102] Step 1.1: Get two consecutive frames of images

[0103] During the video stream or image transmission process, two consecutive image frames are first obtained, which are respectively recorded as the original frame (Image A) and the target frame (Image B).

[0104] Step 1.2: Calculate the difference between frames

[0105] The two images are compared using a pixel difference algorithm to generate a difference image. The result is a difference image containing the difference values, where only the pixels in the different parts of the image are retained, while other identical pixels can be ignored or set to zero.

[0106] 2. Compression and optimization of differential data

[0107] Step 2.1: Sparse processing

[0108] Perform sparse processing on the difference image, delete areas with small or no changes, and optimize storage. You can use threshold judgment to decide whether to keep the difference value of a certain pixel. For example, if the difference value of a certain pixel is lower than the preset threshold, ignore the difference to avoid transmitting meaningless small changes.

[0109] Step 2.2: Run-Length Encoding (RLE)

[0110] Run-length encoding is further applied to the differential image to compress consecutive identical difference values ​​into "value-count" pairs. In particular, a large number of zero values ​​can be efficiently represented by run-length encoding. Run-length encoding can effectively reduce the amount of data, especially in scenes with small changes in high-resolution images or videos.

[0111] 3. Protocol Buffers Serialization

[0112] Step 3.1: Define the Protobuf message structure

[0113] In order to efficiently transmit differential data, Protocol Buffers is used to serialize the data. A message structure is defined to store image differential data, including image size, pixel differential data, compressed results and other information.

[0114] Step 3.2: Serialize differential data

[0115] The optimized differential data is filled into the ImageDiff message structure and converted into binary format using the Protobuf serialization tool. The serialization method of Protocol Buffers is efficient and compact, suitable for real-time transmission of large-scale image data.

[0116] 4. Data transmission and reception

[0117] Step 4.1: Transfer binary data

[0118] The serialized differential data is transmitted to the receiving end through the network. Since Protobuf uses binary format, the amount of transmitted data is small, which is suitable for real-time applications in low-bandwidth or unstable network environments.

[0119] Step 4.2: Receive and decode

[0120] The receiving end obtains the serialized binary data through the network, deserializes it using the Protobuf tool, and restores the ImageDiff object. The decoded data contains the structural information of the differential image.

[0121] 5. Image reconstruction and display

[0122] Step 5.1: Difference image reconstruction

[0123] At the receiving end, the differential value is applied on the basis of the original image according to the deserialized PixelDiff information.

[0124] Step 5.2: Display of reconstructed image

[0125] After reconstruction is complete, the target frame (Image B) is displayed or further processed as the next frame in the video stream.

[0126] It can be seen from the above technical solutions that the image difference processing method based on the data serialization format provided by the above embodiments of the present invention significantly improves the data transmission efficiency and the real-time performance of the system by combining image difference with Protocol Buffers. Its performance in reducing data volume, adapting to distributed environments, enhancing real-time performance and reliability, etc., shows strong practicality and innovation. This system can be widely used in scenarios that require efficient image transmission and meet application requirements in low-bandwidth, high-latency environments.

[0127] 1. Significantly reduce the amount of data transmission: By using image difference technology, only the changed parts between images are transmitted instead of the entire image. This method effectively reduces the amount of data that needs to be transmitted, especially in video streaming or high-resolution image transmission, where the differential data is far less than the complete image data, thereby significantly reducing bandwidth consumption. Furthermore, by combining with Protocol Buffers, the differential data is serialized, and the compressed data format is more compact, efficient and fast. This serialization is not only applicable to image data, but also can transmit data with complex structures, significantly improving the efficiency of the system.

[0128] 2. Improve transmission efficiency: Protocol Buffers is a lightweight and efficient serialization format. Its binary encoding structure has faster transmission speed and lower overhead than traditional formats such as JSON and XML, and is particularly suitable for real-time data transmission scenarios. By transmitting the image difference results in Protocol Buffers format, the network bandwidth usage is reduced, which meets the application requirements of low latency and high throughput.

[0129] 3. Enhance the real-time and reliability of the system: Compared with traditional image transmission methods, the use of image differentiation and Protocol Buffers can greatly improve real-time performance. In network transmission, the size of data packets is smaller and the transmission is faster, thereby reducing latency. In addition, differential technology can reduce the time it takes for the system to process each frame of an image in the scenario of continuous video frame transmission. Even if there is packet loss or instability in network transmission, the differential data has a strong ability to recover, especially through efficient differentiation and serialization methods, which enhances the reliability of the transmission system.

[0130] 4. Adapt to distributed and multi-device environments: This technical solution is particularly suitable for distributed systems or IoT scenarios. The cross-platform characteristics of Protocol Buffers make it easy to transmit and parse data between different devices. Whether it is a low-power device, server or mobile device, it can efficiently decode and process image differential data.

[0131] In a large-scale distributed environment, the data transmission burden between system nodes is reduced, which is suitable for scenarios that require efficient data exchange, such as smart cities, unmanned driving, and telemedicine.

[0132] 5. Flexibility and scalability: By customizing the data structure of Protocol Buffers, the data content can be flexibly adjusted according to specific application requirements to adapt to different application scenarios. At the same time, the differential calculation method can be dynamically adjusted according to the image content, transmission conditions and real-time requirements, and has strong scalability. The system performs well in image compression and transmission scenarios, but can also be extended to other application scenarios, such as video surveillance, remote transmission and streaming media services.

[0133] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. An image difference processing method based on data serialization format, characterized in that: include: Step 1: Obtain the original image and the target image, perform image difference calculation, and generate a difference image; Step 2: Based on the differential image, data compression and optimization are performed to generate differential image data; Step 3: Design a message format for specific data serialization; Step 4: Using the setting method, the generated differential image data is serialized and converted into a binary format; Step 5: Decode and reconstruct the differential image for display or processing.

2. The image difference processing method based on data serialization format according to claim 1 is characterized in that: In step 1, The original image and the target image may be images of two consecutive video frames, or images of the same scene at different time points.

3. The image difference processing method based on data serialization format according to claim 2 is characterized in that: In step 1, the method comprises: Set the original image to Image A and the target image to Image B; Use the pixel difference method to compare the RGB value difference of each pixel in Image A and Image B; For each pixel location (x,y), calculate the difference: D t (x,y)=|I t (x,y)-I t-1 (x,y)| Among them, D t (x, y) represents the difference image at time t; I t (x, y) represents the image at time t, which represents the target image Image B; I t-1 (x, y) indicates that the image at time t-1 represents the original image Image A; (x,y) is the pixel coordinate of the image; If D t If (x,y) is greater than the set threshold T, it is considered that there is a moving target at the pixel; Generate a difference image to show the changed area between Image A and Image B.

4. The image difference processing method based on data serialization format according to claim 3 is characterized in that: The difference image is a grayscale image or a color image containing RGB difference information.

5. The image difference processing method based on data serialization format according to claim 4 is characterized in that: In step 2, the method comprises: For the generated differential image, a sparse matrix representation or other compression technology is used to reduce the storage and transmission overhead of unchanged areas; If the pixel position has not changed, it will be skipped and not stored, and only the changed pixels will be retained; Through sparse storage, only the pixel difference information of the changed part of the image is saved.

6. The image difference processing method based on data serialization format according to claim 5 is characterized in that: In step 3, the method comprises: Use Protobuf for data serialization and design the Protobuf message format to represent the information of the differential image; The defined Protobuf message includes image size information, pixel position, and difference value; ImageDiff is used to represent the differential image, which contains the basic information of the image and the list of pixel differences; PixelDiff is used to store the difference value of each pixel, including the pixel position (x, y) and the corresponding difference value.

7. The image difference processing method based on data serialization format according to claim 6 is characterized in that: In step 4, the method comprises: Use the API provided by Protobuf to serialize the generated differential image data and convert it into a compact binary format.

8. The image difference processing method based on data serialization format according to claim 7 is characterized in that: In step 4, the method comprises: The serialized differential image data is transmitted to a remote client via a network or stored in a file, wherein the differential image data can be decoded and reconstructed in a variety of programming languages ​​and operating systems.

9. The image difference processing method based on data serialization format according to claim 8 is characterized in that: In step 5, the method comprises: On the receiving end, the received binary data is decoded into an ImageDiff object through the Protobuf deserialization API; Reconstruct the differential image according to the pixel difference information recorded in ImageDiff; For each changed pixel position (x, y), restore it to the target image Image B according to the difference value: B(x,y)=A(x,y)+diff(x,y) If necessary, the target image is reconstructed using the original image and the difference image.

Images