Low-delay full-frame-rate image transmission method based on distributed architecture

Abstract

The invention relates to the technical field of system architecture design, in particular to a low-delay full-frame-rate image transmission method based on a distributed architecture, which comprises the following steps of: firstly, acquiring original data of an image by an edge end, publishing the original data to a distributed architecture system after compressing and packaging the original data, and receiving images of other edge ends or the edge end by the edge end; the image processing module receives the image and then decompresses the image through a JEPG compression algorithm, the image processing module receives the decompressed image and inputs image data into an identification tracking algorithm to process the image, and the image fusion module receives original image data of an edge end in the system and a processing result of the image processing module and completes fusion. By using a scheme of separately transmitting an original image and a result, adopting a parallel computing method and simultaneously adopting a compression algorithm for the original image, the pressure on the bandwidth is reduced, the delay of the image in the distributed system is improved, and the transmission bandwidth is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of system architecture design, and in particular to a low-latency full-frame rate image transmission method based on a distributed architecture. Background Art

[0002] With the development of emerging technologies such as cloud computing and the Internet of Things, video transmission technology under distributed architecture has received more and more attention. In a distributed architecture environment, video is transmitted from one node to another through network technology. Each node can cooperate with each other to complete the video transmission task. At present, there are mainly the following forms of video transmission technology in a distributed environment: point-to-point transmission, cloud computing-based transmission, edge computing transmission, etc. Different transmission forms are suitable for different scenarios. In the process of sensor, edge and terminal transmission, ensuring the transmission frame rate and image quality of the image, reducing the transmission rate of the image, and synchronizing the original image and the result are the difficulties of video transmission.

[0003] The existing image transmission process is a sensor-edge-terminal process. The combination of sensors and edge devices has a crucial impact on the efficiency and performance of edge computing. As a node for data processing and transmission, edge devices need to work closely with sensors to achieve real-time data collection and transmission, which can reduce latency, protect data privacy, and improve local decision-making capabilities. The structure is also relatively simple and easy to perform logical processing, without data asynchrony and inconsistency, and easy to maintain and manage.

[0004] Although the combination of edge computing and sensors brings many advantages, such as low latency, data privacy protection, and local decision-making capabilities, there are also some challenges and potential drawbacks in practical applications. When multiple edge devices need to share or synchronize data in a distributed architecture, maintaining data consistency becomes a challenge. Secondly, edge devices are usually numerous and widely distributed, which increases the complexity of management, monitoring, and maintenance. Finally, the edge calculation results must be transmitted to the terminal. Edge processing consumes some time, and the bandwidth limitation between the terminal and the edge and the delay between transmissions will reduce the terminal's display frame rate, increase latency, and occupy too much bandwidth.

[0005] In summary, the existing video transmission technology under the distributed architecture takes a long time to process at the edge, and the bandwidth limitation between the terminal and the edge and the delay between transmission will reduce the display frame rate of the terminal, increase the delay, and occupy too much bandwidth. Summary of the invention

[0006] The purpose of the present invention is to provide a low-latency full-frame rate image transmission method based on a distributed architecture, so as to solve the problem that the existing video transmission technology under the distributed architecture takes a long time to perform edge processing, and the bandwidth limitation between the terminal and the edge and the delay between transmission will reduce the display frame rate of the terminal, increase the delay, and occupy too much bandwidth.

[0007] To achieve the above object, the present invention provides a low-latency full-frame rate image transmission method based on a distributed architecture, and the low-latency full-frame rate image transmission method based on a distributed architecture comprises the following steps: Get the camera image, obtain the raw data of the image through the camera driver at the edge closest to the image, and convert the image into the common cvMAT format through opencv; At the edge, the image compression algorithm is used to compress the data in cvMAT format, and the compressed data is encapsulated as ROS messages and published to the distributed architecture system; The terminal receives the compressed image sent by the edge end, and the terminal decodes the data, decoding the compressed image data through the compression algorithm; The edge receives images from other edge ends or the edge end itself, decompresses them using the JPEG compression algorithm, converts them into cvMAT format, records the timestamp, and outputs them to the image processing module; The image processing module receives the decompressed image, inputs the image data into the recognition and tracking algorithm to process the image, and encapsulates the processing result as a ROS message and publishes it to the distributed architecture system; The image fusion module receives the original image data at the edge of the system and the processing results of the image processing module. After receiving and completing, the image result data with close time is fused by comparing the timestamps to obtain an image containing the processing results. Compress the image containing the processing results, attach a timestamp, and publish it to the distributed architecture system; The terminal receives the compressed result image in the distributed architecture system, decompresses it, and displays it.

[0008] Among them, in the step of "using an image compression algorithm at the edge to compress the data in cvMAT format, and encapsulating the compressed data as a ROS message and publishing it to the distributed architecture system", the image compression algorithm is a JPEG lossy compression algorithm based on discrete cosine transform.

[0009] Among them, in the step of "using an image compression algorithm at the edge to compress the data in cvMAT format, and encapsulating the compressed data as a ROS message and publishing it to the distributed architecture system", the compressed data includes image data, image size, compression method and timestamp.

[0010] Among them, in the step "the image processing module receives the decompressed image, inputs the image data into the recognition and tracking algorithm to process the image, and encapsulates the processing result as a ROS message and publishes it to the distributed architecture system", the processing result includes the recognition box, the recognition ID, the recognition time and the recognition type.

[0011] The present invention provides a low-latency full-frame rate image transmission method based on a distributed architecture. First, the edge end obtains the original data of the image, and publishes it to the distributed architecture system after compression and encapsulation, and the edge end receives the image of other edge ends or the edge end, and decompresses it through the JPEG compression algorithm after receiving. The image processing module receives the decompressed image, and inputs the image data into the recognition and tracking algorithm to process the image. The image fusion module receives the original image data of the edge end in the system and the processing result of the image processing module, and completes the fusion. The technical solution is adopted by using a solution in which the original image and the result are transmitted separately, and a parallel computing method is adopted. At the same time, the original image adopts a compression algorithm to reduce the pressure on the bandwidth, improve the delay of the image in the distributed system and reduce the transmission bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 It is an image processing flow chart of the image transmission method with low delay and full frame rate based on distributed architecture provided by the present invention.

[0014] Figure 2 It is a flow monitoring diagram of an uncompressed image provided by the present invention.

[0015] Figure 3 This is a flow monitoring diagram of the compressed image provided by the present invention. DETAILED DESCRIPTION

[0016] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0017] See also Figures 1 to 3The present invention provides a low-latency full-frame rate image transmission method based on a distributed architecture, and the low-latency full-frame rate image transmission method based on a distributed architecture comprises the following steps: S1: Get the camera image, obtain the raw data of the image through the camera driver at the edge closest to the image, and convert the image into the common cvMAT format through opencv; S2: Use image compression algorithm to compress the data in cvMAT format at the edge, encapsulate the compressed data into ROS messages and publish them to the distributed architecture system; S3: The terminal receives the compressed image sent by the edge end, and the terminal decodes the data, decoding the compressed image data using a compression algorithm; S4: The edge receives images from other edge or local edge, decompresses them using the JPEG compression algorithm, converts them into cvMAT format, records timestamps, and outputs them to the image processing module. S5: The image processing module receives the decompressed image, inputs the image data into the recognition and tracking algorithm to process the image, and encapsulates the processing result as a ROS message and publishes it to the distributed architecture system; S6: The image fusion module receives the original image data at the edge of the system and the processing result of the image processing module. After receiving and completing, the image result data with close time are fused by comparing the timestamps to obtain an image containing the processing result. S7: compress the image containing the processing result, attach a timestamp, and publish it to the distributed architecture system; S8: The terminal receives the compressed result image in the distributed architecture system, decompresses it, and displays it.

[0018] In this embodiment, first, the edge end obtains the original data of the image, and publishes it to the distributed architecture system after compression and encapsulation, and the edge end receives images from other edge ends or the edge end itself, and decompresses them through the JPEG compression algorithm after receiving them. The image processing module receives the decompressed image, and inputs the image data into the recognition and tracking algorithm to process the image. The image fusion module receives the original image data of the edge end in the system and the processing results of the image processing module, and completes the fusion. The technical solution is adopted by using a solution in which the original image and the result are transmitted separately, and a parallel computing method is adopted. At the same time, the original image adopts a compression algorithm to reduce the pressure on the bandwidth, improve the image delay in the distributed system and reduce the transmission bandwidth.

[0019] Furthermore, in the step of "using an image compression algorithm at the edge to compress the data in the cvMAT format, and encapsulating the compressed data as a ROS message and publishing it to the distributed architecture system", the image compression algorithm is a JPEG lossy compression algorithm based on discrete cosine transform.

[0020] Furthermore, in the step of "using an image compression algorithm at the edge to compress the data in the cvMAT format, and encapsulating the compressed data as a ROS message and publishing it to the distributed architecture system", the compressed data includes image data, image size, compression method, timestamp, etc.

[0021] Furthermore, in the step "the image processing module receives the decompressed image, inputs the image data into the recognition and tracking algorithm to process the image, and encapsulates the processing result as a ROS message and publishes it to the distributed architecture system", the processing result includes the recognition box, the recognition ID, the recognition time and the recognition type, etc.

[0022] See also Figure 2 and Figure 3 The present invention also provides an application of a low-delay full-frame-rate image transmission method based on a distributed architecture: First, the acquired image is compressed. The compression algorithm uses the JPEG lossy compression algorithm based on discrete cosine transform (DCT). It can compress the image to a smaller file size while ensuring good image quality. In the actual test process, the bandwidth of sending a color image (640*480) at 30fps is about 18MB / s~20MB / s (see attached Figure 2 ), however, after JPEG compression, the bandwidth is between 350KB / s and 450KB / s (see Appendix Figure 3 ), after compression, it is 1 / 50 of the original size. In this way, low-bandwidth transmission can be achieved without losing too much image quality and published to the system.

[0023] Secondly, separate the image and the result. According to the traditional edge computing method, when the frame rate is 30fps, the recognition time is 30ms, and the output frame rate is 1000 / 30≈33fps. Add the transmission time of about 10ms (generally 7ms, and the maximum tested time did not exceed 10ms), 1000 / 40=25fps, which greatly reduces the image frame rate and generates an image delay of about 43ms.

[0024] In summary, the new distributed architecture is designed by separating images and results, starting a node to receive images and results, and merging them on the premise of aligning timestamps. This way, the terminal always displays at full frame rate, and the delay is about 10ms, which greatly improves the image quality and system stability.

[0025] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A low-latency full-frame rate image transmission method based on a distributed architecture, characterized in that: The steps include: Get the camera image, obtain the raw data of the image through the camera driver at the edge closest to the image, and convert the image into the common cvMAT format through opencv; At the edge, the image compression algorithm is used to compress the data in cvMAT format, and the compressed data is encapsulated as ROS messages and published to the distributed architecture system; The terminal receives the compressed image sent by the edge end, and the terminal decodes the data, decoding the compressed image data through the compression algorithm; The edge receives images from other edge ends or the edge end itself, decompresses them using the JPEG compression algorithm, converts them into cvMAT format, records the timestamp, and outputs them to the image processing module; The image processing module receives the decompressed image, inputs the image data into the recognition and tracking algorithm to process the image, and encapsulates the processing result as a ROS message and publishes it to the distributed architecture system; The image fusion module receives the original image data at the edge of the system and the processing results of the image processing module. After receiving and completing, the image result data with close time is fused by comparing the timestamps to obtain an image containing the processing results. Compress the image containing the processing results, attach a timestamp, and publish it to the distributed architecture system; The terminal receives the compressed result image in the distributed architecture system, decompresses it, and displays it.

2. The low-latency full-frame rate image transmission method based on a distributed architecture as claimed in claim 1, characterized in that: In the step "using an image compression algorithm at the edge to compress the data in cvMAT format, and encapsulating the compressed data as a ROS message and publishing it to the distributed architecture system", the image compression algorithm is a JPEG lossy compression algorithm based on discrete cosine transform.

3. The low-latency full-frame rate image transmission method based on a distributed architecture as claimed in claim 2, characterized in that: In the step of "compressing the data in cvMAT format by using an image compression algorithm at the edge, and encapsulating the compressed data as a ROS message and publishing it to the distributed architecture system", the compressed data includes image data, image size, compression method and timestamp.

4. The method for low-latency full-frame-rate image transmission based on a distributed architecture as claimed in claim 3, characterized in that: In step "the image processing module receives the decompressed image, inputs the image data into the recognition and tracking algorithm to process the image, and encapsulates the processing result as a ROS message and publishes it to the distributed architecture system", the processing result includes the recognition box, the recognition ID, the recognition time and the recognition type.

Images