Multipath camera data processing method

Abstract

The invention discloses a multi-camera data processing method, which belongs to the technical field of monitoring data processing, and comprises the following steps: S1, constructing a multi-camera data processing framework; s2, the multi-path camera transmits the collected original data to the transmitting end, the transmitting end processes the received original data and packs and outputs the data according to a preset protocol, the optical fiber transmits an output data packet to the receiving end, the receiving end processes the received data packet and unpacks the data packet according to the preset protocol, and the optical fiber transmits the unpacked data packet to the receiving end; and assembling the frame data into complete frame data and displaying the frame data. According to the invention, the wiring problem that a plurality of lines are needed for transmitting original data by a plurality of cameras and the delay aggravation problem caused by decompression of a receiving end due to compression of an acquisition end can be solved at the same time, the processing burden is reduced, and the method is suitable for long-distance transmission.

Description

Technical Field

[0001] The invention belongs to the technical field of monitoring data processing, and in particular relates to a multi-channel camera data processing method. Background Art

[0002] With the continuous development of monitoring technology, multi-channel cameras have been widely used in public safety, intelligent transportation, intelligent security and other fields.

[0003] At present, there are generally two methods for processing data from multiple cameras. One is to directly transmit the data collected by multiple cameras from the sending end to the receiving end, and the receiving end displays the data after receiving it. The other is to encode the data collected by multiple cameras from the sending end before transmitting it, and the receiving end decodes the data before displaying it. The above methods have the following problems:

[0004] 1. The first method is only applicable to the case where the distance between the transmitter and the receiver is short due to the large amount of raw data from multiple cameras and limited bandwidth. In addition, due to the limited bandwidth, basically each camera from the transmitter to the receiver corresponds to one cable. When there are many cameras, the requirements for wiring space and wiring specifications are met.

[0005] 2. The encoding and decoding method of the second method increases the processing burden of the sender and the receiver, and inevitably aggravates the delay problem of image display.

[0006] In view of this, a multi-channel camera data processing method is designed to solve the above problems. Summary of the invention

[0007] To solve the problems raised in the above background technology. The present invention provides a multi-channel camera data processing method, which has the characteristics of solving the wiring problem of multiple lines required for multiple cameras to transmit raw data, solving the delay problem caused by compression at the acquisition end and decompression at the receiving end, reducing the processing burden, and being suitable for long-distance transmission.

[0008] To achieve the above object, the present invention provides the following technical solution: a multi-channel camera data processing method, comprising the following steps:

[0009] S1: construct a multi-channel camera data processing framework, including multi-channel cameras, a transmitter, an optical fiber and a receiver, wherein the multi-channel cameras are connected to the transmitter input, the transmitter output is connected to the optical fiber input, and the optical fiber output is connected to the receiver input;

[0010] S2: Multiple cameras transmit the collected raw data to the transmitter, which processes the received raw data and packages it for output according to the preset protocol. The optical fiber transmits the output data packets to the receiver, which processes the received data packets and unpacks them according to the preset protocol, and then assembles them into complete frame data for display.

[0011] Furthermore, in step S1, the specific steps of constructing multiple cameras, a transmitting end and an optical fiber include:

[0012] The transmitting end includes an input port, an electric-to-digital converter, a DSP processor, a digital-to-optical converter and an output port. The input port input end is connected to the cable for transmitting data from multiple cameras, the input port output end is connected to the electric-to-digital converter input end, the electric-to-digital converter output end is connected to the DSP processor input end, the DSP processor output end is connected to the digital-to-optical converter input end, the digital-to-optical converter output end is connected to the output port input end, and the output port output end is connected to the optical fiber input end.

[0013] Furthermore, in step S1, the specific steps of constructing the optical fiber and the receiving end include:

[0014] The receiving end includes an input port, an optical-to-digital converter, a DSP processor and an output port. The input port input end is connected to the optical fiber output end, the input port output end is connected to the optical-to-digital converter input end, the optical-to-digital converter output end is connected to the DSP processor input end, the DSP processor output end is connected to the output port input end, and the output port output end is connected to the terminal device input end.

[0015] Furthermore, in step S2, the specific steps of the sending end processing the received raw data and packaging and outputting it according to the preset protocol include:

[0016] The raw data collected by multiple cameras is transmitted to the input port, and then input into the electric-to-digital converter. The electric-to-digital converter converts the electrical signal into a digital signal. The converted digital signal is the original YUV format data of the camera. The YUV data is packaged by the DSP processor according to the preset protocol format. The packaged data is converted into an optical signal by the digital-to-optical converter. The converted optical signal is output to the optical fiber through the output port.

[0017] Furthermore, in step S2, when the DSP processor packages the YUV data according to the preset protocol format, it is necessary to add header information before each packet of frame data, including the packet start flag, frame rate, resolution, image format, sequence number of the current packet in the frame, camera sequence number and timestamp.

[0018] Furthermore, in step S2, when the DSP processor packages the YUV data according to the preset protocol format, the portion of the frame data that is received can start packaging, and the size of the portion of the frame data that can start packaging is related to the resolution, frame rate and image format.

[0019] Furthermore, in step S2, the receiving end processes the received data packets and unpacks them according to a preset protocol, and then assembles them into complete frame data for display, including:

[0020] The optical fiber transmits the optical signal output from the output port of the transmitter to the input port of the receiver, and then inputs it into the optical-to-digital converter. The optical-to-digital converter converts the optical signal into a digital signal and inputs it into the DSP processor. The DSP processor unpacks the data according to the preset protocol, and then assembles it into a complete frame data according to the sequence number of the packet, and inputs it into the output port. The output port outputs the complete frame data for display.

[0021] Furthermore, in step S2, the complete frame data output by the output port is transmitted to the terminal device and displayed by a display of the terminal device.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. The present invention is composed of a multi-channel camera, a transmitting end, an optical fiber and a receiving end. The original data collected by the multi-channel camera is gathered at the transmitting end, then transmitted to the receiving end by the optical fiber for processing, and then transmitted to the terminal device by the receiving end for display. Compared with the prior art, the data of the multi-channel camera can be integrated together and transmitted through optical fiber communication, which solves the wiring problem of multiple lines required for multiple cameras to transmit original data. At the same time, since optical fiber communication transmits uncompressed original data, it solves the problem of increased delay caused by compression at the collecting end and decompression at the receiving end. At the same time, since there is no encoding and decoding program, the processing burden is reduced. At the same time, since optical fiber communication has the advantages of strong anti-interference ability and weak signal attenuation, it is suitable for long-distance transmission.

[0024] 2. The transmitting end of the present invention converts the raw data collected by multiple cameras into electrical digital signals before sending, and then packages them. During the packaging process, the data can be packaged when part of the frame data is received. After packaging, the data is converted into optical signals and then output. The receiving end converts the optical digital signals after receiving, and then unpacks them. Finally, they are assembled into a complete frame data according to the sequence number and then output. Compared with the existing technology, it can reduce delays, improve processing speed, and design a compensation mechanism for frame loss that may be caused by packaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a framework diagram of the multi-channel camera data processing method of the present invention;

[0026] Figure 2 It is a flow chart of the multi-channel camera data processing method of the present invention;

[0027] Figure 3 This is a system architecture diagram of a receiving end without data processing function. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] See also Figure 1-2 The present invention provides the following technical solution: a multi-channel camera data processing method, comprising the following steps:

[0030] S1: construct a multi-channel camera data processing framework, including multi-channel cameras, a transmitter, an optical fiber and a receiver, wherein the multi-channel cameras are connected to the transmitter input, the transmitter output is connected to the optical fiber input, and the optical fiber output is connected to the receiver input;

[0031] S2: Multiple cameras transmit the collected raw data to the transmitter, which processes the received raw data and packages it for output according to the preset protocol. The optical fiber transmits the output data packets to the receiver, which processes the received data packets and unpacks them according to the preset protocol, and then assembles them into complete frame data for display.

[0032] Specifically, in step S1, the specific steps of constructing multiple cameras, a transmitting end, and an optical fiber include:

[0033] The transmitting end includes an input port, an electric-to-digital converter, a DSP processor, a digital-to-optical converter and an output port. The input port input end is connected to the cable for transmitting data from multiple cameras, the input port output end is connected to the electric-to-digital converter input end, the electric-to-digital converter output end is connected to the DSP processor input end, the DSP processor output end is connected to the digital-to-optical converter input end, the digital-to-optical converter output end is connected to the output port input end, and the output port output end is connected to the optical fiber input end.

[0034] Specifically, in step S1, the specific steps of constructing the optical fiber and the receiving end include:

[0035] The receiving end includes an input port, an optical-to-digital converter, a DSP processor and an output port. The input port input end is connected to the optical fiber output end, the input port output end is connected to the optical-to-digital converter input end, the optical-to-digital converter output end is connected to the DSP processor input end, the DSP processor output end is connected to the output port input end, and the output port output end is connected to the terminal device input end.

[0036] like Figure 3 As shown, in another embodiment, the receiving end is only responsible for converting the optical fiber signal to the digital signal, outputting the digital signal to the host system, and the host system is responsible for restoring the digital signal to the image frame. After receiving the complete data packet, the host system unpacks the packet according to the protocol format, and then assembles it into a complete frame according to the sequence number of the packet and hands it over to the display system for display. Because the image is displayed in frames, it must wait until a complete frame of data is spliced ​​before it can be displayed.

[0037] Specifically, in step S2, the specific steps of the sending end processing the received raw data and packaging and outputting it according to the preset protocol include:

[0038] The raw data collected by multiple cameras is transmitted to the input port and then input to the electric-to-digital converter. The electric-to-digital converter uses the least squares curve fitting method to compensate for the transmission error of the raw data and uses the A / D conversion module to convert the electrical signal into a digital signal. The converted digital signal is the original YUV format data of the camera. The YUV data is packaged by the DSP processor according to the preset protocol format. The packaged data is converted into an optical signal by the digital-to-optical converter. The converted optical signal is output to the optical fiber through the output port.

[0039] Specifically, in step S2, when the DSP processor packages the YUV data according to the preset protocol format, it is necessary to add header information before each packet of frame data, including the packet start flag, frame rate, resolution, image format, sequence number of the current packet in the frame, camera sequence number and timestamp.

[0040] The format of the entire packet data is as follows:

[0041] Start bit Frame rate Resolution Image Format Serial number Camera serial number Timestamp Image data

[0042] Specifically, in step S2, when the DSP processor packages the YUV data according to the preset protocol format, the portion of the received frame data can start packaging, and the size of the portion of the frame data that can start packaging is related to the resolution, frame rate and image format.

[0043] Assume that the camera resolution is 1920x1080, the frame rate is 25 frames, the data interval of each frame is 40 milliseconds, and the format is YUV420.

[0044] Receiving a frame and then sending it out means that the receiver sees the image from 40 milliseconds ago, which does not include the transmission delay;

[0045] If the above frame of data is split into four packets, the latency will be reduced to 10 milliseconds. Theoretically, the smaller the packet size, the lower the latency. However, the impact of frequent packet disassembly and assembly on performance must be considered comprehensively. Therefore, it is necessary to preset the delay by analyzing the resolution, frame rate and image format.

[0046] Specifically, in step S2, the receiving end processes the received data packets and unpacks them according to the preset protocol, and then assembles them into complete frame data for display. The specific steps include:

[0047] The optical fiber transmits the optical signal output from the output port of the transmitter to the input port of the receiver, and then inputs it into the optical-to-digital converter. The optical-to-digital converter converts the optical signal into a digital signal and inputs it into the DSP processor. After receiving the data, the DSP processor takes out the camera number in the header information and groups the data according to the camera number. In each group, the data is buffered in the order of the data packets in each frame. After the last packet of data is buffered, the complete frame and header information are given to the display system for display.

[0048] Furthermore, in order to avoid data loss caused by splitting the data frame, among the multiple data packets formed by splitting the data frame, two adjacent data packets have part of the same data, and only one of the two parts of the same data needs to be retained when assembling the packets.

[0049] Specifically, for the split data packets, the data at the beginning of the data packet is the same as the data at the end of the previous data packet; and the data at the end of the data packet is the same as the data at the beginning of the next data packet.

[0050] The length of the same data is preset according to the resolution, frame rate and image format.

[0051] When assembling packets, the DSP processor verifies whether the same data area of ​​two adjacent packets is complete. As long as one of the same data areas of two adjacent data packets is complete, the complete data area is retained, the same data area of ​​the other adjacent data packet is removed, and the retained data redundant area is connected between the two adjacent data packets to complete the packet assembly. Specifically, in step S2, the complete frame data output by the output port is transmitted to the terminal device and displayed by the display of the terminal device.

[0052] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for processing multi-channel camera data, characterized in that: The following steps are involved: S1: construct a multi-channel camera data processing framework, including multi-channel cameras, a transmitter, an optical fiber and a receiver, wherein the multi-channel cameras are connected to the transmitter input, the transmitter output is connected to the optical fiber input, and the optical fiber output is connected to the receiver input; S2: Multiple cameras transmit the collected raw data to the transmitter, which processes the received raw data and packages it for output according to the preset protocol. The optical fiber transmits the output data packets to the receiver, which processes the received data packets and unpacks them according to the preset protocol, and then assembles them into complete frame data for display.

2. A multi-channel camera data processing method according to claim 1, characterized in that: In step S1, the specific steps of constructing multiple cameras, a transmitting end and an optical fiber include: The transmitting end includes an input port, an electric-to-digital converter, a DSP processor, a digital-to-optical converter and an output port. The input port input end is connected to the cable for transmitting data from multiple cameras, the input port output end is connected to the electric-to-digital converter input end, the electric-to-digital converter output end is connected to the DSP processor input end, the DSP processor output end is connected to the digital-to-optical converter input end, the digital-to-optical converter output end is connected to the output port input end, and the output port output end is connected to the optical fiber input end.

3. A multi-channel camera data processing method according to claim 1, characterized in that: In step S1, the specific steps of constructing the optical fiber and the receiving end include: The receiving end includes an input port, an optical-to-digital converter, a DSP processor and an output port. The input port input end is connected to the optical fiber output end, the input port output end is connected to the optical-to-digital converter input end, the optical-to-digital converter output end is connected to the DSP processor input end, the DSP processor output end is connected to the output port input end, and the output port output end is connected to the terminal device input end.

4. The method for processing multi-channel camera data according to claim 1, characterized in that: In step S1, the specific steps of constructing the optical fiber and the receiving end include: The receiving end includes an input port, an optical digital converter and an output port. The input port input end is connected to the optical fiber output end, the input port output end is connected to the optical digital converter input end, the optical digital converter output end is connected to the output port input end, and the output port output end is connected to the host system.

5. The method for processing multi-channel camera data according to claim 1, characterized in that: In step S2, the specific steps of the sending end processing the received raw data and packaging and outputting it according to the preset protocol include: The raw data collected by multiple cameras is transmitted to the input port, and then input into the electric-to-digital converter. The electric-to-digital converter converts the electrical signal into a digital signal. The converted digital signal is the original YUV format data of the camera. The YUV data is packaged by the DSP processor according to the preset protocol format. The packaged data is converted into an optical signal by the digital-to-optical converter. The converted optical signal is output to the optical fiber through the output port.

6. A multi-channel camera data processing method according to claim 5, characterized in that: In step S2, when the DSP processor packages the YUV data according to the preset protocol format, it is necessary to add header information before each packet of frame data, including the start flag of the packet, frame rate, resolution, image format, sequence number of the current packet in the frame, camera sequence number and timestamp.

7. The method for processing multi-channel camera data according to claim 5, characterized in that: In step S2, when the DSP processor packages the YUV data according to the preset protocol format, the portion of the received frame data can be packaged. The size of the portion of the frame data that can be packaged is related to the resolution, frame rate and image format.

8. A multi-channel camera data processing method according to claim 7, characterized in that: Among the multiple data packets formed by splitting the data frame, two adjacent data packets have some data that are the same, the starting part of the data packet is the same as the data at the end part of the data packet before the data packet; the end part of the data packet is the same as the data at the starting part of the data packet after the data packet; The length of the identical portion of data is preset according to the resolution, frame rate, and image format.

9. A multi-channel camera data processing method according to claim 8, characterized in that: In step S2, the DSP processor verifies whether the same data area of ​​two adjacent data packets is complete. As long as one of the same data areas of two adjacent data packets is complete, the complete data area is retained, the same data area of ​​the other adjacent data packet is removed, and the retained data redundant area is connected between the two adjacent data packets to complete the packet assembly.

10. The method for processing multi-channel camera data according to claim 1, characterized in that: In step S2, the receiving end processes the received data packets and unpacks them according to the preset protocol, and then assembles them into complete frame data for display. The specific steps include: The optical fiber transmits the optical signal output from the output port of the transmitter to the input port of the receiver, and then inputs it into the optical-to-digital converter. The optical-to-digital converter converts the optical signal into a digital signal and inputs it into the DSP processor. The DSP processor parses the data according to the preset protocol, and then assembles it into a complete frame data according to the timestamp and sequence number of the packet, and inputs it to the output port. The output port outputs the complete frame data for display.

Images