Video processing device and equipment, video data transmission method, medium and product

Abstract

The invention discloses a video processing device and equipment, a video data transmission method, a medium and a product, and relates to the technical field of industrial vision, the video processing device comprises a plurality of transmission channels, a data sending side and a data receiving side, and the data sending side comprises a distribution module. The distribution module is used for dividing a to-be-processed video data stream into a plurality of sub-data streams according to a set distribution mode, and distributing the plurality of divided sub-data streams to a plurality of transmission channels for transmission; the data receiving side comprises a merging module, the merging module is connected with the distribution module through the plurality of transmission channels, and the merging module is used for respectively and mutually matching the received mark bits of the sub-data streams of the plurality of transmission channels; and when the mark bits of the sub-data streams of the plurality of transmission channels are successfully matched, caching and combining the received sub-data streams of the plurality of transmission channels according to the corresponding distribution mode. The method and the device are used for solving the technical problem that the transmission requirements of high-resolution and high-line-frequency scenes cannot be met.

Description

Technical Field

[0001] The present application relates to the field of industrial vision technology, and in particular to a video processing device, equipment, video data transmission method, medium and product. Background Art

[0002] Most existing video transmission solutions can only transmit one channel of data. In some high-resolution and high-line-rate application scenarios, there is a problem of insufficient bandwidth. If multiple channels of data are to be transmitted, there will also be problems such as difficulty in data synchronization adjustment. Summary of the invention

[0003] The main purpose of the present application is to provide a video processing device, equipment, video data transmission method, medium and product, aiming to solve the technical problem of not being able to meet the transmission requirements of high-resolution and high-line frequency scenarios.

[0004] To achieve the above object, the present application proposes a video processing device, the video processing device comprising:

[0005] Multiple transmission channels;

[0006] The data transmission side includes a distribution module, which is used to divide the video data stream to be processed into multiple sub-data streams according to a set distribution mode, and distribute the multiple sub-data streams to multiple transmission channels for transmission;

[0007] The data receiving side includes a merging module, which is connected to the distribution module via the multiple transmission channels. The merging module is used to match the identification bits of the sub-data streams of the multiple transmission channels respectively; when the identification bits of the sub-data streams of the multiple transmission channels are matched successfully, the sub-data streams of the multiple transmission channels are cached and merged according to the corresponding distribution mode.

[0008] In one embodiment, the data transmission side is provided with a plurality of transmission modules, and the distribution module is connected to the first ends of the plurality of transmission channels through the plurality of transmission modules, and is used to divide the to-be-processed video data stream into a plurality of sub-data streams, and distribute the divided sub-data streams to the plurality of transmission channels for transmission;

[0009] The data receiving side is provided with a plurality of receiving modules, and the merging module is connected to the second ends of the plurality of transmission channels through the plurality of receiving modules, and is used for receiving the sub-data streams transmitted by the plurality of transmission channels.

[0010] In one embodiment, each of the sending modules includes a first mipi interface and a serializer connected in sequence, and the distribution module is connected to multiple first mipi interfaces, and is used to divide the video data stream to be processed into multiple mipi sub-data streams; multiple serializers are connected to the first ends of multiple transmission channels, and are used to convert the divided multiple mipi sub-data streams into GMSL serial sub-data streams and then distribute them to multiple transmission channels for transmission;

[0011] Each of the receiving modules includes a deserializer and a second mipi interface connected in sequence, multiple second mipi interfaces are connected to the merging module, and multiple deserializers are connected to the second ends of multiple transmission channels, and are used to deserialize the GMSL serial sub-data streams transmitted by the multiple transmission channels and then convert them into mipi sub-data streams.

[0012] In one embodiment, the video data stream to be processed is in frames, and each frame includes a plurality of lines of pixel data stream;

[0013] The distribution module is used to divide the data stream in each row of pixel sub-data stream into an odd pixel sub-data stream and an even pixel sub-data stream, and output the odd pixel sub-data stream to one or more of the multiple transmission channels, and output the even pixel sub-data stream to another one or more of the multiple transmission channels;

[0014] Alternatively, the distribution module is used to divide the data stream in each frame into an odd-row pixel sub-data stream and an even-row pixel sub-data stream, and output the odd-row pixel sub-data stream to one or more of the multiple transmission channels, and output the even-row pixel sub-data stream to another one or more of the multiple transmission channels;

[0015] Alternatively, the distribution module is used to divide each row of pixel sub-data stream into a left sub-data stream and a right sub-data stream, and output the left sub-data stream to one or more of the multiple transmission channels, and output the right sub-data stream to another one or more of the multiple transmission channels.

[0016] In one embodiment, the data receiving side further includes a cache module, and each of the transmission channels is provided with the cache module;

[0017] When the data stream in each row of pixel sub-data stream is divided into an odd pixel sub-data stream and an even pixel sub-data stream, or the data stream in each frame is divided into an odd pixel sub-data stream and an even pixel sub-data stream, or each row of pixel sub-data stream is divided into a left sub-data stream and a right sub-data stream, the buffer module is used to buffer the received sub-data stream of each transmission channel respectively;

[0018] When the data stream in each frame is divided into odd-row pixel sub-data streams and even-row pixel sub-data streams, or each row of pixel sub-data streams is divided into a left sub-data stream and a right sub-data stream, the cache module is used to cache the sub-data stream that is output later among the overlapping sub-data streams in multiple transmission channels.

[0019] In addition, to achieve the above purpose, the present application also proposes a video data transmission method, which is applied to the video processing device as described above, and the video data transmission method comprises the following steps:

[0020] The data sending side divides the video data stream to be processed into multiple sub-data streams according to the set distribution mode, and distributes the multiple sub-data streams to multiple transmission channels for transmission;

[0021] The data receiving side matches the identification bits of the sub-data streams of the multiple transmission channels received respectively with each other;

[0022] When the identification bits of the sub-data streams of the plurality of transmission channels are matched successfully, the received sub-data streams of the plurality of transmission channels are buffered according to the corresponding distribution mode.

[0023] In one embodiment, the video data stream to be processed is divided into frames, each frame includes multiple lines of pixel data streams, and the distribution mode includes:

[0024] Dividing the data stream in each row of pixel sub-data streams into an odd pixel sub-data stream and an even pixel sub-data stream, outputting the odd pixel sub-data stream to one or more of the plurality of transmission channels, and outputting the even pixel sub-data stream to another one or more of the plurality of transmission channels;

[0025] Dividing the data stream in each frame into an odd-row pixel sub-data stream and an even-row pixel sub-data stream, outputting the odd-row pixel sub-data stream to one or more of the multiple transmission channels, and outputting the even-row pixel sub-data stream to another one or more of the multiple transmission channels;

[0026] Each row of pixel sub-data streams is divided into a left sub-data stream and a right sub-data stream, the left sub-data stream is output to one or more of the multiple transmission channels, and the right sub-data stream is output to another one or more of the multiple transmission channels.

[0027] In one embodiment, the step of the data receiving side matching the identification bits of the received sub-data streams of the multiple transmission channels with each other comprises:

[0028] When dividing the data stream in each row of pixel sub-data stream into odd pixel sub-data stream and even pixel sub-data stream, or dividing each row of pixel sub-data stream into left sub-data stream and right sub-data stream, or dividing the data stream in each frame into odd row pixel sub-data stream and even row pixel sub-data stream, the identification bits of the sub-data streams of multiple transmission channels are determined by frame number and row number, and the data receiving side matches the row numbers of the sub-data streams in the same frame among the received sub-data streams of the multiple transmission channels.

[0029] In one embodiment, when dividing a data stream in each row of pixel sub-data streams into an odd pixel sub-data stream and an even pixel sub-data stream, or dividing each row of pixel sub-data streams into a left sub-data stream and a right sub-data stream, the data receiving side performs a step of matching row numbers of sub-data streams in the same frame among the received sub-data streams of the multiple transmission channels, specifically including:

[0030] Detecting the frame of the sub-data stream, when detecting the rising edge of the signal for indicating the first pixel sub-data stream in the sub-data stream of each transmission channel, obtaining the indication signal of the transmission channel for indicating the sub-data stream and caching the frame number of the sub-data stream;

[0031] After obtaining the indication signals of all transmission channels, the frame numbers of the sub-data streams of the multiple transmission channels are compared;

[0032] When the frame numbers of the sub-data streams of the multiple transmission channels are the same, determining that the sub-data streams of the multiple transmission channels received by the data receiving side are located in the same frame; when the frame numbers of the sub-data streams of the multiple transmission channels are different, re-performing the step of detecting the frame of the sub-data stream;

[0033] Detecting the row numbers of multiple sub-data streams in the same frame, and when the row numbers of the multiple sub-data streams in the same frame are the same, determining that the multiple sub-data streams in the same frame are in the same row, and outputting a counting instruction for counting the current number of rows of the sub-data streams; when the row numbers of the multiple sub-data streams in the same frame are different, determining that the multiple sub-data streams in the same frame are not in the same row, discarding all the sub-data streams in the frame, and re-performing the step of detecting the frame of the sub-data streams;

[0034] When the current number of rows counted does not match the actual number of rows of the sub-data stream, the step of detecting the row numbers of multiple sub-data streams located in the same frame is re-executed; when the current number of rows counted for the sub-data stream matches the actual number of rows of the sub-data stream, it is determined that the sub-data streams located in the same frame and with the same row number in the sub-data streams of multiple transmission channels are matched.

[0035] In one embodiment, when the data stream in each frame is divided into odd-row pixel sub-data streams and even-row pixel sub-data streams, the data receiving side performs a step of matching row numbers of sub-data streams in the same frame among the sub-data streams of the received multiple transmission channels, specifically including:

[0036] Detecting the frame of the sub-data stream, when detecting the rising edge of the signal for indicating the first pixel sub-data stream in the sub-data stream of each transmission channel, obtaining the indication signal of the transmission channel for indicating the sub-data stream and caching the frame number of the sub-data stream;

[0037] After obtaining the indication signals of all transmission channels, the frame numbers of the sub-data streams of the multiple transmission channels are compared;

[0038] When the frame numbers of the sub-data streams of the multiple transmission channels are the same, determining that the sub-data streams of the multiple transmission channels received by the data receiving side are located in the same frame; when the frame numbers of the sub-data streams of the multiple transmission channels are different, re-performing the step of detecting the frame of the sub-data stream;

[0039] Detecting the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel; when the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel are the same, determining that the multiple sub-data streams are in adjacent rows in the same frame, and outputting a counting instruction for counting the current number of rows of the sub-data streams in the corresponding transmission channel; when the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel are different, determining that the multiple sub-data streams are not in adjacent rows in the same frame, discarding all the sub-data streams of the frame and then re-performing the step of detecting the frame of the sub-data streams;

[0040] When the sum of the current number of rows of all transmission channels counted does not match the actual number of rows of the sub-data streams, the step of detecting the row numbers of multiple sub-data streams located in the same frame is re-executed; when the sum of the current number of rows of all transmission channels counted matches the actual number of rows of the sub-data streams, it is determined that the sub-data streams located in the same frame and with adjacent row numbers in the sub-data streams of the multiple transmission channels are matched.

[0041] In one embodiment, the step of caching the received sub-data streams of the multiple transmission channels according to the corresponding distribution mode specifically includes:

[0042] The received sub-data streams of each transmission channel are cached respectively according to the corresponding distribution mode.

[0043] In one embodiment, the step of caching the received sub-data streams of the multiple transmission channels according to the corresponding distribution mode specifically includes:

[0044] When dividing the data stream in each frame into odd-row pixel sub-data streams and even-row pixel sub-data streams, or dividing each row pixel sub-data stream into a left sub-data stream and a right sub-data stream, the sub-data stream output later among the overlapping sub-data streams in multiple transmission channels is cached.

[0045] In one embodiment, the data sending side divides the to-be-processed video data stream into a plurality of sub-data streams according to a set distribution mode, and distributes the divided sub-data streams to a plurality of transmission channels for transmission, specifically comprising:

[0046] The data sending side divides the video data stream to be processed into multiple mipi sub-data streams according to the set distribution mode, converts the divided multiple mipi sub-data streams into GMSL serial sub-data streams and distributes them to multiple transmission channels for transmission;

[0047] Before executing the step of matching the identification bits of the received sub-data streams of the multiple transmission channels at the data receiving side, the video data transmission method further includes the following steps:

[0048] The data receiving side deserializes the GMSL serial sub-data streams of multiple transmission channels and converts them into mipi sub-data streams for reception.

[0049] In addition, to achieve the above-mentioned purpose, the present application also proposes a video processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the video data transmission method as described above.

[0050] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the video data transmission method described above are implemented.

[0051] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, characterized in that the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the video data transmission method described above are implemented.

[0052] One or more technical solutions proposed in this application have at least the following technical effects:

[0053] The distribution module on the data sending side is used to divide the video data stream to be processed into multiple sub-data streams according to the set distribution mode, and use multiple transmission channels for synchronous transmission, thereby significantly improving the bandwidth of data transmission and meeting the transmission requirements of high-resolution and high-line frequency scenarios; after receiving the sub-data streams, the merging module on the data receiving side first matches the identification bits of each sub-data stream with each other to ensure that all sub-data streams come from the same frame or the same line of the same frame of the same video data stream to be processed; after the match is successful, the merging module caches and merges the multiple sub-data streams received according to the corresponding distribution mode;

[0054] Through marker bit matching and cache processing, data dislocation and loss problems can be effectively avoided, ensuring the integrity and accuracy of the video data stream, thereby improving the stability of video processing;

[0055] Through the coordinated work of multiple transmission channels and distribution modules, the bandwidth resources of multiple transmission channels can be fully utilized to achieve high-speed transmission of the video data stream to be processed; the design of the video processing device allows the configuration of more transmission channels and distribution modules, and merging modules to meet the needs of higher-resolution video data stream transmission, thereby improving the flexibility and practicality of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0058] Figure 1 A schematic diagram of modules provided for an embodiment of a video processing device of the present application;

[0059] Figure 2 A schematic diagram of a distribution mode provided for an embodiment of a video processing device of the present application;

[0060] Figure 3 A schematic diagram of a distribution mode provided for another embodiment of the video processing device of the present application;

[0061] Figure 4 A schematic diagram of a distribution mode provided for another embodiment of the video processing device of the present application;

[0062] Figure 5 A flowchart provided for an embodiment of the video data transmission method of the present application;

[0063] Figure 6 One of the flowcharts provided for step S200 of an embodiment of the video data transmission method of the present application;

[0064] Figure 7 A second flowchart provided for step S200 of an embodiment of the video data transmission method of the present application;

[0065] Figure 8 One of the flowcharts provided for step S200 of another embodiment of the video data transmission method of the present application;

[0066] Fig. 9 The second flowchart is provided for step S200 of another embodiment of the video data transmission method of the present application.

[0067] Description of Figure Numbers:

[0068] 100, transmission channel;

[0069] 200, data transmission side; 210, distribution module; 221, first mipi interface; 222, serializer;

[0070] 300, data receiving side; 310, merging module; 321, second mipi interface; 322, deserializer.

[0071] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0072] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0073] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0074] In the related art, most video transmission schemes can only transmit one channel of data. In some high-resolution and high-line frequency application scenarios, there is a problem of insufficient bandwidth. If multiple channels of data are to be transmitted, there will be problems such as difficulty in data synchronization adjustment. Taking the mipi (Mobile Industry Processor Interface) interface transmission as an example, the mipi interface is an interface specifically used to connect image sensors and image processors. It is widely used in smart phones, industrial cameras, automotive electronics, security monitoring and other devices that require high-performance image transmission. The mipi interface uses differential signal transmission and has the characteristics of high bandwidth and low power consumption. Each transmission channel 100 can even support a transmission rate of up to 3.2Gbps. Through parallel transmission of multiple lane transmission channels, mipi can achieve very high data bandwidth. In general, the mipi interface has become the mainstream interface standard in modern image sensor systems due to its advantages of high performance, high bandwidth and low power consumption, and can be applied to sensor data transmission of most resolution specifications.

[0075] Although the transmission rate of the mipi interface in related technologies is already very high, most of them transmit one or more video data streams through one interface. Taking large-target area array sensors and 8K and 16K high-line-rate line array sensors in high-resolution and high-line-rate application scenarios as examples, mipi still has the problem of insufficient bandwidth: large-target area array sensors usually mean higher resolution, resulting in a very large amount of image data. When this data needs to be transmitted through the mipi interface, the demand for bandwidth will increase dramatically. If the bandwidth of a single mipi interface is not enough to support such a large amount of data, it will affect the image quality and real-time performance; and 8K and 16K high-line-rate line array sensors mean that many lines of image data can be scanned and transmitted per second, and the traditional one-way mipi cannot meet the transmission requirements.

[0076] To meet the transmission requirements of high resolution and high line frequency scenarios, refer to Figures 1 to 9 The present application proposes a video processing device, equipment, video data transmission method, medium and product, wherein the device is specifically a video processing device, the medium is specifically a storage medium such as a computer-readable storage medium, and the product is specifically a computer program product, and the video data transmission method of the following embodiment is mainly implemented through the video processing device.

[0077] The video processing device includes a plurality of transmission channels 100 , a data sending side 200 and a data receiving side 300 .

[0078] The data sending side 200 includes a distribution module 210, which is used to divide the video data stream to be processed into multiple sub-data streams according to a set distribution mode, and distribute the multiple sub-data streams to multiple transmission channels 100 for transmission.

[0079] The data receiving side 300 includes a merging module 310, which is connected to the distribution module 210 via multiple transmission channels 100. The merging module 310 is used to match the identification bits of the sub-data streams of the multiple transmission channels 100 received respectively with each other; when the identification bits of the sub-data streams of the multiple transmission channels 100 are successfully matched, the sub-data streams of the multiple transmission channels 100 received are cached according to the corresponding distribution mode.

[0080] It is understandable that the transmission channel 100 refers to a physical or logical path for data transmission. In the video processing device, these channels can be mipi interfaces, GMSL (Gigabit Multimedia Serial Link) links, etc., which are responsible for transmitting the sub-data streams generated by the distribution module 210 from the data sending side 200 to the data receiving side 300.

[0081] The data sending side 200 is the part of the video processing device that is responsible for (generating or receiving the data stream to be processed transmitted from the outside and) sending the data stream to be processed. Its main function is to divide the video data stream to be processed into multiple sub-data streams, and distribute these sub-data streams to multiple transmission channels 100 for transmission according to the set distribution mode. The data receiving side 300 is the part of the video processing device that is used to receive and process the sub-data streams transmitted from these transmission channels 100. The design and implementation of the data sending side 200 and the data receiving side 300, as well as the number and implementation method of the transmission channels 100, can be flexibly adjusted according to actual needs to adapt to different video data stream transmission requirements.

[0082] The distribution module 210 is a key component of the data transmission side 200. Its main function is to divide the video data stream to be processed into two, three or other multiple sub-data streams according to the preset distribution mode, and distribute part of these sub-data streams to at least one of the multiple transmission channels 100, and distribute the other parts of these sub-data streams to at least one of the multiple transmission channels 100, so that multiple sub-data streams can be distributed to multiple transmission channels 100 synchronously. Multiple transmission channels 100 are used for synchronous transmission, thereby significantly improving the bandwidth of data transmission and meeting the transmission requirements of high-resolution and high-line frequency scenarios (such as large-target area array sensors and high-line frequency linear array sensors). The merging module 310 is a key component of the data receiving side 300. It is mainly responsible for synchronizing, splicing and restoring the received multiple sub-data streams according to the transmission order of each sub-data stream according to the different distribution modes adopted to generate a complete video data stream. Taking the odd-even pixel transmission mode in which the distribution module 210 is used to divide the video data stream to be processed into an odd-pixel sub-data stream and an even-pixel sub-data stream as an example, the merging module 310 is used to synchronize, splice and restore the odd-pixel sub-data stream and the even-pixel sub-data stream received from different transmission channels 100 according to the transmission order of each sub-data stream, so as to generate a complete video data stream that is identical to the video data stream to be processed (original video data stream), thereby further obtaining a video identical to the original video.

[0083] The functions of the merging module 310 include but are not limited to detection function and cache function. The detection function is used to match the identification bits of the sub-data streams of the received multiple transmission channels 100 to ensure that they come from the same frame or the same line of the same frame of the same original video data stream (video data stream to be processed); the cache function is used to cache the received multiple sub-data streams according to the corresponding distribution mode for subsequent processing and recovery. After receiving the sub-data stream, the data receiving side 300 first matches the identification bits (such as frame number and row number) of each sub-data stream to ensure that all sub-data streams come from the same video data stream to be processed. After the match is successful, the merging module 310 caches and merges the received multiple sub-data streams according to the corresponding distribution mode to obtain a complete video data stream that is the same as the video data stream to be processed (original video data stream), thereby further obtaining a video that is the same as the original video. Through identification bit matching and cache processing, data dislocation and loss problems can be effectively avoided, the integrity and accuracy of the video data stream can be ensured, and the stability of video processing can be improved.

[0084] Through the coordinated work of multiple transmission channels 100 and distribution modules 210, the bandwidth resources of multiple transmission channels 100 can be fully utilized to achieve high-speed transmission of the video data stream to be processed. The design of the video processing device allows the configuration of more transmission channels 100, distribution modules 210, and merging modules 310 to meet the requirements of higher-resolution video data stream transmission, thereby improving the flexibility and practicality of the system.

[0085] Reference Figure 1 In one embodiment, the data transmission side 200 is provided with a plurality of transmission modules, and the distribution module 210 is connected to the first ends of the plurality of transmission channels 100 through the plurality of transmission modules, and is used to divide the video data stream to be processed into a plurality of sub-data streams, and distribute the divided sub-data streams to the plurality of transmission channels 100 for transmission. The data receiving side 300 is provided with a plurality of receiving modules, and the merging module 310 is connected to the second ends of the plurality of transmission channels 100 through the plurality of receiving modules, and is used to receive the sub-data streams transmitted by the plurality of transmission channels 100.

[0086] The sending module may include but is not limited to the following interfaces and components: mipi (Mobile Industry Processor Interface) interface, transmitter, FPD-Link IV serial interface. These sending modules are responsible for sending the video data stream to be processed from the source device to the transmission channel 100. The receiving module may include but is not limited to the following interfaces and components: mipi interface, FPD-Link IV deserializer. These receiving modules are responsible for receiving the video sub-data stream from the transmission channel 100 and performing necessary processing and conversion. Specifically, the corresponding sending module and receiving module can be set according to actual application requirements, which is not limited here. Through the reasonable setting of the sending module and the receiving module, the data transmission rate and transmission efficiency can be significantly improved, so that the video processing device or other image processing device can process data faster and meet the transmission requirements of high-resolution and high-line frequency scenarios.

[0087] This application mainly uses the mipi interface (Mobile Industry Processor Interface) as an example to illustrate, and embodiments using other sending modules and receiving modules can be referred to accordingly without repeating them one by one:

[0088] Reference Figure 1Specifically, each sending module includes a first mipi interface 221 and a serializer 222 connected in sequence, and the distribution module 210 is connected to multiple first mipi interfaces 221, and is used to divide the video data stream to be processed into multiple mipi sub-data streams; multiple serializers 222 are connected to the first ends of multiple transmission channels 100, and are used to convert the divided multiple mipi sub-data streams into GMSL (Gigabit Multimedia Serial Link) serial sub-data streams and then distribute them to multiple transmission channels 100 for transmission;

[0089] Each receiving module includes a deserializer 322 and a second mipi interface 321 connected in sequence. Multiple second mipi interfaces 321 are connected to the merging module 310. Multiple deserializers 322 are connected to the second ends of multiple transmission channels 100, and are used to deserialize the GMSL serial sub-data streams transmitted by the multiple transmission channels 100 and then convert them into mipi sub-data streams.

[0090] The distribution module 210, as the entry of the video data stream to be processed, is responsible for dividing the video data stream into multiple mipi sub-data streams, and transmitting these sub-data streams to multiple first mipi interfaces 221 respectively, in preparation for subsequent serialization processing. GMSL (Gigabit Multimedia Serial Link) supports long-distance, low-latency and high-bandwidth video transmission. The main function of the serializer 222 is to convert these mipi sub-data streams from parallel format to GMSL (Gigabit Multimedia Serial Link) serial sub-data streams, and then distribute them to multiple transmission channels 100 for transmission.

[0091] On the data receiving side 300, the deserializer 322 is the reverse process of the serializer 222, responsible for deserializing the GMSL serial sub-data streams transmitted by the multiple transmission channels 100 and converting them back into mipi sub-data streams. Subsequently, these mipi sub-data streams are further merged into the original video data stream by the merging module 310 for subsequent video processing or display. The deserializer 322 achieves data synchronization and correct parsing through an internal timing control unit to ensure the accuracy and integrity of the data. The deserializer 322 also performs error detection and correction to ensure the integrity of the data.

[0092] By dividing the video data stream to be processed into multiple mipi sub-data streams and using multiple transmission channels 100 for synchronous transmission, the bandwidth of data transmission can be significantly improved, thereby meeting the transmission requirements of high-resolution and high-frame rate scenes. At the same time, since the deserializer 322 usually performs error detection and correction to ensure the integrity of the data, the coordinated work of the serializer 222, the transmission channel 100 and the deserializer 322 can effectively avoid data misalignment and loss problems, ensure the integrity and accuracy of the processed video data stream, and thus improve the stability of video processing.

[0093] This application uses the data distribution and merging technology of transmitting the same source data through multiple MIPI interfaces, merges the processes of multiple MIPIs with the same video source, and can use the MIPI interface to transmit high-bandwidth video streams. It provides a new interface option for the transmission requirements of high-resolution and high-line frequency scenes such as large-target area array cameras and high-line frequency line array cameras, broadens the use scenarios of the MIPI interface, and adapts to future needs in the field of industrial vision.

[0094] In the present application, the number of transmission channels 100 may be two, three, four or more. In one embodiment, the video data stream to be processed is in frames, and each frame includes a plurality of lines of pixel data stream.

[0095] Reference Figure 2 As an example, the distribution module 210 is used to divide the data stream in each row of pixel sub-data stream into an odd pixel sub-data stream and an even pixel sub-data stream, and output the odd pixel sub-data stream to one or more of the multiple transmission channels 100, and output the even pixel sub-data stream to another one or more of the multiple transmission channels 100.

[0096] The odd pixel sub-data stream and the even pixel sub-data stream in each row of pixel sub-data stream are alternated, and the distribution module 210 transmits the odd pixel sub-data stream and the even pixel sub-data stream in each row of pixel sub-data stream separately through different transmission channels 100, and defines the distribution mode adopted by the distribution module 210 as the odd-even pixel transmission mode. When the transmission channel 100 includes two channels 1 and 2, and the multiple pixel sub-data streams in each row of sub-data stream are marked with p0 to p5 or in other ways, the odd pixel sub-data stream includes p1, p3, p5, etc., and the even pixel sub-data stream includes p0, p2, p4, etc., one of the channels 1 and 2 is used to transmit the odd pixel sub-data stream in each row of pixel sub-data streams such as p1, p3, p5, etc., and the other is used to transmit the even pixel sub-data stream in each row of pixel sub-data streams such as p0, p2, p4, etc.

[0097] The odd-even pixel transmission mode can be used to reduce the amount of data in each transmission channel 100 when transmitting each row of pixel sub-data streams, reduce the bandwidth requirement of each transmission channel 100, and ensure that more data streams can be transmitted under limited mipi bandwidth. The data receiving side 300 is used to merge the non-continuous pixel sub-data streams of each transmission channel 100 to obtain a complete row of pixel sub-data streams such as p0 to p5 for output.

[0098] Reference Figure 3 As another example, the distribution module 210 is used to divide the data stream in each frame into an odd-row pixel sub-data stream and an even-row pixel sub-data stream, and output the odd-row pixel sub-data stream to one or more of the multiple transmission channels 100, and output the even-row pixel sub-data stream to another one or more of the multiple transmission channels 100.

[0099] The odd-row pixel sub-data stream and the even-row pixel sub-data stream of the multiple-row pixel sub-data stream are alternated, and the distribution module 210 transmits the odd-row pixel sub-data stream and the even-row pixel sub-data stream of the multiple-row pixel sub-data stream separately through different transmission channels 100, and defines the distribution mode adopted by the distribution module 210 as the odd-even row transmission mode. When the transmission channel 100 includes two channels 1 and 2, and the multiple pixel sub-data streams of each row sub-data stream are marked with p0 to p5 or in other ways, the channels 1 and 2 are used to simultaneously transmit the adjacent multiple rows of pixel sub-data streams.

[0100] The odd-even row transmission mode can ensure that more data streams can be transmitted under limited mipi bandwidth. Since each row of pixel sub-data streams is complete, when the data receiving side 300 synchronizes the data streams of the two transmission channels 100, it only needs to ensure that the row pixel sub-data streams in the same frame with the same frame number and transmitted by different transmission channels 100 are adjacent, which effectively reduces the complexity of synchronization.

[0101] Reference Figure 4 As another example, the distribution module 210 is used to divide each row of pixel sub-data stream into a left sub-data stream and a right sub-data stream, and output the left sub-data stream to one or more of the multiple transmission channels 100, and output the right sub-data stream to another one or more of the multiple transmission channels 100.

[0102] The distribution module 210 transmits the left sub-data stream and the right sub-data stream of each row of pixel sub-data stream separately through different transmission channels 100, and defines the distribution mode adopted by the distribution module 210 as a left-right partial transmission mode. When the transmission channel 100 includes two channels 1 and 2, and the multiple pixel sub-data streams of each row of sub-data stream are marked with p0 to p5 or in other ways, the left sub-data stream includes p0, p1, p2, etc., and the right sub-data stream includes p3, p4, p5, etc., one of the channels 1 and 2 is used to transmit the left sub-data stream of each row of pixel sub-data streams such as p0, p1, p2, etc., and the other is used to transmit the right sub-data stream of each row of pixel sub-data streams such as p3, p4, p5, etc.

[0103] The left-right partial transmission mode can be used to reduce the amount of data in each transmission channel 100 when transmitting each row of pixel sub-data streams, reduce the bandwidth requirements of each transmission channel 100, and ensure that more data streams can be transmitted under limited mipi bandwidth. Dividing each row of pixel sub-data streams into a left sub-data stream and a right sub-data stream can flexibly adapt to video streams of different resolutions, and adjust the size of the left sub-data stream and the right sub-data stream as needed. The data receiving side 300 is used to merge the left sub-data stream and the right sub-data stream of the same row of pixel sub-data streams in different transmission channels 100 to obtain a complete pixel sub-data stream output.

[0104] In one embodiment, the data receiving side 300 further includes a cache module, and each transmission channel 100 is provided with a cache module. The cache module is provided to ensure that two, three or other multiple sub-data streams are kept synchronized during the transmission process. Since the transmission speed and transmission time of data in different transmission channels 100 may be different, the cache can temporarily store the sub-data stream of at least one of the transmission channels 100, so as to ensure that the sub-data streams transmitted by the multiple transmission channels can be synchronously merged and processed at the data receiving side.

[0105] The embodiments of the present application are mainly described by taking the distribution module 210 as an odd-even pixel transmission mode in which the data stream in each row of pixel sub-data stream is divided into an odd pixel sub-data stream and an even pixel sub-data stream, or an odd-even row transmission mode in which the data stream in each frame is divided into an odd row pixel sub-data stream and an even row pixel sub-data stream, or a left-right division transmission mode in which each row of pixel sub-data stream is divided into a left sub-data stream and a right sub-data stream as examples:

[0106] As an example, the buffer module is used to buffer the received sub-data streams of each transmission channel 100 respectively.

[0107] As another example, the buffer module is used to buffer the sub-data stream that is output later among the overlapping sub-data streams in the multiple transmission channels 100 .

[0108] When the data stream in each row of pixel sub-data stream is divided into odd pixel sub-data stream and even pixel sub-data stream, because the original video stream bandwidth is larger than the mipi bandwidth, the original inter-line interval is insufficient to send EOT and SOT, where SOT (Start of Transmission) is a signal used to mark the starting position of data transmission in the mipi interface protocol; EOT (End of Transmission) is a signal used to mark the end position of data transmission in the mipi interface protocol. Therefore, after sampling the odd and even pixels, each channel needs to add a fifo to convert non-continuous pixels into continuous pixel transmission to increase the inter-line interval transmission EOT and SOT, and use the cache module to cache these non-continuous pixel sub-data streams of each transmission channel 100 respectively; wherein, (First-In-First-Out, first-in-first-out queue) is used as a data storage and management structure to realize the cache function in the data stream. Specifically, each transmission channel adds a fifo for caching pixels, which are output in sequence when reading, so that the cached pixel sub-data streams can be output in sequence on the data receiving side 300 to ensure the integrity and correctness of data transmission.

[0109] When the data stream in each frame is divided into odd-row pixel sub-data streams and even-row pixel sub-data streams, because the inter-row spacing between each row of pixel sub-data is sufficient (such as when the MIPI interface is used, the inter-row spacing is sufficient to meet the requirements of the MIPI physical layer), it can be directly transmitted. However, considering that the hardware delays of different transmission channels 100 are different, the transmission time of the row pixel sub-data stream may be different, resulting in the possibility of overlap of the sub-data streams of multiple transmission channels 100. Therefore, the sub-data streams of each transmission channel 100 received can be cached separately through the cache module, so that when the data is spliced ​​through the merging module 310, the sub-data streams of all transmission channels 100 are synchronized to avoid data misalignment; or, the sub-data streams output later in the overlapping sub-data streams in multiple transmission channels 100 can also be cached through the cache module. When the sub-data streams overlap, the cache module will give priority to caching the sub-data streams of the channel that outputs the sub-data stream later to ensure the integrity and correctness of the data. Specifically, each transmission channel 100 is added with a fifo, and if the sub-data streams transmitted by multiple transmission channels 100 overlap, it is used to cache the sub-data stream output later, and the sub-data stream output earlier can be directly sent without buffering, so that the pixel sub-data stream of each frame can be output in sequence at the data receiving side 300.

[0110] When each row of pixel sub-data stream is divided into a left sub-data stream and a right sub-data stream, each channel is spaced about half a row, and the interval of each transmission channel 100 is usually sufficient (such as when the mipi interface is used, the interval is sufficient to meet the requirements of the mipi physical layer), so it can be directly transmitted. However, considering that the hardware delays of different transmission channels 100 are different, the transmission time of the left sub-data stream and the right sub-data stream may be different, resulting in the left sub-data stream and the right sub-data stream of the same row of pixel sub-data streams being overlapped when transmitted through different transmission channels 100, so the sub-data streams of each transmission channel 100 received can be cached separately by the cache module, so that when the data is spliced ​​by the merging module 310, the sub-data streams of all transmission channels 100 are synchronized to avoid data misalignment; or, the sub-data streams output later in the overlapping sub-data streams in multiple transmission channels 100 can also be cached by the cache module. When the sub-data streams overlap, the cache module will give priority to caching the sub-data streams of the channel that outputs the sub-data stream later to ensure the integrity and correctness of the data. Specifically, a fifo is added to each transmission channel 100. If the sub-data streams transmitted by multiple transmission channels 100 overlap, it is used to cache the sub-data stream output later, and the sub-data stream output earlier can be directly sent without buffering, so that the pixel sub-data stream of each row can be output in sequence at the data receiving side 300.

[0111] It should be noted that, in the embodiment of the present application, the number of transmission channels 100 may be two, three or more, and the cache module is used to cache the sub-data stream output later among the overlapping sub-data streams in the multiple transmission channels 100. When the number of transmission channels 100 is two, the cache module is used to cache the sub-data stream output later among the overlapping sub-data streams in the two transmission channels 100; when the number of transmission channels 100 is not less than three, the cache module is used to cache at least two sub-data streams output later among the overlapping sub-data streams in not less than three transmission channels 100.

[0112] Reference Figures 5 to 9 The present application also proposes a video data transmission method, which is applied to the above video processing device. The video data transmission method specifically includes the following steps:

[0113] Step S100: The data sending side divides the video data stream to be processed into multiple sub-data streams according to the set distribution mode, and distributes the multiple sub-data streams to multiple transmission channels for transmission.

[0114] The data transmission side 200 divides the video data stream to be processed into multiple sub-data streams according to a preset distribution mode (such as odd-even pixel transmission mode, odd-even line transmission mode, left-right division transmission mode, etc.). This process is intended to use multiple transmission channels for synchronous transmission, optimize data transmission efficiency, reduce the load of a single transmission channel 100, and thus significantly increase the bandwidth of data transmission to meet the transmission requirements of high-resolution and high-line frequency scenarios.

[0115] Step S200: The data receiving side matches the identification bits of the received sub-data streams of the multiple transmission channels with each other.

[0116] After receiving the sub-data streams of multiple transmission channels 100, the data receiving side 300 will match the identification bits of these sub-data streams. The purpose of this step is to ensure that the sub-data streams of all transmission channels 100 are complete and arrive in a predetermined order, in preparation for subsequent caching and merging. Specifically, the identification bit matching includes detecting the frame number and the row number to ensure that all sub-data streams are from the same frame of the same video data stream to be processed, or from the same row in the same frame, or adjacent rows in the same frame.

[0117] Step S300: When the identification bits of the sub-data streams of the multiple transmission channels are matched successfully, the received sub-data streams of the multiple transmission channels are cached according to the corresponding distribution mode.

[0118] Due to the possible transmission time differences between different transmission channels 100, or due to reasons such as data packet scheduling, the sub-data streams may be discontinuous or overlapping when arriving at the data receiving side 300. The cache module temporarily stores this data until all data arrives and can be output in sequence. The use of the cache module ensures the synchronization and integrity of the data and improves the stability and reliability of the system. The cache module can not only perform error detection and correction, but also ensure the integrity and correctness of the received sub-data streams. If data errors or losses are detected, the cache module can try to recover the data from other transmission channels 100 or previous caches to avoid data dislocation and loss.

[0119] Through the coordinated work of multiple transmission channels 100 and distribution modules 210, the bandwidth resources of multiple transmission channels 100 can be fully utilized to achieve high-speed transmission of the video data stream to be processed; the design of the video processing device allows the configuration of more transmission channels 100 and distribution modules 210, merging modules 310 to meet the needs of higher-resolution video data stream transmission, thereby improving the flexibility and practicality of the system.

[0120] In one embodiment, the video data stream to be processed is divided into frames, each frame includes multiple lines of pixel data streams, and the distribution mode includes but is not limited to any one or more of the following modes:

[0121] Reference Figure 2 As an example, the distribution mode includes dividing the data stream in each row of pixel sub-data stream into an odd pixel sub-data stream and an even pixel sub-data stream, outputting the odd pixel sub-data stream to one or more of the multiple transmission channels 100, and outputting the even pixel sub-data stream to another one or more of the multiple transmission channels 100.

[0122] This distribution mode is defined as an odd-even pixel transmission mode, which can be used to reduce the amount of data in each transmission channel 100 when transmitting each row of pixel sub-data streams, reduce the bandwidth requirement of each transmission channel 100, and ensure that more data streams can be transmitted under limited mipi bandwidth. The data receiving side 300 is used to merge the non-continuous pixel sub-data streams of each transmission channel 100 to obtain a complete row of pixel sub-data stream output.

[0123] Reference Figure 3 As another example, the distribution mode includes dividing the data stream in each frame into an odd-row pixel sub-data stream and an even-row pixel sub-data stream, outputting the odd-row pixel sub-data stream to one or more of the multiple transmission channels 100, and outputting the even-row pixel sub-data stream to another one or more of the multiple transmission channels 100.

[0124] This distribution mode is defined as an odd-even row transmission mode, which can ensure that more data streams can be transmitted under limited mipi bandwidth. Since each row of pixel sub-data streams is complete, when the data receiving side 300 synchronizes the data streams of the two transmission channels 100, it only needs to ensure that the frame numbers are the same, which effectively reduces the complexity of synchronization. The data receiving side 300 is used to merge the odd-row pixel sub-data streams and even-row pixel sub-data streams of adjacent rows in the same frame data stream in different transmission channels 100 to obtain a complete pixel sub-data stream output.

[0125] Reference Figure 4 As another example, the distribution mode includes dividing each row of pixel sub-data stream into a left sub-data stream and a right sub-data stream, outputting the left sub-data stream to one or more of the multiple transmission channels 100, and outputting the right sub-data stream to another one or more of the multiple transmission channels 100.

[0126] This distribution mode is defined as a left-right partial transmission mode, which can be used to reduce the amount of data in each transmission channel 100 when transmitting each row of pixel sub-data streams, reduce the bandwidth requirements of each transmission channel 100, and ensure that more data streams can be transmitted under limited mipi bandwidth. Dividing each row of pixel sub-data streams into a left sub-data stream and a right sub-data stream can flexibly adapt to video streams of different resolutions, and adjust the size of the left sub-data stream and the right sub-data stream as needed. The data receiving side 300 is used to merge the left sub-data stream and the right sub-data stream of the same row of pixel sub-data streams in different transmission channels 100 to obtain a complete pixel sub-data stream output.

[0127] The specific implementation of the distribution mode adopted by the method may correspond to the embodiment of the distribution module 210 described above, and will not be described in detail here.

[0128] In one embodiment, step S200, the step of the data receiving side 300 matching the marker bits of the received sub-data streams of the plurality of transmission channels 100 respectively comprises:

[0129] When dividing the data stream in each row of pixel sub-data streams into odd pixel sub-data streams and even pixel sub-data streams, or dividing each row of pixel sub-data streams into left sub-data streams and right sub-data streams, the identification bits of the sub-data streams of multiple transmission channels are determined by frame numbers and row numbers, and the data receiving side matches the row numbers of the sub-data streams in the same frame among the sub-data streams of the received multiple transmission channels.

[0130] It can be understood that the video data stream to be processed is in frames, each frame includes multiple rows of pixel data streams, each video frame will have a unique frame number, and each row of pixel data streams in each frame has a corresponding row number. For sub-data streams transmitted in odd-even pixel transmission mode and left-right partial transmission mode, the sub-data streams transmitted by each transmission channel 100 will be attached with the frame number of the frame to which they belong and the corresponding row number. The frame number and row number are used as the identification bits of the sub-data streams of the transmission channel 100 to determine whether the sub-data streams of different transmission channels 100 are sub-data streams located in the same row of the same frame. This helps to determine and match the data streams of different transmission channels 100, ensuring that the sub-data streams of different transmission channels 100 received by the data receiving side 300 are from the same row of the same frame, thereby facilitating the correct reorganization of the original video data stream.

[0131] When adopting odd-even pixel transmission mode or left-right divided transmission mode, taking the mipi interface as an example, when synchronizing multiple mipi data on the data receiving side, it is necessary to insert the frame number (frame_num) and line number (line_num) in the sub-data stream. Specifically, the frame number (frame_num) is inserted into the sub-data stream according to the detected frame of the sub-data stream, and the line number (line_num) is inserted into the sub-data stream of the corresponding transmission channel according to the order in which the sub-data stream enters the corresponding transmission channel.

[0132] Reference Figure 6 , Figure 7 In one embodiment, the step of matching the row numbers of the sub-data streams in the same frame among the sub-data streams received from the multiple transmission channels by the data receiving side specifically includes:

[0133] Step S211, detecting the frame of the sub-data stream, and when detecting the rising edge of the signal for indicating the first pixel sub-data stream in the sub-data stream of each transmission channel, obtaining the indication signal of the transmission channel for indicating the sub-data stream and caching the frame number of the sub-data stream;

[0134] Step S212: After the indication signals of all transmission channels are obtained, the frame numbers of the sub-data streams of the multiple transmission channels are compared.

[0135] Specifically, when the data receiving side 300 is in the initial IDLE state, when the channel tuser is detected (tuser is pulled high to indicate the arrival of the first pixel of the data stream), an indication signal of the transmission channel is generated or obtained for indicating the sub-data stream and caching the frame number of the sub-data stream. Specifically, the sub-data stream of the corresponding transmission channel can be indicated by pulling up the signal, and the frame number is cached by using a cache module such as a register, and after obtaining the indication signals of all transmission channels (determining that the signals of all transmission channels are pulled high), the frame numbers frame_num of all transmission channels are compared.

[0136] Step S2131, when the frame numbers of the sub-data streams of multiple transmission channels are the same, determine that the sub-data streams of multiple transmission channels received by the data receiving side are located in the same frame; Step S2132, when the frame numbers of the sub-data streams of multiple transmission channels are different, re-execute the step of detecting the frames of the sub-data streams.

[0137] Specifically, when the frame_num values ​​of the sub-data streams of multiple transmission channels are the same, it means that the corresponding multiple sub-data streams are in the same frame. If the frame_num values ​​are not equal, it means that they are not in the same frame. Through steps S2131 and S2132, the system determines whether the sub-data streams of multiple transmission channels 100 are in the same frame, and ensures that all sub-data streams belong to the same frame through frame number matching to avoid confusion of cross-frame data. If the frame number does not match, the rising edge and falling edge of the signal are re-detected to ensure accuracy.

[0138] Step S2141, detecting the row numbers of multiple sub-data streams in the same frame; Step S2142, when the row numbers of multiple sub-data streams in the same frame are the same, determining that the multiple sub-data streams in the same frame are in the same row, and outputting a counting instruction for counting the current number of rows of the sub-data streams; Step S2143, when the row numbers of multiple sub-data streams in the same frame are different, determining that the multiple sub-data streams in the same frame are not in the same row, discarding all sub-data streams of the frame, and re-executing the step of detecting the frame of the sub-data streams.

[0139] After confirming that the sub-data streams corresponding to the multiple transmission channels 100 belong to the same frame, the system further detects and matches the row numbers. Specifically, the data sending side 200 distributes the multiple sub-data streams to multiple transmission channels for transmission, and the sub-data streams entering different transmission channels will determine the row numbers according to the order of entering the corresponding transmission channels. When the odd-even pixel transmission mode and the left-right division transmission mode are used for transmission, the sub-data streams with the same row numbers of different transmission channels come from the same row in the same frame of the video data stream to be processed, that is, the row number of the sub-data stream in the corresponding transmission channel is consistent with the actual number of rows of the sub-data stream. The row numbers are matched through steps S2141, S2142 and S2143, which are mainly used to match the row numbers of the sub-data streams in the corresponding transmission channels. If the row number matches successfully (that is, the row numbers of the sub-data streams of multiple transmission channels are the same), the current row number is recorded to ensure that only the sub-data streams from the same frame and the same row are matched. If the row number does not match (that is, the row numbers of the sub-data streams of multiple transmission channels are different), all sub-data streams of the current frame are discarded and re-detected. Specifically, if the line_num value is detected to be the same, it means that the sub-data streams are in the same row, and the counting instruction for counting the current row number of the sub-data stream is output to the cache module such as the register h_cnt, so that the register h_cnt is increased by 1. If the line_num value is detected to be different, it means that the corresponding sub-data streams are not in the same row, and after discarding all the sub-data streams of the frame, the frame of the sub-data stream is re-detected, and the next frame synchronization is waited.

[0140] Step S2151, when the current number of rows counted does not match the actual number of rows of the sub-data stream, the step of detecting the row numbers of multiple sub-data streams located in the same frame is re-executed; Step S2152, when the current number of rows counted of the sub-data stream matches the actual number of rows of the sub-data stream, it is determined that the sub-data streams located in the same frame and with the same row number in the sub-data streams of multiple transmission channels are matched.

[0141] Specifically, when the current number of rows counted does not match the actual number of rows in the sub-data stream, the value of register h_cnt is not equal to the number of rows; if the value of register h_cnt is equal to the number of rows, it indicates that the row merging is completed. The accuracy of the row number matching is verified through steps S2151 and S2152 to ensure the correctness of data reorganization. If the current number of rows counted matches the actual number of rows, subsequent data reorganization and processing are performed.

[0142] These steps ensure the integrity and correctness of the data by matching the frame number and the row number, and improve the stability and reliability of the system. They are used to ensure that the sub-data streams of the multiple transmission channels 100 received by the data receiving side 300 are located in the same row of the same frame when the odd-even pixel transmission mode or the left-right partial transmission mode is adopted, so as to facilitate the correct reorganization of the original video data stream.

[0143] In some other optional embodiments of the present application, the aforementioned step S211 and step S212 may be replaced by:

[0144] Detect the frame of the sub-data stream, and when the rising edge of the signal indicating the first pixel sub-data stream in the sub-data stream of each transmission channel 100 is detected, output a cache instruction for starting to cache the frame number; determine the frame number of the sub-data stream according to the subsequently detected falling edge of the signal.

[0145] It can be understood that the rising edge of the signal marks the beginning of a new frame, and at this time, the cache instruction is output to the cache module such as the register to cache the frame number to prepare for subsequent caching; the falling edge of the signal marks the end of the frame, and the frame number of the sub-data stream can be determined by detecting the rising edge and the falling edge, ensuring that the system can accurately identify the frame number of the current sub-data stream. When the data receiving side 300 is in the initial IDLE state, when the rising edge of the tuser_flag (tuser is pulled high to indicate the arrival of the first pixel of the data stream, tuser_flag is pulled high when one tuser is pulled high, and pulled low when the other tuser is pulled high) signal is detected, the frame number of the current channel data stream is cached by the cache module such as the register. In this way, the frame number of the sub-data streams of different transmission channels can be determined. Compared with the aforementioned steps S211 and S212, this replacement scheme does not directly indicate the sub-data streams of different transmission channels, and is mainly applicable to the embodiment of transmitting the video data stream to be processed through two transmission channels; other implementations of this replacement scheme can refer to the relevant embodiments of the aforementioned steps S211 to S2151, which will not be repeated here.

[0146] In another embodiment, step S200, the step of the data receiving side 300 matching the identification bits of the received sub-data streams of the plurality of transmission channels 100 with each other comprises:

[0147] When the data stream in each frame is divided into odd-row pixel sub-data streams and even-row pixel sub-data streams, the identification bits of the sub-data streams of multiple transmission channels are determined by the frame number and the row number, and the data receiving side matches the row numbers of the sub-data streams in the same frame among the sub-data streams of the received multiple transmission channels.

[0148] It can be understood that the video data stream to be processed is in frames, each frame includes multiple rows of pixel data streams, each video frame will have a unique frame number, and each row of pixel data streams in each frame will have a corresponding row number. For sub-data streams transmitted in odd-even row transmission mode, the sub-data streams transmitted by each transmission channel 100 will be attached with the frame number of the frame to which it belongs and the corresponding row number. The frame number and row number are used as the identification bits of the sub-data streams of the transmission channel 100 to determine whether the sub-data streams of different transmission channels 100 are located in adjacent rows in the same frame. This helps to determine and match the data streams of different transmission channels 100, ensuring that the sub-data streams of different transmission channels 100 received by the data receiving side 300 are from adjacent rows in the same frame, thereby facilitating the correct reorganization of the original video data stream.

[0149] When using the odd-even row transmission mode, taking the mipi interface as an example, when synchronizing the data of multiple mipis on the data receiving side, it is necessary to insert the frame number (frame_num) and the row number (line_num) in the sub-data stream. Specifically, the frame number (frame_num) is inserted into the sub-data stream according to the detected frame of the sub-data stream, and the row number (line_num) is inserted into the sub-data stream of the corresponding transmission channel according to the order in which the sub-data stream enters the corresponding transmission channel.

[0150] Reference Figure 8 , Fig. 9 In one embodiment, the step of matching the row numbers of the sub-data streams in the same frame among the sub-data streams received from the multiple transmission channels by the data receiving side specifically includes:

[0151] Step S221, detecting the frame of the sub-data stream, and when detecting the rising edge of the signal for indicating the first pixel sub-data stream in the sub-data stream of each transmission channel, obtaining the indication signal of the transmission channel for indicating the sub-data stream and caching the frame number of the sub-data stream;

[0152] Step S222: after obtaining the indication signals of all transmission channels, comparing the frame numbers of the sub-data streams of the multiple transmission channels;

[0153] Step S2231: when the frame numbers of the sub-data streams of the multiple transmission channels are the same, determining that the sub-data streams of the multiple transmission channels received by the data receiving side are located in the same frame; Step S2232: when the frame numbers of the sub-data streams of the multiple transmission channels are different, re-performing the step of detecting the frame of the sub-data stream;

[0154] Step S2241, detecting the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel; Step S2242, when the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel are the same, determining that the multiple sub-data streams are located in adjacent rows in the same frame, and outputting a counting instruction for counting the current number of rows of the sub-data streams in the corresponding transmission channel; Step S2243, when the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel are different, determining that the multiple sub-data streams are not located in adjacent rows in the same frame, discarding all the sub-data streams of the frame and then re-performing the step of detecting the frame of the sub-data stream;

[0155] Step S2251: When the sum of the current number of rows of all transmission channels counted does not match the actual number of rows of the sub-data streams, the step of detecting the row numbers of multiple sub-data streams located in the same frame is re-executed; Step S2252: When the sum of the current number of rows of all transmission channels counted matches the actual number of rows of the sub-data streams, it is determined that the sub-data streams located in the same frame and with adjacent row numbers in the sub-data streams of the multiple transmission channels are matched.

[0156] Steps S221 to S2243 in this embodiment may correspond to the relevant steps of steps S211 to S2143 in the aforementioned embodiment. When the odd-even row transmission mode is adopted for transmission, the sub-data streams with the same row number of different transmission channels come from adjacent rows in the same frame of the video data stream to be processed, that is, the row numbers of the sub-data streams in the corresponding transmission channels are the same, and the sum of the current row numbers of all transmission channels is calculated according to the number of transmission channels, and the sum of the current row numbers of all transmission channels is consistent with the number of rows currently actually processed by the sub-data streams (that is, the actual number of rows of the sub-data streams). The row numbers are matched through steps S2241, S2242 and S2243, which are mainly used to match the row numbers of the sub-data streams in the corresponding transmission channels. If the row number matches successfully (that is, the row numbers of the sub-data streams of multiple transmission channels are the same), the current row number is recorded to ensure that only the sub-data streams from the same frame and adjacent rows are matched. If the row number does not match (that is, the row numbers of the sub-data streams of multiple transmission channels are different), all sub-data streams of the current frame are discarded and re-detected. Specifically, if the line_num values ​​are detected to be the same, it means that the row numbers of the sub-data streams of the multiple transmission channels are the same, and the counting instruction for counting the current number of rows of the sub-data stream is output to the cache module such as the register h_cnt of the corresponding channel, so that the register h_cnt is increased by 1. If the line_num values ​​are detected to be different, it means that the row numbers of the sub-data streams of the multiple transmission channels are different, and after discarding all the sub-data streams of the frame, the frame of the sub-data stream is re-detected, and the next frame synchronization is waited.

[0157] When the sum of the current number of rows of all transmission channels counted does not match the number of rows currently actually processed by the sub-data stream (i.e., the actual number of rows of the sub-data stream), the sum of all register h_cnt values ​​is not equal to the actual number of rows processed; if the sum of the register h_cnt values ​​is equal to the actual number of rows processed, it means that the row merging is completed. The accuracy of the row number matching is verified through steps S2251 and S2252 to ensure that all sub-data streams with the same number of rows come from adjacent rows in the same frame, and further ensure the correctness of data reorganization. If the sum of the current number of rows of all transmission channels matches the actual number of rows of the sub-data stream, subsequent data reorganization and processing are performed. It is used to determine whether the sub-data streams of different transmission channels 100 are located in adjacent rows in the same frame of the video data stream to be processed by frame number when adopting the odd-even row transmission mode, and ensure that the row pixel sub-data streams of multiple transmission channels 100 received by the data receiving side 300 come from adjacent rows of the same frame of the video data to be processed, so as to facilitate the correct reorganization of the original video data stream.

[0158] In one embodiment, the step of caching the received sub-data streams of the plurality of transmission channels 100 according to the corresponding distribution mode in step S300 specifically includes:

[0159] The received sub-data streams of each transmission channel 100 are buffered respectively according to the corresponding distribution mode.

[0160] The cache step can be specifically applicable to any one of the odd-even pixel transmission mode, the odd-even row transmission mode, the left-right division transmission mode or other multiple distribution modes, and is mainly described by taking the odd-even pixel transmission mode as an example: when the data stream in each row of pixel sub-data stream is divided into an odd pixel sub-data stream and an even pixel sub-data stream, because the sub-data stream transmitted by each transmission channel 100 is a non-continuous pixel sub-data stream, it is necessary to cache these non-continuous pixel sub-data streams of each transmission channel 100 through a cache module. At the data receiving side 300, the cached pixel sub-data streams can be output in sequence to ensure the integrity and correctness of data transmission.

[0161] The specific implementation of the cache step can refer to the relevant embodiments of the cache module mentioned above, which will not be described in detail here.

[0162] In one embodiment, the step of caching the received sub-data streams of the plurality of transmission channels 100 according to the corresponding distribution mode in step S300 specifically includes:

[0163] When the data stream in each frame is divided into odd-row pixel sub-data streams and even-row pixel sub-data streams, or each row pixel sub-data stream is divided into a left sub-data stream and a right sub-data stream, the sub-data stream output later among the overlapping sub-data streams in multiple transmission channels 100 is cached.

[0164] The caching step can be specifically applicable to adopting any one of the odd-even row transmission mode, the left-right partial transmission mode or other multiple distribution modes, and caching the sub-data streams output later among the overlapping sub-data streams in the multiple transmission channels 100 through the caching module. When the sub-data streams overlap, the caching module will give priority to caching the later output parts of these overlapping sub-data streams to ensure the integrity and correctness of the data. At the data receiving side 300, the pixel sub-data streams of each row can be output in sequence.

[0165] In one embodiment, step S100, the data sending side 200 divides the video data stream to be processed into multiple sub-data streams according to the set distribution mode, and distributes the divided multiple sub-data streams to multiple transmission channels 100 for transmission, specifically includes:

[0166] The data sending side 200 divides the video data stream to be processed into multiple mipi sub-data streams according to the set distribution mode, converts the divided multiple mipi sub-data streams into GMSL serial sub-data streams and distributes them to multiple transmission channels 100 for transmission.

[0167] The distribution module 210 serves as the entry point for the video data stream to be processed, and is responsible for dividing the video data stream into multiple mipi sub-data streams, and transmitting these sub-data streams to multiple first mipi interfaces 221 respectively, in preparation for subsequent serialization processing. These mipi sub-data streams are converted from a parallel format to a GMSL (GigabitMultimedia Serial Link) serial sub-data stream through the serializer 222 of the data sending side 200, and then distributed to multiple transmission channels 100 for transmission. By dividing the video data stream to be processed into multiple mipi sub-data streams and using multiple transmission channels 100 for synchronous transmission, the bandwidth of data transmission can be significantly improved, thereby meeting the transmission requirements of high-resolution and high-frame rate scenarios.

[0168] Before executing step S200, in which the data receiving side 300 matches the identification bits of the received sub-data streams of the plurality of transmission channels 100, the video data transmission method further includes the following steps:

[0169] The data receiving side 300 deserializes the GMSL serial sub-data streams of the multiple transmission channels 100 and converts them into mipi sub-data streams for reception.

[0170] At the data receiving side 300, the GMSL serial sub-data streams transmitted by the multiple transmission channels 100 are deserialized and converted back into mipi sub-data streams through the deserializer 322. Since the deserializer 322 usually performs error detection and correction to ensure the integrity of the data, the coordinated work of the serializer 222, the transmission channel 100 and the deserializer 322 can effectively avoid data misalignment and loss problems, ensure the integrity and accuracy of the processed video data stream, and thus improve the stability of video processing.

[0171] This application uses the data distribution and merging technology of transmitting the same source data through multiple MIPI interfaces, merges the processes of multiple MIPIs with the same video source, and can use the MIPI interface to transmit high-bandwidth video streams. It provides a new interface option for the transmission requirements of high-resolution and high-line frequency scenes such as large-target area array cameras and high-line frequency line array cameras, broadens the use scenarios of the MIPI interface, and adapts to future needs in the field of industrial vision.

[0172] The video processing device includes at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video data transmission method in the above-mentioned embodiment. The video processing device in the embodiment of the present application may include but is not limited to FPGA (Field-Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application-Specific Integrated Circuit). The computer device such as the video processing device shown in the present application is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0173] Computer devices such as video processing equipment may include control components such as processing devices (such as central processing units, graphics processing units, etc.), which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). In RAM, various programs and data required for the operation of the computer device are also stored. The processing device, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus. Generally, the following systems can be connected to the I / O interface: input devices such as touch screens, touch pads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices such as liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices such as magnetic tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices wirelessly or by wire to exchange data.

[0174] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0175] The video processing device provided by the present application adopts the video data transmission method in the above embodiment to solve the technical problem of not being able to meet the transmission requirements of high-resolution and high-line frequency scenes. Compared with the prior art, the beneficial effects of the computer device provided by the present application are the same as the beneficial effects of the video data transmission method provided by the above embodiment, and the other technical features in the computer device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0176] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0177] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0178] The present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video data transmission method in the above embodiment are implemented.

[0179] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0180] The computer-readable storage medium may be included in the computer device; or it may exist independently without being assembled into the computer device. The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the computer device, the computer device: the data sending side 200 divides the video data stream to be processed into multiple sub-data streams according to the set distribution mode, and distributes the divided multiple sub-data streams to multiple transmission channels 100 for synchronous transmission; the data receiving side 300 matches the identification bits of the sub-data streams of the multiple transmission channels 100 received respectively; when the identification bits of the sub-data streams of the multiple transmission channels 100 are successfully matched, the sub-data streams of the multiple transmission channels 100 received are cached according to the corresponding distribution mode.

[0181] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including languages ​​such as Verilog, VHDL, and System Verilog. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0184] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned video data transmission method, and is used to solve the technical problem of not being able to meet the transmission requirements of high-resolution and high-line-frequency scenes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the video data transmission method provided in the above-mentioned embodiment, and will not be elaborated here.

[0185] The embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the video data transmission method of the above embodiment when the computer program is executed by a processor.

[0186] The computer program product provided in this application is used to solve the technical problem that the transmission requirements of high-resolution and high-line frequency scenarios cannot be met. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the video data transmission method provided in the above embodiment, which will not be repeated here.

[0187] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A video processing device, characterized in that: The video processing device comprises: Multiple transmission channels; The data transmission side includes a distribution module, which is used to divide the video data stream to be processed into multiple sub-data streams according to a set distribution mode, and distribute the multiple sub-data streams to multiple transmission channels for transmission; The data receiving side includes a merging module, which is connected to the distribution module via the multiple transmission channels. The merging module is used to match the identification bits of the sub-data streams of the multiple transmission channels respectively; when the identification bits of the sub-data streams of the multiple transmission channels are matched successfully, the sub-data streams of the multiple transmission channels are cached and merged according to the corresponding distribution mode.

2. The video processing device according to claim 1, characterized in that: The data transmission side is provided with a plurality of transmission modules, and the distribution module is connected to the first ends of the plurality of transmission channels through the plurality of transmission modules, and is used to divide the video data stream to be processed into a plurality of sub-data streams, and distribute the divided sub-data streams to the plurality of transmission channels for transmission; The data receiving side is provided with a plurality of receiving modules, and the merging module is connected to the second ends of the plurality of transmission channels through the plurality of receiving modules, and is used for receiving the sub-data streams transmitted by the plurality of transmission channels.

3. The video processing device according to claim 2, characterized in that: Each of the sending modules includes a first mipi interface and a serializer connected in sequence, and the distribution module is connected to a plurality of the first mipi interfaces, and is used to divide the video data stream to be processed into a plurality of mipi sub-data streams; The plurality of serializers are connected to the first ends of the plurality of transmission channels, and are used to convert the divided plurality of mipi sub-data streams into GMSL serial sub-data streams and then distribute them to the plurality of transmission channels for transmission; Each of the receiving modules includes a deserializer and a second mipi interface connected in sequence, multiple second mipi interfaces are connected to the merging module, and multiple deserializers are connected to the second ends of multiple transmission channels, and are used to deserialize the GMSL serial sub-data streams transmitted by the multiple transmission channels and then convert them into mipi sub-data streams.

4. The video processing device according to any one of claims 1 to 3, characterized in that: The video data stream to be processed is in frames, and each frame includes a plurality of lines of pixel data stream; The distribution module is used to divide the data stream in each row of pixel sub-data stream into an odd pixel sub-data stream and an even pixel sub-data stream, and output the odd pixel sub-data stream to one or more of the multiple transmission channels, and output the even pixel sub-data stream to another one or more of the multiple transmission channels; Alternatively, the distribution module is used to divide the data stream in each frame into an odd-row pixel sub-data stream and an even-row pixel sub-data stream, and output the odd-row pixel sub-data stream to one or more of the multiple transmission channels, and output the even-row pixel sub-data stream to another one or more of the multiple transmission channels; Alternatively, the distribution module is used to divide each row of pixel sub-data stream into a left sub-data stream and a right sub-data stream, and output the left sub-data stream to one or more of the multiple transmission channels, and output the right sub-data stream to another one or more of the multiple transmission channels.

5. The video processing device according to claim 1, wherein: The data receiving side also includes a cache module, and each of the transmission channels is provided with the cache module; When the data stream in each row of pixel sub-data stream is divided into an odd pixel sub-data stream and an even pixel sub-data stream, or the data stream in each frame is divided into an odd pixel sub-data stream and an even pixel sub-data stream, or each row of pixel sub-data stream is divided into a left sub-data stream and a right sub-data stream, the buffer module is used to buffer the received sub-data stream of each transmission channel respectively; When the data stream in each frame is divided into odd-row pixel sub-data streams and even-row pixel sub-data streams, or each row of pixel sub-data streams is divided into a left sub-data stream and a right sub-data stream, the cache module is used to cache the sub-data stream that is output later among the overlapping sub-data streams in multiple transmission channels.

6. A video data transmission method, characterized in that: Applied to the video processing device according to any one of claims 1 to 5, the video data transmission method comprises the following steps: The data sending side divides the video data stream to be processed into multiple sub-data streams according to the set distribution mode, and distributes the multiple sub-data streams to multiple transmission channels for transmission; The data receiving side matches the identification bits of the sub-data streams of the multiple transmission channels received respectively with each other; When the identification bits of the sub-data streams of the plurality of transmission channels are matched successfully, the received sub-data streams of the plurality of transmission channels are buffered and merged according to the corresponding distribution mode.

7. The video data transmission method according to claim 6, characterized in that: The video data stream to be processed is divided into frames, each frame includes multiple lines of pixel data stream, and the distribution mode includes: Dividing the data stream in each row of pixel sub-data streams into an odd pixel sub-data stream and an even pixel sub-data stream, outputting the odd pixel sub-data stream to one or more of the plurality of transmission channels, and outputting the even pixel sub-data stream to another one or more of the plurality of transmission channels; Dividing the data stream in each frame into an odd-row pixel sub-data stream and an even-row pixel sub-data stream, outputting the odd-row pixel sub-data stream to one or more of the multiple transmission channels, and outputting the even-row pixel sub-data stream to another one or more of the multiple transmission channels; Each row of pixel sub-data streams is divided into a left sub-data stream and a right sub-data stream, the left sub-data stream is output to one or more of the multiple transmission channels, and the right sub-data stream is output to another one or more of the multiple transmission channels.

8. The video data transmission method according to claim 7, characterized in that: The step of the data receiving side matching the identification bits of the sub-data streams of the received multiple transmission channels with each other comprises: When dividing the data stream in each row of pixel sub-data stream into odd pixel sub-data stream and even pixel sub-data stream, or dividing each row of pixel sub-data stream into left sub-data stream and right sub-data stream, or dividing the data stream in each frame into odd row pixel sub-data stream and even row pixel sub-data stream, the identification bits of the sub-data streams of multiple transmission channels are determined by frame number and row number, and the data receiving side matches the row numbers of the sub-data streams in the same frame among the received sub-data streams of the multiple transmission channels.

9. The video data transmission method according to claim 8, characterized in that: When dividing the data stream in each row of pixel sub-data streams into odd pixel sub-data streams and even pixel sub-data streams, or dividing each row of pixel sub-data streams into left sub-data streams and right sub-data streams, the data receiving side performs a step of matching row numbers of sub-data streams in the same frame among the received sub-data streams of the multiple transmission channels, specifically comprising: Detecting the frame of the sub-data stream, when detecting the rising edge of the signal for indicating the first pixel sub-data stream in the sub-data stream of each transmission channel, obtaining the indication signal of the transmission channel for indicating the sub-data stream and caching the frame number of the sub-data stream; After obtaining the indication signals of all transmission channels, the frame numbers of the sub-data streams of the multiple transmission channels are compared; When the frame numbers of the sub-data streams of the multiple transmission channels are the same, determining that the sub-data streams of the multiple transmission channels received by the data receiving side are located in the same frame; when the frame numbers of the sub-data streams of the multiple transmission channels are different, re-performing the step of detecting the frame of the sub-data stream; Detecting the row numbers of multiple sub-data streams in the same frame, and when the row numbers of the multiple sub-data streams in the same frame are the same, determining that the multiple sub-data streams in the same frame are in the same row, and outputting a counting instruction for counting the current number of rows of the sub-data streams; when the row numbers of the multiple sub-data streams in the same frame are different, determining that the multiple sub-data streams in the same frame are not in the same row, discarding all the sub-data streams in the frame, and re-performing the step of detecting the frame of the sub-data streams; When the current number of rows counted does not match the actual number of rows of the sub-data stream, the step of detecting the row numbers of multiple sub-data streams located in the same frame is re-executed; when the current number of rows counted for the sub-data stream matches the actual number of rows of the sub-data stream, it is determined that the sub-data streams located in the same frame and with the same row number in the sub-data streams of multiple transmission channels are matched.

10. The video data transmission method according to claim 8, characterized in that: When the data stream in each frame is divided into odd-row pixel sub-data streams and even-row pixel sub-data streams, the data receiving side performs a step of matching row numbers of sub-data streams in the same frame among the sub-data streams of the received multiple transmission channels, specifically comprising: Detecting the frame of the sub-data stream, when detecting the rising edge of the signal for indicating the first pixel sub-data stream in the sub-data stream of each transmission channel, obtaining the indication signal of the transmission channel for indicating the sub-data stream and caching the frame number of the sub-data stream; After obtaining the indication signals of all transmission channels, the frame numbers of the sub-data streams of the multiple transmission channels are compared; When the frame numbers of the sub-data streams of the multiple transmission channels are the same, determining that the sub-data streams of the multiple transmission channels received by the data receiving side are located in the same frame; when the frame numbers of the sub-data streams of the multiple transmission channels are different, re-performing the step of detecting the frame of the sub-data stream; Detecting the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel; when the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel are the same, determining that the multiple sub-data streams are in adjacent rows in the same frame, and outputting a counting instruction for counting the current number of rows of the sub-data streams in the corresponding transmission channel; when the row numbers of the multiple sub-data streams in the same frame in the corresponding transmission channel are different, determining that the multiple sub-data streams are not in adjacent rows in the same frame, discarding all the sub-data streams of the frame and then re-performing the step of detecting the frame of the sub-data streams; When the sum of the current number of rows of all transmission channels counted does not match the actual number of rows of the sub-data streams, the step of detecting the row numbers of multiple sub-data streams located in the same frame is re-executed; when the sum of the current number of rows of all transmission channels counted matches the actual number of rows of the sub-data streams, it is determined that the sub-data streams located in the same frame and with adjacent row numbers in the sub-data streams of the multiple transmission channels are matched.

11. The video data transmission method according to claim 6, characterized in that: The step of caching the received sub-data streams of the multiple transmission channels according to the corresponding distribution mode specifically includes: The received sub-data streams of each transmission channel are cached respectively according to the corresponding distribution mode.

12. The video data transmission method according to claim 7, characterized in that: The step of caching the received sub-data streams of the multiple transmission channels according to the corresponding distribution mode specifically includes: When dividing the data stream in each frame into odd-row pixel sub-data streams and even-row pixel sub-data streams, or dividing each row pixel sub-data stream into a left sub-data stream and a right sub-data stream, the sub-data stream output later among the overlapping sub-data streams in multiple transmission channels is cached.

13. The video data transmission method according to any one of claims 6 to 12, characterized in that: The data sending side divides the to-be-processed video data stream into a plurality of sub-data streams according to a set distribution mode, and distributes the divided sub-data streams to a plurality of transmission channels for transmission, specifically comprising: The data sending side divides the video data stream to be processed into multiple mipi sub-data streams according to the set distribution mode, converts the divided multiple mipi sub-data streams into GMSL serial sub-data streams and distributes them to multiple transmission channels for transmission; Before executing the step of matching the identification bits of the received sub-data streams of the multiple transmission channels at the data receiving side, the video data transmission method further includes the following steps: The data receiving side deserializes the GMSL serial sub-data streams of multiple transmission channels and converts them into mipi sub-data streams for reception.

14. A video processing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the video data transmission method according to any one of claims 6 to 13.

15. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the video data transmission method according to any one of claims 6 to 13 are implemented.

16. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the video data transmission method according to any one of claims 6 to 13 are implemented.

Images