Data shunt acquisition system, method and device, storage medium and electronic equipment

Abstract

The invention relates to a data shunt acquisition system, method and device, a storage medium and electronic equipment. The system comprises a data acquisition module, a shunt module and a control module, a deserializer of the data acquisition module is used for deserializing a video differential signal sent by an image sensor to obtain a video data signal, and sending the video data signal to a first programmable logic gate array through a first type of data link; a first programmable logic gate array of the shunt module is used for copying a video data signal and sending the video data signal to a first computing power unit, a second computing power unit and a third computing power unit of the control module through a first type of data link; and the first computing power unit, the second computing power unit and the third computing power unit are used for performing automatic driving calculation based on the video data signal, and performing comparison and voting processing on three calculation results obtained by calculation to obtain a target calculation result. According to the invention, the sensing data in the hovercar can be transmitted in a one-to-three shunting manner, and the reliability of the sensing data is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data transmission, and in particular to a data shunt acquisition system, method, device, storage medium and electronic equipment. BACKGROUND

[0002] With the development of low-altitude economy, travel traffic develops from two-dimensional traffic on the ground to three-dimensional traffic, and various flying cars begin to travel in the public eye. Current flying cars are mainly manually driven, but with the continuous development of artificial intelligence (AI) technology, flying cars will also develop to intelligent unmanned driving like cars.

[0003] Artificial intelligence unmanned driving technology involves environmental perception, decision planning and control execution, etc., and the sensors involved in environmental perception mainly include visual sensors, laser radars, millimeter wave radars, and integrated navigation, etc. Due to the need of aviation electronic safety design, the flight management control module generally adopts three-redundancy design, and the current video splitter is mainly a one-to-two splitter. As shown in FIG. 1, the deserializer divides the Gigabit Multimedia Serial Link (GMSL) input data into two parts, and sends them to two serializers respectively. If the video data needs to be divided into three parts, as shown in FIG. 2, two one-to-two splitters need to be connected in series, that is, the GMSL input data is first processed by one-to-two, and then the two new GMSL data streams are processed by one-to-two, thereby generating four GMSL data. However, this method causes the video stream data link to be too long, the delay is large, and multiple deserializers and serializers need to be configured in the link, which makes the configuration method complex and the actual operability not strong. Figure 1 Figure 2 SUMMARY

[0004] In order to solve at least one of the above technical problems, the present disclosure provides a data shunt acquisition system, device, storage medium and electronic equipment.

[0005] According to an aspect of the present disclosure, a data shunt acquisition system is provided, the system comprising a data acquisition module, a shunt module and a control module; The data acquisition module comprises at least one image acquisition unit, the image acquisition unit comprising an image sensor and a deserializer, the shunt module comprising a first programmable logic gate array, and the control module comprising a first computing power unit, a second computing power unit and a third computing power unit; The deserializer is configured to deserialize the video differential signal sent by the image sensor to obtain a video data signal, and send the video data signal to the first programmable logic gate array through the first type of data link.​​ The first programmable logic gate array is configured to replicate the video data signal and transmit the video data signal to the first computing unit, the second computing unit and the third computing unit through a first type of data link. The first computing unit, the second computing unit and the third computing unit are configured to perform automatic driving calculation based on the video data signal, compare and vote the three calculation results to obtain a target calculation result.

[0006] In some possible embodiments, the shunt module further comprises a second programmable logic gate array, and the first programmable logic gate array and the second programmable logic gate array are communicatively connected through a second type of data link. The deserializer is further configured to replicate the video data signal and transmit the video data signal to the second programmable logic gate array through the first type of data link. The second programmable logic gate array is configured to replicate the received video data signal and transmit the video data signal to the first computing unit, the second computing unit and the third computing unit through the second type of data link.

[0007] In some possible embodiments, the system further comprises a first storage module, the first storage module is connected to the first programmable logic gate array through a corresponding interface of the second type of data link, and the first storage module is configured to store backup data transmitted by the first programmable logic gate array.

[0008] In some possible embodiments, the system further comprises a second storage module, the second storage module is connected to the second programmable logic gate array through the second type of data link, and the second storage module is configured to store backup data transmitted by the second programmable logic gate array.

[0009] In some possible embodiments, the data acquisition module further comprises a laser radar unit, an optical fiber combined navigation unit and a communication link unit. The laser radar unit, the optical fiber combined navigation unit and the communication link unit are all communicatively connected to the first programmable logic gate array, the second programmable logic gate array, the first computing unit, the second computing unit and the third computing unit based on a third type of data link through a first switch. The first programmable logic gate array and the second programmable logic gate array are both communicatively connected to the first computing unit, the second computing unit and the third computing unit based on the third type of data link through a second switch.

[0010] In some possible implementation manners, the data acquisition module further includes a millimeter wave radar unit, a integrated navigation unit, and an air data sensor unit; The millimeter wave radar unit, the integrated navigation unit, and the air data sensor unit are in communication connection with the first programmable logic gate array, the second programmable logic gate array, the first computing unit, the second computing unit, and the third computing unit through the fourth type of data link. The first programmable logic gate array and the second programmable logic gate array are in communication connection with the first computing unit, the second computing unit, and the third computing unit through the fourth type of data link.

[0011] According to a second aspect of the present disclosure, a data shunting acquisition method is provided, which is applied to the system as described above, and the method includes: The deserializer deserializes the video differential signal sent by the image sensor to obtain a video data signal, and sends the video data signal to the first programmable logic gate array through the first type of data link. The first programmable logic gate array copies the video data signal and sends the video data signal to the first computing unit, the second computing unit, and the third computing unit through the first type of data link. The first computing unit, the second computing unit, and the third computing unit respectively perform automatic driving calculation based on the video data signal, compare and vote the three calculation results obtained through calculation, and obtain a target calculation result.

[0012] According to a third aspect of the present disclosure, a data shunting acquisition device is provided, which is applied to the system as described above, and the device includes: The deserializer deserializes the video differential signal sent by the image sensor to obtain a video data signal, and sends the video data signal to the first programmable logic gate array through the first type of data link. The first programmable logic gate array copies the video data signal and sends the video data signal to the first computing unit, the second computing unit, and the third computing unit through the first type of data link. The first computing unit, the second computing unit, and the third computing unit respectively perform automatic driving calculation based on the video data signal, compare and vote the three calculation results obtained through calculation, and obtain a target calculation result.

[0013] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising at least one processor, and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the data shunting acquisition system according to any one of the first aspect by executing the instructions stored in the memory.

[0014] According to a fifth aspect of the present disclosure, a computer readable storage medium is provided, the computer readable storage medium stores at least one instruction or at least one program, the at least one instruction or at least one program is loaded and executed by a processor to implement the data shunting acquisition system according to any one of the first aspect.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory, but not limiting the present disclosure.

[0016] The present disclosure is implemented, and has the following beneficial effects: The deserializer is used to deserialize the video differential signal sent by the image sensor to obtain a video data signal, and send the video data signal to the first programmable logic gate array through the first type of data link. The video signal is deserialized by the deserializer, which improves the efficiency and reliability of data transmission. The first programmable logic gate array is used to copy the video data signal and send it to the first computing unit, the second computing unit and the third computing unit through the first type of data link. The programmable logic gate array is used to reliably divide the video data signal into three parts and improve its reliability. The first computing unit, the second computing unit and the third computing unit are used to perform automatic driving calculation based on the video data signal, and compare and vote the three calculation results to obtain a target calculation result. The control platform adopts a triple redundancy design to ensure the reliability and safety of the system.

[0017] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0019] Figure 1 The structure schematic diagram of the deserializer two-way shunting system in the background art is shown; Figure 2A de-serializer one-to-four flow system schematic diagram according to the background art is shown. Figure 3 A structure schematic diagram of a data flow acquisition system according to an embodiment of the present disclosure is shown. Figure 4 A flow schematic diagram of a data flow acquisition storage system according to an embodiment of the present disclosure is shown. Figure 5 A flow schematic diagram of a multi-channel data flow acquisition storage system according to an embodiment of the present disclosure is shown. Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0020] Reference signs 100, a data flow acquisition system; 10, a data acquisition module; 11, an image acquisition unit; 12, a laser radar unit; 13, a fiber-optic integrated navigation unit; 14, a communication link unit; 15, a millimeter wave radar unit; 16, an integrated navigation unit; 17, an atmospheric data sensor unit; 20, a flow module; 21, a first programmable logic gate array; 22, a second programmable logic gate array; 30, a control module; 31, a first computing power unit; 311, a first micro-control area; 32, a second computing power unit; 321, a second micro-control area; 33, a third computing power unit; 331, a third micro-control area; 41, a first storage module; 42, a second storage module; 51, a first switch; 52, a second switch; 61, a target communication link; DETAILED DESCRIPTION The technical solutions in the embodiments of the present specification will be described clearly and completely below in combination with the drawings in the embodiments of the present specification. Obviously, the described embodiments are only some of the embodiments of the present specification, rather than all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in different drawings represent the same or similar elements. Although various aspects of embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically noted.

[0023] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0024] The term "and / or" used herein only means an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of the plurality or any combination of at least two of the plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0025] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail in order to highlight the main idea of the present disclosure.

[0026] Figure 3 A structural schematic diagram of a data shunt acquisition system 100 according to an embodiment of the present disclosure is shown, as shown in Figure 3 The above system includes a data acquisition module 10, a shunt module 20, and a control module 30; The data acquisition module 10 includes at least one image acquisition unit 11, the image acquisition unit 11 includes an image sensor and a deserializer, the shunt module 20 includes a first programmable logic gate array 21, and the control module 30 includes a first computing power unit 31, a second computing power unit 32, and a third computing power unit 33; The deserializer is used to deserialize the video differential signal sent by the image sensor to obtain a video data signal, and send the video data signal to the first programmable logic gate array 21 through the first type of data link; The first programmable logic gate array 21 is used to copy the video data signal and send it to the first computing power unit 31, the second computing power unit 32, and the third computing power unit 33 through the first type of data link; The first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 are used for automatic driving calculation based on the video data signal, and the three calculation results obtained by calculation are compared and voted to obtain a target calculation result.

[0027] The image acquisition unit 11 is connected with the first programmable logic gate array 21 through the first type of data link, and the first programmable logic gate array 21 is connected with the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through three first type of data link interfaces. The video data signal obtained by deserializing the video signal collected by the image sensor is sent to the first programmable logic gate array 21 through the first type of data link. The first programmable logic gate array 21 buffers the received video data signal, and copies and distributes the video data signal to the three computing power units through the multiple first type of data links.

[0028] In some embodiments, the data shunt acquisition system 100 can be applied to a flying car, and the image sensor includes but is not limited to an RGB camera, a passive infrared camera and an event camera. The deserializer restores the high-speed serial data stream input by the image sensor into parallel data. The control module 30 adopts a triple redundancy design, including the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33. The first type of data link can adopt a Mobile Industry Processor Interface (MIPI). The first programmable logic gate array 21 includes a Field Programmable Gate Array (FPGA) chip, and the first programmable logic gate array 21 includes multiple MIPI interfaces and is connected to the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through the multiple MIPI interfaces. The first computing power platform performs automatic driving calculation based on the video data signal to obtain a first calculation result, the second computing power platform performs automatic driving calculation based on the video data signal to obtain a second calculation result, and the third computing power platform performs automatic driving calculation based on the video data signal to obtain a third calculation result; the first computing power platform, the second computing power platform and the third computing power platform compare and vote the first calculation result, the second calculation result and the third calculation result to determine a final target calculation result.

[0029] In some embodiments, the image acquisition unit 11 includes a first image acquisition unit 11, a second image acquisition unit 11 and a third image acquisition unit 11, the first image acquisition unit 11 includes an RGB camera and a first deserializer, the second image acquisition unit 11 includes a passive infrared camera and a second deserializer, and the third image acquisition unit 11 includes an event camera and a third deserializer. The first image acquisition unit 11, the second image acquisition unit 11 and the third image acquisition unit 11 are connected through the first type of data link and the first programmable logic gate array 21 respectively.

[0030] The technical scheme has the advantages that the video signal is deserialized by the deserializer, and the efficiency and reliability of data transmission are improved. The video data signal is reliably split into three parts for transmission by the programmable logic gate array, and the reliability is improved. The control platform adopts a triple-redundancy design to ensure the reliability of the system and the safety of the control.

[0031] In some embodiments, the shunt module 20 further comprises a second programmable logic gate array, and the first programmable logic gate array 21 and the second programmable logic gate array are connected in communication through a second type of data link; The deserializer is further configured to copy the video data signal and send the video data signal to the second programmable logic gate array through the first type of data link; The second programmable logic gate array is configured to copy the received video data signal and send the video data signal to the first computing unit 31, the second computing unit 32, and the third computing unit 33 through the second type of data link.

[0032] The deserializer can configure the video signal to be copied and output based on a two-way first type of data link. The second programmable logic gate array comprises a field programmable gate array (FPGA) chip, and the second programmable logic gate array is connected to each image acquisition unit 11 in the data acquisition module 10 through the first type of data link. The second type of data link adopts a peripheral component interconnect express (PCIe) interface. The second programmable logic gate array comprises multiple PCIe interfaces, and is connected to the first computing unit 31, the second computing unit 32, and the third computing unit 33 through the multiple PCIe interfaces.

[0033] The first programmable logic gate array 21 and the second programmable logic gate array each adopt an independent power supply and crystal oscillator design. The video data signal obtained after the video signal acquired by the image sensor is deserialized by the deserializer is sent to the second programmable logic gate array through the first type of data link. The second programmable logic gate array buffers the received video data signal and copies and distributes the video data signal to the three computing units through the multiple second type of data links.

[0034] The technical scheme has the advantages that the shunt module 20 adopts a redundancy design of two programmable logic gate arrays to ensure the independence and safety of their respective work, and at the same time, ensure the reliability of the system.

[0035] Please refer to Figure 4In some embodiments, the system further comprises a first storage module 41 connected with the first programmable logic gate array 21 through the second type of data link, and the first storage module 41 is configured to store backup data transmitted by the first programmable logic gate array 21.

[0036] The first programmable logic gate array 21 comprises an interface corresponding to the second type of data link, and the first programmable logic gate array 21 is connected with the first storage module 41 through the interface corresponding to the second type of data link. The first storage module 41 can be a solid state disk (SSD).

[0037] In some embodiments, the first programmable logic gate array 21 comprises an interface corresponding to a plurality of second type of data links, and the first programmable logic gate array 21 is connected with the first storage module 41 through the interface corresponding to the plurality of second type of data links, forming a high-bandwidth large-capacity storage module.

[0038] In some embodiments, the first programmable logic gate array 21 comprises an interface corresponding to four second type of data links, and the first programmable logic gate array 21 is connected with the first storage module 41 through the interface corresponding to the four second type of data links.

[0039] The above technical solution adds a storage module to the system through the first storage module 41, which serves as a data backup disk data channel for the first programmable logic gate array 21.

[0040] Please refer to Figure 5 In some embodiments, the system further comprises a second storage module 42 connected with the second programmable logic gate array through the second type of data link, and the second storage module 42 is configured to store backup data transmitted by the second programmable logic gate array.

[0041] The second programmable logic gate array comprises an interface corresponding to the second type of data link, and the second programmable logic gate array is connected with the second storage module 42 through the interface corresponding to the second type of data link. The second storage module 42 can be a solid state disk (SSD).

[0042] In some embodiments, the second programmable logic gate array comprises an interface corresponding to a plurality of second type of data links, and the second programmable logic gate array is connected with the second storage module 42 through the interface corresponding to the plurality of second type of data links, forming a high-bandwidth large-capacity storage module.

[0043] In some embodiments, the second programmable logic gate array includes an interface corresponding to the fourth type of data link, and the second programmable logic gate array is connected to the second storage module 42 through the interface corresponding to the fourth type of data link.

[0044] In some embodiments, a PCIe interface is designed on the second programmable logic gate array, and the second storage module 42, the first storage module 41 and the second storage module 42 are all non-volatile memory express (NVMe) solid state disks. When an abnormality occurs in the storage channel of the nVME SSD, i.e., the first storage module 41, on the first programmable logic gate array 21, sensor data, communication link data, control data, and the calculation results of the three computing units and flight state information can be cached in the second programmable logic gate array, and then written to the nVME SSD storage module, i.e., the second storage module 42, for backup, thereby preventing data loss and improving the safety of system data acquisition and storage.

[0045] The above technical solution further increases the storage module of the system through the second storage module 42, which serves as a data backup disk channel for the second programmable logic gate array. This improves the safety design of the system and prevents the SSD interface of the first programmable logic gate array 21 connected to the PCIe-based SSD storage module, i.e., the first storage module 41, from being abnormal, and even prevents the first programmable logic gate array 21 itself from being abnormal, which causes the data acquisition function to fail to work normally.

[0046] In some embodiments, the data acquisition module 10 further includes a laser radar unit 12, a fiber-optic combined navigation unit 1613, and a communication link unit 14. The laser radar unit 12, the fiber-optic combined navigation unit 1613, and the communication link unit 14 are all connected to the first programmable logic gate array 21, the second programmable logic gate array, the first computing unit 31, the second computing unit 32, and the third computing unit 33 based on the third type of data link through the first switch 51. The first programmable logic gate array 21 and the second programmable logic gate array are both connected to the first computing unit 31, the second computing unit 32, and the third computing unit 33 based on the third type of data link through the second switch 52.

[0047] The laser radar unit 12 acquires laser radar point cloud data, the fiber-optic combined navigation unit 1613 integrates multiple sensors, such as a fiber-optic gyroscope, an accelerometer, and a satellite receiver. The communication link unit 14 includes, but is not limited to, a satellite link channel, a 5G cellular network channel, and a radio channel. Figure 5As shown, the laser radar unit 12, the fiber-optic integrated navigation unit 16 13, the positioning unit and the communication link unit 14 are respectively connected with the first programmable logic gate array 21, the second programmable logic gate array, the first computing power platform, the second computing power platform and the third computing power platform through a third type of data link through Ethernet. The third type of data link is a data link based on the Avionics Full-Duplex Switched Ethernet (AFDX) bus protocol specification, and the first switch 51 can be an AFDX Ethernet switch. The laser radar unit 12, the fiber-optic integrated navigation unit 16 13 and the communication link unit 14 bridge the first programmable logic gate array 21, the second programmable logic gate array, the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through the first switch 51. The second switch 52 can be an AFDX Ethernet switch. The first programmable logic gate array 21 and the second programmable logic gate array are both bridged with the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through the second switch 52, and the first programmable logic gate array 21, the second programmable logic gate array, the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 are bridged with other communication links, i.e., target communication links 61, through the second switch 52. Exemplarily, the target communication links 61 include but are not limited to a satellite link channel, a 5G cellular network channel and a radio channel.

[0048] In some embodiments, the first computing power unit 31 includes a first micro-control area 311, the second computing power unit 32 includes a second micro-control area 321, and the third computing power unit 33 includes a third micro-control area 331. The first micro-control area 311, the second micro-control area 321 and the third micro-control area 331 are respectively used to control the automatic driving of the flying car based on the target calculation result.

[0049] In some embodiments, the first programmable logic gate array 21 and the second programmable logic gate array are also connected through a 10G Ethernet interface. The sensing data and the like of the laser radar unit 12, the fiber-optic integrated navigation unit 16 13 and the communication link unit 14 can be respectively input into the first programmable logic gate array 21 and the second programmable logic gate array through the double-redundant AFDX aviation Ethernet for caching. The sensing data is preferentially quickly written to a large-capacity SSD storage module, i.e., the first storage module 41, through the PCIe interface of the first programmable logic gate array 21, so as to complete the collection and storage of the sensing data. The control data can also be cached in the second programmable logic gate array first, then transmitted to the first programmable logic gate array 21 through the 10G Ethernet interface, and finally written to the high-speed SSD storage module, i.e., the first storage module 41, which is extendedly connected to the first programmable logic gate array 21, as a backup data writing channel.

[0050] The technical solution described above collects and stores the data collected by the laser radar unit 12, the optical fiber combined navigation unit 1613 and the communication link unit 14 in real time and reliably with high bandwidth, facilitating subsequent automatic driving data analysis and safety processing.

[0051] In some possible embodiments, the data acquisition module 10 further includes a millimeter wave radar unit 15, a combined navigation unit 16 and an atmospheric data sensor unit 17. The millimeter wave radar unit 15, the combined navigation unit 16 and the atmospheric data sensor unit 17 are connected to the first programmable logic gate array 21, the second programmable logic gate array, the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through the fourth type of data link. The first programmable logic gate array 21 and the second programmable logic gate array are respectively connected to the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through the fourth type of data link.

[0052] The combined navigation unit 16 includes but is not limited to a MEMS inertial measurement unit, a magnetometer and a satellite positioning module. The fourth type of data link can be a data link based on a bus protocol (Controller Area Network, CAN), and the millimeter wave radar unit 15, the combined navigation unit 16 and the atmospheric data sensor unit 17 are connected to the first programmable logic gate array 21, the second programmable logic gate array, the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through the bus protocol interface. In some embodiments, the millimeter wave radar unit 15, the combined navigation unit 16 and the atmospheric data sensor unit 17 are connected to the first programmable logic gate array 21 and the second programmable logic gate array through the CAN interface, respectively. After the sensor data output by the millimeter wave radar unit 15, the combined navigation unit 16 and the atmospheric data sensor unit 17 is cached in the first programmable logic gate array 21, it is preferentially stored in the large-capacity SSD storage module, i.e. the first storage module 41, through the PCIe high-speed interface of the first programmable logic gate array 21, completing the sensor data acquisition and storage of the millimeter wave radar unit 15, the combined navigation unit 16 and the atmospheric data sensor unit 17. The sensor data of the millimeter wave radar unit 15, the combined navigation unit 16 and the atmospheric data sensor unit 17 can also be cached in the second programmable logic gate array, and then transmitted to the first programmable logic gate array 21 through the 10G Ethernet interface, and finally stored in the high-speed SSD storage module, i.e. the first storage module 41, connected to the first programmable logic gate array 21, as a backup storage data channel.

[0053] The technical solution has the advantages that the data collected by the millimeter wave radar unit 15, the integrated navigation unit 16 and the atmospheric data sensor unit 17 are collected and stored in real time and reliably with high bandwidth, and subsequent data analysis and safety processing are facilitated.

[0054] According to a second aspect of the present disclosure, a data shunting collection method is provided, applied to the system as above, and the method comprises: The deserializer deserializes the video differential signal sent by the image sensor to obtain a video data signal, and sends the video data signal to the first programmable logic gate array 21 through the first type of data link; The first programmable logic gate array 21 copies the video data signal and sends it to the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 through the first type of data link; The first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 respectively perform automatic driving calculation based on the video data signal, compare and vote the three calculation results obtained by calculation to obtain a target calculation result.

[0055] In some embodiments, the first programmable logic gate array 21 buffers the MIPI video data signal received from the deserializer. The first programmable logic gate array 21 is connected to the three computing power units through multiple MIPI interfaces, so as to copy and distribute the video data signal to the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33, which is a video data communication link 1 for the image sensor to transmit the video data signal to the computing power units. The second programmable logic gate array buffers the MIPI video data signal sent by the deserializer, and then connects to the three computing power units through multiple PCIe interfaces. The PCIe communication interface can be configured as PCIe3.0 x4 or PCIe4.0 x4 according to the PCIe interface characteristics of the computing power units, and the communication bandwidth can reach 64Gbps, fully meeting the bandwidth communication requirements of multiple image collection units 11. In this way, the video data signal input to the second programmable logic gate array from the image sensor is copied and distributed to the first computing power unit 31, the second computing power unit 32 and the third computing power unit 33 for AI perception calculation, which is a video data communication link 2 for the image sensor to transmit to the three computing power units. By using the above method, two sets of high-speed video data links are established for the image collection unit 11 to the three computing power units, and MIPI communication interface and PCIe communication interface health monitoring circuits are designed in the two programmable logic gate arrays to monitor abnormal conditions of each circuit. Once an abnormality is found in a certain circuit, the system communication is switched to another backup communication link in time, so as to ensure normal operation of the system communication.

[0056] In some embodiments, the first PLG 21 and the second PLG are also connected via a 10G Ethernet interface. Control data from the microcontroller area within the computing unit can be input into the first PLG 21 and the second PLG via dual-redundant AFDX aviation Ethernet for caching. Data collected by the lidar unit 12, fiber-optic navigation unit 1613, and communication link unit 14 is preferentially transferred to a high-capacity SSD storage module, i.e., the first storage module 41, via the PCIe interface of the first PLG 21, completing the collection and storage of control data. Control data can also be cached in the second PLG before being transferred to the first PLG 21 via a 10G Ethernet interface. Finally, data can be transferred to the high-speed SSD storage module, i.e., the first storage module 41, which is connected to the first PLG 21 as a backup data transfer channel.

[0057] In some embodiments, the data branching and acquisition system 100 includes a data acquisition module 10, a branching module 20 and a control module 30. The control module 30 adopts a triple-redundant design. Each first computing unit 31 performs artificial intelligence calculations on all sensor data, and compares and votes on the three calculation results. Under normal circumstances, the calculation results of the three computing units are the same, but in some special cases, the calculation results of the three computing units may be inconsistent. In the case of inconsistent calculation results, the calculation results of each computing unit are transmitted to the first programmable logic gate array 21 and the second programmable logic gate array through the dual-redundant AFDX aviation Ethernet for caching, and are preferentially written to the large-capacity SSD storage module, i.e., the first storage module 41, which is extended and connected to the first programmable logic gate array 21 through the PCIe interface of the first programmable logic gate array 21, to facilitate subsequent artificial intelligence algorithm analysis and iterative evolution to determine the final target calculation result. At the same time, the calculation results of each computing power unit are reserved and cached through the second programmable logic gate array, and then transmitted to the first programmable logic gate array 21 through the 10G Ethernet interface between the two, and finally written to the high-speed SSD storage module extended and connected to the first programmable logic gate array 21, namely the first storage module 41, as a backup data channel for writing to disk.

[0058] The technical solution has the following advantages. The double-redundant, non-volatile, high-bandwidth and large-capacity SSD storage module based on the high-speed PCIe interface link is designed, and various sensor data is reliably accessed into the first programmable logic gate array 21 and the second programmable logic gate array through the backup redundant communication link, and the sensor data is collected in real time and stored in the non-volatile, high-bandwidth and large-capacity SSD storage module. Based on the multi-redundant communication link design method, the computing unit is reliably accessed into the first programmable logic gate array 21 and the second programmable logic gate array, and important data of the computing unit is collected in real time and stored in the high-speed and large-capacity SSD storage module.

[0059] The sensor data, communication link data, control data, computing intermediate result data of the computing unit and flight state information are collected and stored in real time and reliably in a high bandwidth, which facilitates subsequent data analysis and safety processing.

[0060] According to a third aspect of the present disclosure, a data shunting collection device is provided, which is applied to the system as above, and characterized in that the device comprises: The deserializing module is configured to deserialize the video differential signal sent by the image sensor to obtain a video data signal, and send the video data signal to the first programmable logic gate array 21 through the first type of data link. The transmitting module is configured to copy the video data signal by the first programmable logic gate array 21 and send the video data signal to the first computing unit 31, the second computing unit 32 and the third computing unit 33 through the first type of data link. The computing module is configured to perform autonomous driving calculation based on the video data signal by the first computing unit 31, the second computing unit 32 and the third computing unit 33 respectively, and compare and vote the three calculation results to obtain a target calculation result.

[0061] In some embodiments, the device provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0062] The embodiments of the present disclosure provide a data shunting collection device, which can be a terminal or a server. The data shunting collection device includes a processor and a memory. The memory stores at least one instruction or at least one program. The at least one instruction or the at least one program is loaded and executed by the processor to implement the data shunting collection method provided by the above method embodiments.

[0063] The memory can be used to store software programs and modules, and the processor executes various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by functions, etc.; and the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access for the processor to the memory.

[0064] The method embodiments provided by the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or similar computing device, etc. Figure 6 is a hardware structure block diagram of an electronic device provided by a data shunt collection method according to an embodiment of the present application. As shown in Figure 6 The electronic device 900 can have a large difference due to different configurations or performances, and can include one or more central processing units (CPUs) 910 (the processor 910 can include but not limited to a microprocessor MCU or a programmable logic device FPGA processing device), a memory 930 for storing data, one or more storage media 920 (such as one or more mass storage devices) for storing application programs 923 or data 922. Among them, the memory 930 and the storage medium 920 can be temporary storage or persistent storage. The programs stored in the storage medium 920 can include one or more modules, each module can include a series of instruction operations in the electronic device. Further, the central processing unit 910 can be configured to communicate with the storage medium 920 and execute a series of instruction operations in the storage medium 920 on the electronic device 900. The electronic device 900 can also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input and output interfaces 940, and / or one or more operating systems 921, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0065] The input / output interface 940 can be configured to receive or transmit data via a network. Examples of the network can include a wireless network provided by a communication provider of the electronic device 900. In an example, the input / output interface 940 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In an example, the input / output interface 940 can be a radio frequency (RF) module configured to communicate with the Internet through a wireless manner.

[0066] Those skilled in the art can understand that, Figure 6 The structure shown is merely schematic, and does not limit the structure of the electronic device. For example, the electronic device 900 can further include more or less components than those shown, or have a different configuration from that shown. Figure 6 For example, the electronic device 900 can further include more or less components than those shown, or have a different configuration from that shown. Figure 6 For example, the electronic device 900 can further include more or less components than those shown, or have a different configuration from that shown.

[0067] The embodiment of the present application further provides a computer readable storage medium, which can be arranged in the electronic device to store at least one instruction or at least one program for implementing a data shunting collection method in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the data shunting collection method provided by the above method embodiment.

[0068] Optionally, in the embodiment, the storage medium can be located in at least one of a plurality of network servers of a computer network. Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0069] According to an aspect of the present application, a computer program product or a computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the various optional implementation manners.

[0070] It can be seen from the embodiments of the data branching acquisition method, device, equipment, terminal, server, storage medium or computer program provided by the present application that the deserializer of the present application is used to deserialize the video differential signal sent by the image sensor to obtain a video data signal, and send the video data signal to the first programmable logic gate array through a first type of data link; the video signal is deserialized by the deserializer, thereby improving the efficiency and reliability of data transmission. The first programmable logic gate array is used to copy the video data signal and send it to the first computing unit, the second computing unit and the third computing unit through the first type of data link; the programmable logic gate array is used to reliably divide the video data signal into three streams and improve its reliability. The first computing unit, the second computing unit and the third computing unit are used to perform autonomous driving calculations based on the video data signal, and compare and vote on the three calculation results obtained to obtain the target calculation result. The control platform adopts a triple redundancy design to ensure the reliability of the system and the safety of control.

[0071] It should be noted that the order of the embodiments of the present application described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0072] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device, equipment, and storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0073] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0074] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A data branch acquisition system, characterized in that: The system includes a data acquisition module, a branch module and a control module; The data acquisition module includes at least one image acquisition unit, the image acquisition unit includes an image sensor and a deserializer, the branch module includes a first programmable logic gate array, and the control module includes a first computing unit, a second computing unit, and a third computing unit; The deserializer is used to deserialize the video differential signal sent by the image sensor to obtain a video data signal, and send the video data signal to the first programmable logic gate array through the first type of data link; The first programmable logic gate array is used to copy the video data signal and send it to the first computing unit, the second computing unit and the third computing unit through a first type of data link; The first computing unit, the second computing unit, and the third computing unit are used to perform autonomous driving calculations based on the video data signal, and compare and vote on the three calculated results to obtain a target calculation result.

2. The system according to claim 1, wherein: The branching module further includes a second programmable logic gate array, wherein the first programmable logic gate array and the second programmable logic gate array are communicatively connected via a second type of data link; The deserializer is further configured to copy the video data signal and send the signal to the second programmable logic gate array via the first type data link; The second programmable logic gate array is used to copy the received video data signal and send it to the first computing unit, the second computing unit and the third computing unit through the second type of data link.

3. The system according to claim 1, wherein: The system further includes a first storage module, which is connected to the first programmable logic gate array via a second type of data link. The first storage module is used to store backup data transmitted by the first programmable logic gate array.

4. The system according to claim 2, wherein: The system further includes a second storage module, which is connected to the second programmable logic gate array via the second type of data link, and the second storage module is used to store backup data transmitted by the second programmable logic gate array.

5. The system according to claim 2, wherein: The data acquisition module also includes a laser radar unit, an optical fiber combined navigation unit and a communication link unit; The laser radar unit, the optical fiber combined navigation unit and the communication link unit are all connected to the first programmable logic gate array, the second programmable logic gate array, the first computing unit, the second computing unit and the third computing unit through a first switch based on a third type of data link communication; The first programmable logic gate array and the second programmable logic gate array are both connected to the first computing unit, the second computing unit and the third computing unit through a second switch based on the third type of data link.

6. The system according to claim 2, wherein: The data acquisition module also includes a millimeter wave radar unit, an integrated navigation unit, and an atmospheric data sensor unit; The millimeter wave radar unit, the integrated navigation unit, and the atmospheric data sensor unit are all communicatively connected to the first programmable logic gate array, the second programmable logic gate array, the first computing unit, the second computing unit, and the third computing unit via a fourth type of data link; The first programmable logic gate array and the second programmable logic gate array are respectively connected to the first computing power unit, the second computing power unit and the third computing power unit via a fourth type of data link.

7. A data branching acquisition method, applied to the system according to any one of claims 1 to 6, characterized in that: The method comprises: The deserializer deserializes the video differential signal sent by the image sensor to obtain a video data signal, and sends the video data signal to the first programmable logic gate array through the first type of data link; The first programmable logic gate array copies the video data signal and sends it to the first computing unit, the second computing unit, and the third computing unit through the first type of data link; The first computing unit, the second computing unit, and the third computing unit respectively perform autonomous driving calculations based on the video data signal, and compare and vote on the three calculated results to obtain a target calculation result.

8. A data branching acquisition device, applied to the system according to any one of claims 1 to 6, characterized in that: The device comprises: a deserialization module, configured to deserialize the video differential signal sent by the image sensor to obtain a video data signal, and send the video data signal to the first programmable logic gate array through a first type of data link; A transmission module, configured for the first programmable logic gate array to copy the video data signal and send it to the first computing unit, the second computing unit, and the third computing unit through a first type of data link; A computing module is used for the first computing unit, the second computing unit, and the third computing unit to perform autonomous driving calculations based on the video data signal, respectively, and to compare and vote on the three calculated results to obtain a target calculation result.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the data branch acquisition system as described in claim 7.

10. An electronic device, characterized in that: It includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the data branch acquisition system as described in claim 7 by executing the instructions stored in the memory.

Images