Data acquisition system, data reinjection device and test system
By using RAW compression and decompression circuits in the data acquisition system and data reblocking device, the problem of information loss in RAW video data acquisition and reblocking in the prior art is solved, and accurate data reproduction and accurate development of the ECU are achieved.
Patent Information
- Application Number
- CN202420646625.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-01
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2034-04-01
AI Technical Summary
The existing data acquisition system is difficult to effectively collect and recall RAW video data, resulting in lost information and the real scenes during video shooting cannot be accurately reproduced, affecting the development and verification process of the ECU.
It provides a data acquisition system and a data rebill device, adopts a RAW compression circuit and a RAW decompression circuit, and realizes the compression and decompression processing of RAW video data through the data transmission circuit and video transmission unit to ensure the accuracy and completeness of the data.
Through compression and decompression processing, information loss is avoided, the accurate reproduction of RAW video data is ensured, and the accurate development and verification of the ECU is supported.
Smart Images

Figure CN222089647U_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and in particular to a data acquisition system, a data injection device and a test system. Background Art
[0002] In order to meet the needs of the vehicle ECU (Electronic Control Unit) development process, when a road vehicle is driving outside, it is necessary to use a data acquisition device to collect video data taken by the road vehicle, so that the video data can be subsequently injected back into the ECU being tested, verified or trained, that is, the domain controller in the form of a data stream, so as to verify, test or train the ECU being tested.
[0003] In the related art, a camera that shoots video data can output video data through an HDMI (High Definition Multimedia Interface) line. Furthermore, the video data output by HDMI is RGB video data. Correspondingly, a data acquisition device can receive the RGB video data and store it on a disk, and then perform back injection. However, with the development of technology, the video data output by some cameras is RAW video data, and the existing data acquisition devices that collect RGB video data are difficult to meet the demand. Summary of the invention
[0004] Based on this, it is necessary to provide a data acquisition system, a data injection device and a test system to address the above technical problems, which can acquire RAW video data.
[0005] In a first aspect, the present application provides a data acquisition system, the data acquisition system comprising a data acquisition device, the data acquisition device being adapted to be detachably mounted on a vehicle, the vehicle being equipped with a camera device capable of outputting a data stream of RAW video data; the data acquisition device comprising: a first data transmission circuit, and a RAW compression circuit capable of compressing the RAW video data;
[0006] The first data transmission circuit is provided with a data stream receiving unit, and the data stream receiving unit of the first data transmission circuit is used to receive the data stream of the RAW video data. The data transmission circuit is connected to the RAW compression circuit to transmit the RAW video data to the RAW compression circuit.
[0007] In one embodiment, the data acquisition device further comprises:
[0008] The storage component is connected to the RAW compression circuit and is used to store the compressed RAW video data.
[0009] In one embodiment, the RAW compression circuit is provided with a first data sharing connection portion; the first data transmission circuit further comprises:
[0010] a second data sharing connection portion;
[0011] a deserializer, connected to the data stream receiving unit, configured to receive the data stream of the RAW video data through the data stream receiving unit, and perform deserialization processing on the data stream of the RAW video data;
[0012] The first video transmission unit is connected to the deserializer and the second data sharing connection part respectively, and is used to receive the data stream of the RAW video data after deserialization, and transmit the deserialized RAW video data to the RAW compression circuit through the second data sharing connection part and the first data sharing connection part.
[0013] In one embodiment, the system further comprises:
[0014] A video bypass device, the video bypass device is suitable for being detachably installed on the vehicle, and is respectively connected to the camera device, the second domain controller of the vehicle, and the data acquisition device, for receiving the data stream of RAW video data sent by the camera device, and transmitting the data stream of RAW video data to the data acquisition device and the second domain controller respectively.
[0015] In one embodiment, the video bypass device includes: a deserialization unit, a first serialization unit, a second serialization unit, and a third video transmission unit;
[0016] The deserialization unit is connected to the camera device, and is used to receive the data stream of the RAW video data sent by the camera device, and perform deserialization processing on the data stream of the RAW video data to obtain the data stream of the RAW video data after the deserialization processing;
[0017] The third video transmission unit is connected to the deserialization unit, the first serial unit and the second serial unit respectively, and is used to transmit the data stream of the RAW video data after deserialization processing to the first serial unit and the second serial unit respectively;
[0018] The first serial unit is connected to the second domain controller, and is used to perform serialization processing on the received data stream of the RAW video data after the deserialization processing to obtain the data stream of the RAW video data after the serialization processing, and transmit the data stream of the RAW video data after the serialization processing to the second domain controller;
[0019] The second serial unit is connected to the data acquisition device, and is used to perform string addition processing on the data stream of the received RAW video data after deserialization processing to obtain the data stream of the RAW video data after string addition processing, and transmit the data stream of the RAW video data after string addition processing to the data acquisition device.
[0020] In the above data acquisition system, the data acquisition device is detachably connected to the vehicle, and the first data transmission circuit in the data acquisition device can receive the original video data captured by the camera device in the vehicle, that is, the data stream of RAW video data. The first data transmission circuit is connected to the RAW compression circuit to transmit the RAW video data from the first data transmission circuit to the RAW compression circuit, so that the RAW compression circuit compresses the RAW video data to obtain compressed RAW video data. Furthermore, it can be used for subsequent processing. For example, when the acquired RAW video data needs to be injected back into the domain controller or the acquired RAW video data needs to be analyzed, the compressed RAW video data in the data acquisition device can be directly or indirectly extracted and decompressed for injection or analysis. The decompressed RAW video data can accurately reproduce the real scene when the video is shot, avoiding information loss, and ensuring that the RAW video data can be accurately analyzed and processed and the domain controller can be accurately developed.
[0021] In a second aspect, the present application also provides a data reinjection device, which is connected to the first domain controller and is used to receive the compressed RAW video data obtained by the data acquisition device described in the above embodiment, decompress it, and reinject the decompressed RAW video data to the first domain controller.
[0022] In one embodiment, the data injection device includes:
[0023] A RAW decompression circuit, wherein the RAW decompression circuit is provided with a third data sharing connection part, and the RAW decompression circuit is used to decompress the compressed RAW video data to obtain the decompressed RAW video data;
[0024] A second data transmission circuit, wherein a data stream output unit and a fourth data sharing connection unit are provided on the second data transmission circuit, wherein the second data transmission circuit is connected to the third data sharing connection unit of the RAW decompression circuit via the fourth data sharing connection unit to obtain the decompressed RAW video data, and the second data transmission circuit is also connected to the first domain controller via the data stream output unit to inject the data stream of the decompressed RAW video data back to the first domain controller via the data stream output unit.
[0025] In one embodiment, the second data transmission circuit further includes:
[0026] a second video transmission unit, the second video transmission unit being connected to the fourth data sharing connection portion to receive the decompressed RAW video data through the fourth data sharing connection portion and the third data sharing connection portion;
[0027] A serializer is respectively connected to the second video transmission unit and the data stream output unit, and is used to receive the decompressed RAW video data, and perform string addition processing on the decompressed RAW video data to obtain a data stream of the string-added RAW video data, and inject the data stream of the string-added RAW video data back to the first domain controller through the data stream output unit.
[0028] In one embodiment, the data injection device further comprises:
[0029] A RAW decompression circuit, wherein the RAW decompression circuit is provided with a video stream output interface, and the RAW decompression circuit is used to decompress the compressed RAW video data to obtain the decompressed RAW video data;
[0030] The second data transmission circuit is provided with a video stream input interface and a data stream output unit. The second data transmission circuit is connected to the video stream output interface of the RAW decompression circuit via the video stream input interface and a video connection line to receive the video stream of the RAW video data. The second data transmission circuit is also connected to the first domain controller via the data stream output unit to inject the data stream of the decompressed RAW video data back to the first domain controller via the data stream output unit.
[0031] Through the data reinjection device of the present application, when the acquired RAW video data needs to be transmitted to the first domain controller, the data reinjection device is connected to the first domain controller, and the data reinjection device can receive the RAW video data compressed by the data acquisition device, and decompress the compressed RAW video data to restore the data stream of the RAW video data, thereby avoiding information loss, and then injecting the restored RAW video data data stream back to the first domain controller, ensuring that the RAW video data received by the first domain controller can accurately reproduce the real scene during video shooting, so as to accurately develop the first domain controller.
[0032] In a third aspect, the present application further provides a test system, which includes the data acquisition device described in any of the above embodiments and the data reinjection device described in any of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0034] FIG1 is a schematic diagram of the structure of a data acquisition device in one embodiment;
[0035] FIG2 is a schematic diagram of the structure of a data acquisition device in another embodiment;
[0036] FIG3 is a schematic diagram of the structure of a data acquisition device in another embodiment;
[0037] FIG4 is a schematic diagram of the structure of a data re-injection device in one embodiment;
[0038] FIG5 is a schematic diagram of the structure of a data re-injection device in another embodiment;
[0039] FIG6 is a schematic diagram of the structure of a data re-injection device in another embodiment;
[0040] FIG7 is a schematic diagram of the structure of a data re-injection device in another embodiment;
[0041] FIG8 is a schematic structural diagram of a data acquisition device in yet another embodiment;
[0042] FIG. 9 is a schematic diagram of the structure of a data acquisition device in yet another embodiment.
[0043] Explanation of the accompanying drawings: 1-data acquisition device, 11-first data transmission circuit, 111-deserializer, 112-first video transmission unit, 12-RAW compression circuit, 13-storage component, 3-data reinjection device, 31-RAW decompression circuit, 32-second data transmission circuit, 321-second video transmission unit, 322-serializer, 2-vehicle, 21-camera device, 22-second domain controller, 4-first domain controller, 5-video bypass device, 51-deserialization unit, 52-third video transmission unit, 53-first serial unit, 54-second serial unit. DETAILED DESCRIPTION
[0044] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.
[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.
[0046] It can be understood that the terms "first", "second", etc. used in the present application can be used in this article to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element.
[0047] Spatially relative terms such as "under," "beneath," "below," "under," "above," "above," and the like may be used herein to describe the relationship of an element or feature shown in the figures to other elements or features. It should be understood that, in addition to the orientations shown in the figures, spatially relative terms also include different orientations of the device in use and operation. For example, if the device in the accompanying drawings is flipped, an element or feature described as "under other elements" or "under it" or "under it" will be oriented as being "above" the other elements or features. Thus, the exemplary terms "under" and "under" may include both upper and lower orientations. In addition, the device may also include additional orientations (e.g., rotated 90 degrees or other orientations), and the spatial descriptors used herein are interpreted accordingly.
[0048] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intermediate element. In addition, the "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc. if there is transmission of electrical signals or data between the connected objects.
[0049] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof.
[0050] During the development of algorithms (such as autonomous driving algorithms and assisted driving algorithms), it is necessary to train, verify, and test the algorithms (such as neural networks) in the domain controller, which means that various video image data need to be injected into the domain controller. Some of the video image data can be collected by the vehicle-mounted data acquisition system.
[0051] At present, in the process of algorithm development, the existing data acquisition system can only realize the acquisition and storage of video data in RGB format, but the video data output by some cameras is RAW video data, not RGB video data. Therefore, in order to achieve compatibility with RAW video data, the vehicle-mounted data acquisition system in the related technology usually converts RAW data into YUV or RGB video data, and then stores it. Furthermore, the data acquisition device in the related technology is usually equipped with: RAW-YUV conversion device, YUV data compression device and storage module. Correspondingly, in the re-injection device, a YUV data decompression device and a YUV-RAW conversion device are usually provided.
[0052] In the data acquisition device in the related art, taking the conversion to YUV format as an example, the RAW-YUV format conversion process is usually performed by a RAW-YUV conversion device, specifically, the numerical value in the RAW data is converted into the numerical value in the RGB data based on the conversion formula, and then the numerical value of the RGB data is converted into the numerical value of the YUV data. However, information loss often occurs during this data conversion process. Even if the RAW format data is finally restored, the information contained therein will still be a lot of loss compared to the image originally given by the camera. This will lead to the inability to accurately reproduce the real scene during video shooting during the verification, testing or training of the domain controller under development, introducing greater distortion.
[0053] In one embodiment, as shown in FIG. 1 , the present application provides a data acquisition device 1 , and further provides a data acquisition system including the data acquisition device 1 .
[0054] The data acquisition device 1 comprises a first data transmission circuit 11 and a RAW compression circuit 12. The data acquisition device 1 is suitable for being detachably mounted on a vehicle 2, and a camera device 21 capable of outputting a data stream of RAW video data is mounted on the vehicle 2.
[0055] The detachable installation can be understood as the data acquisition device can be installed on the vehicle and can also be removed from the vehicle, and the installation includes: the data acquisition device is in the vehicle and moves with the vehicle, and: the data acquisition device is directly or indirectly (for example, via other devices) electrically connected to the electronic circuits and / or electronic devices on the vehicle.
[0056] The data acquisition device 1 can be directly placed on the vehicle 2, or installed on the vehicle 2 through a mounting bracket or other structures. The electronic circuits and electronic devices therein can be original to the vehicle 2, or can be retrofitted. Furthermore, through the "electrical connection", data collection can be achieved. For example, when the data acquisition device 1 is directly or indirectly electrically connected to the camera device 21, RAW video data from the camera device 21 can be obtained.
[0057] The first data transmission circuit 11 is provided with a data stream receiving unit G1, and the first data transmission circuit 11 receives the data stream of the RAW video data in the camera device 21 through the data stream receiving unit G1. The RAW compression circuit 12 is connected to the first data transmission circuit 11, and is used to receive the RAW video data and compress the RAW video data. The data stream receiving unit G1 can be a GMSL (Gigabit Multimedia Serial Links) type interface, or a FPDLink (Flat Panel Display Link) type interface.
[0058] It can be understood that the vehicle 2 is a road running vehicle or a road test vehicle, and is a vehicle installed with various sensors and traveling on the road for collecting data. The vehicle 2 is installed with an image sensor, i.e., a camera 21. In order to obtain the data stream of the original video image data captured by the camera 21, i.e., the data stream of RAW video data, a data acquisition device 1 can be installed on the vehicle 2. The data acquisition device 1 is directly or indirectly connected to the camera 21 to receive the data stream of RAW video data transmitted by the camera 21.
[0059] Specifically, the data acquisition device 1 may include a first data transmission circuit 11 and a RAW compression circuit 12. If the camera device 21 does not perform a serialization process on the data stream of the RAW video data, the transmission distance of the data stream of the RAW video data will be very short. Therefore, the camera device 21 outputs a data stream of the RAW video data after the serialization process, and then transmits the data stream of the RAW video data after the serialization process through the GMSL data line. The first data transmission circuit 11 may include a data stream receiving unit G1. When the data acquisition device 1 is installed on the vehicle 2, the first data transmission circuit 11 is directly or indirectly connected to the camera device 21 through the data stream receiving unit G1 to receive the data stream of the RAW video data after the serialization process, and deserialize the data stream of the RAW video data after the serialization process.
[0060] After receiving the deserialized RAW video data, the RAW compression circuit 12 can compress the deserialized RAW video data. The compression method can be conventional compression methods such as H.254 and H.265. In one example, the RAW compression circuit 12 can include a processing chip for video compression and can also include its peripheral circuits. The processing chip can be, for example, Hi3510, Vweb's VW2005 and other chips. In other examples, the RAW compression circuit 12 can also be implemented based on FPGA or other processors. Any existing or improved RAW video data compression algorithm in the field can be applied to this compression component to constitute an implementation method of the present application. Among them, the RAW compression circuit 12 of the present application can also perform some pre-processing work such as image enhancement and image segmentation on the deserialized RAW video data before compressing the deserialized RAW video data to reduce noise and improve image quality and visualization. The pre-processing work can be implemented by the RAW compression circuit or by the first video transmission unit.
[0061] In the above data acquisition system, the data acquisition device is detachably connected to the vehicle, and the first data transmission circuit in the data acquisition device can receive the original video data captured by the camera device in the vehicle, that is, the data stream of RAW video data. The first data transmission circuit is connected to the RAW compression circuit to transmit the RAW video data from the first data transmission circuit to the RAW compression circuit, so that the RAW compression circuit compresses the RAW video data to obtain compressed RAW video data. Furthermore, it can be used for subsequent processing. For example, when the acquired RAW video data needs to be injected back into the domain controller or the acquired RAW video data needs to be analyzed, the compressed RAW video data in the data acquisition device can be directly or indirectly extracted and decompressed for injection or analysis. The decompressed RAW video data can accurately reproduce the real scene when the video is shot, avoiding information loss, and ensuring that the RAW video data can be accurately analyzed and processed and the domain controller can be accurately developed.
[0062] In one embodiment, as shown in Fig. 2, the data acquisition device 1 further includes a storage component 13. The storage component 13 is connected to the RAW compression circuit 12 and is used to store the compressed RAW video data.
[0063] In the application, after the compression of the RAW video data is completed, the compressed RAW video data can be stored in the storage component 13 so that the RAW video data can be called at any time later.
[0064] In this embodiment, the storage component 13 can be not only a component of the data acquisition device 1, but also an external memory of the data acquisition device 1. When the storage component 13 is arranged outside the data acquisition device 1, the storage component 13 can be connected to the data acquisition device 1, and then the storage component 13 can be connected to the RAW compression circuit 12 in the data acquisition device 1 to achieve the storage of the compressed RAW video data. When data extraction is required, the storage component 13 is connected to a device capable of extracting information to achieve the extraction of the compressed RAW video data. Among them, even if the storage component 13 is a component of the data acquisition device 1, the storage component 13 can also be disassembled from the data acquisition device 1 to achieve the extraction of the compressed RAW video data.
[0065] In one embodiment, as shown in Fig. 3, the first data transmission circuit 11 further includes: a second data sharing connection part P1, a deserializer 111 and a first video transmission unit 112. The RAW compression circuit 12 is provided with a first data sharing connection part P2.
[0066] In one example, the first data transmission circuit 11 and the RAW compression circuit 12 may be two circuits on a circuit board, and the first data transmission circuit 11 may directly transmit the RAW video data to the RAW compression circuit 12. In another example, the RAW compression circuit 12 may be provided with a first data sharing connection portion P2, and the first data transmission circuit 11 may be provided with a second data sharing connection portion P1, and the first data transmission circuit 11 is connected to the RAW compression circuit 12 through the second data sharing connection portion P1 and the first data sharing connection portion P2, and the RAW video data in the first data transmission circuit 11 is transmitted to the RAW compression circuit 12 through the second data sharing connection portion P1 and the first data sharing connection portion P2, so that the RAW compression circuit 12 can compress the RAW video data.
[0067] The second data sharing connection part P2 and the first data sharing connection part P1 may be PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) type interfaces, or may be CXL (Compute Express Link) type interfaces.
[0068] The deserializer 111 is connected to the data stream receiving unit G1 . The deserializer 111 is used to receive the data stream of the RAW video data through the data stream receiving unit G1 , and perform deserialization processing on the data stream of the RAW video data.
[0069] In applications, in order to enable the RAW video data collected by the camera device 21 to be transmitted stably and over long distances, the camera device 21 usually performs serialization processing on the collected RAW video data. Therefore, the first data transmission circuit 11 of the present application needs to include a deserializer 111. The deserializer 111 can receive the data stream of the RAW video data after the serialization processing through the data stream receiving unit G1, and deserialize the data stream of the RAW video data after the serialization processing.
[0070] The first video transmission unit 112 is connected to the deserializer 111 and the second data sharing connection part P1 respectively, and is used for receiving the data stream of the deserialized RAW video data, and transmitting the deserialized RAW video data to the RAW compression circuit 12 through the second data sharing connection part P1 and the first data sharing connection part P2.
[0071] Among them, MIPI (Mobile Industry Processor Interface) is an open standard and a specification developed for mobile application processors, so the first video transmission unit 112 can be provided with or connected to a first MIPI input interface, the deserializer 111 can be provided with or connected to a first MIPI output interface, and the first video transmission unit 112 is connected to the deserializer 111 through the first MIPI output interface and the first MIPI input interface, so that the deserializer 111 can transmit the data stream of the deserialized RAW video data to the first video transmission unit 112.
[0072] In other examples, DVP (digital video port) and LVDS (Low-Voltage Differential Signaling) interfaces are also common interface forms for transmitting video streams and data streams. Therefore, the first video transmission unit 112 may be provided with a first DVP input interface, and the deserializer 111 may be provided with a first DVP output interface. The first video transmission unit 112 is connected to the deserializer 111 through the first DVP output interface and the first DVP input interface, so that the deserializer 111 can transmit the deserialized RAW video data data stream to the first video transmission unit 112; or, the first video transmission unit 112 may be provided with a first LVDS input interface, and the deserializer 111 may be provided with a first LVDS output interface. The first video transmission unit 112 is connected to the deserializer 111 through the first LVDS output interface and the first LVDS input interface, so that the deserializer 111 can transmit the deserialized RAW video data data stream to the first video transmission unit 112.
[0073] In the application, after the second video transmission unit 112 receives the data stream of the RAW video data after deserialization, the first video transmission unit 112 can transmit the deserialized RAW video data to the RAW compression circuit 12 through the second data sharing connection part P1 and the first data sharing connection part P2. The second data sharing connection part P1 and the first data sharing connection part P2 can be PCIe interfaces. Among them, the deserializer 111, the first video transmission unit 112, the data stream receiving part G1 and the second data sharing connection part P1 can be a circuit structure set on the first board, that is, the first data transmission circuit 11 can be a circuit structure set on the first board, and the first board can be a single circuit board or a combination of multiple circuit boards.
[0074] In one embodiment, as shown in Figure 4, the present application also provides a data reinjection device 3, which is connected to the first domain controller 4, and can obtain the compressed RAW video data obtained by the data acquisition device 1, and decompress the compressed RAW video data, and inject the decompressed RAW video data back to the first domain controller 4.
[0075] In one example, after the data acquisition device 1 completes the acquisition of the RAW video data, the data acquisition device 1 (or the storage component 13, or other carriers for storing the RAW video data compressed by the data acquisition device 1) can be connected to the data reinjection device 3, and the compressed RAW video data can be imported into the data reinjection device 3. The data reinjection device 3 can then decompress the compressed RAW video data. On this basis, when the data reinjection device 3 is connected to the first domain controller 4, the data stream of the decompressed RAW video data can be injected back to the first domain controller 4.
[0076] Through the data reinjection device of the present application, when the acquired RAW video data needs to be reinjected into the first domain controller, the data reinjection device can directly decompress the compressed RAW video data to restore the data stream of the RAW video data, thereby avoiding information loss, and then inject the restored data stream of the RAW video data back into the first domain controller, thereby ensuring that the RAW video data received by the first domain controller can accurately reproduce the real scene during video shooting, so as to accurately develop the first domain controller.
[0077] In one embodiment, as shown in FIG. 5 , the data re-injection device 3 includes a RAW decompression circuit 31 and a second data transmission circuit 32 .
[0078] The RAW decompression circuit 31 is provided with a third data sharing connection part P3. The RAW decompression circuit 31 is used to decompress the compressed RAW video data to obtain decompressed RAW video data.
[0079] In one application, when data reinjection is required, the data reinjection device 3 can be connected to the data acquisition device 1. Specifically, the RAW decompression circuit 31 can be connected to the storage component 13. The RAW decompression circuit 31 receives the compressed RAW video data stored in the storage component 13 and decompresses the compressed RAW video data.
[0080] The data reinjection device 3 may also include a memory. When the data reinjection device 3 is connected to the data acquisition device 1, the memory is connected to the storage component 13 in the data reinjection device 3 to receive and store the compressed RAW video data. The compressed RAW video data is pre-stored in the memory. When the first domain controller 4 is developed, there is no need to connect the data reinjection device 3 with the data acquisition device 1, and data reinjection can be performed through the pre-stored compressed RAW video data.
[0081] The second data transmission circuit 32 is provided with a fourth data sharing connection part P4 and a data stream output part G2. The second data transmission circuit 32 is connected to the third data sharing connection part P3 of the RAW decompression circuit 31 through the fourth data sharing connection part P4, and is connected to the first domain controller 4 through the data stream output part G2, for obtaining the decompressed RAW video data, performing string addition processing on the decompressed RAW video data, and returning the string added RAW video data to the first domain controller 4. Among them, the third data sharing connection part P3 and the fourth data sharing connection part P4 can be PCIe type interfaces or CXL type interfaces.
[0082] When the data reinjection device 3 is connected to the first domain controller 4, that is, when the second data transmission circuit 32 is connected to the first domain controller 4, the second data transmission circuit 32 can receive the RAW video data decompressed by the RAW decompression circuit 31 through the fourth data sharing connection part P4 and the third data sharing connection part P3, and add strings to the decompressed RAW video data, and then inject the added string RAW video data back to the first domain controller 4.
[0083] It can be understood that the domain controller is equipped with an ISP (Image Signal Processor) module, which is used to process the original image data captured from the image sensor, that is, RAW video data. During the normal operation of the vehicle, the camera device, and the domain controller, the ISP module usually performs ISP processing on the RAW video data fed back by the camera device, and then performs at least one subsequent algorithm processing such as intelligent driving algorithm, control algorithm, recognition algorithm, etc. on the video data after ISP processing. The first domain controller is a domain controller to be developed or under development, so the first domain controller 4 needs to receive the RAW video data captured by the camera device 21 in the test vehicle through the data acquisition device 1 and the data injection device 3 to train or verify the first domain controller 4 and its ISP algorithm.
[0084] In one embodiment, as shown in FIG. 6 , the second data transmission circuit 32 further includes: a second video transmission unit 321 and a serializer 322 .
[0085] The second video transmission unit 321 is connected to the fourth data sharing connection portion P4 to receive the decompressed RAW video data through the fourth data sharing connection portion P4 and the third data sharing connection portion P3.
[0086] The serializer 322 is connected to the second video transmission unit 321 and the data stream output part G2 respectively, and is used for receiving the decompressed RAW video data, and performing string processing on the decompressed RAW video data to obtain a data stream of the string-processed RAW video data, and injecting the data stream of the string-processed RAW video data back to the first domain controller 4 through the data stream output part G2.
[0087] In the data re-injection device 3, when the RAW decompression circuit 31 completes the decompression of the compressed RAW video data, the decompressed RAW video data can be obtained. At this time, the RAW decompression circuit 31 can send the decompressed RAW video data to the second video transmission unit 321 through the third data sharing connection part P3 and the fourth data sharing connection part P4. The second video transmission unit 321 then sends the un-stringified data stream of the RAW video data to the serializer 322, so as to perform string processing on the decompressed, un-stringified data stream of the RAW video data, so that the data stream of the string-added RAW video data can be transmitted to the first domain controller 4 over a long distance through the data stream output part G2. Among them, the serializer 322, the second video transmission unit 321, the data stream output part G2 and the fourth data sharing connection part P4 can be a circuit structure set on the second board, that is, the second data transmission circuit can be a circuit structure set on the second board, and the second board can be a single circuit board or a combination of multiple circuit boards.
[0088] In the application, the second video transmission unit 321 may be provided with a second MIPI output interface, the serializer 322 may be provided with or connected to a second MIPI input interface, and the second video transmission unit 321 is connected to the serializer 322 through the second MIPI output interface and the second MIPI input interface, so that the second video transmission unit 321 can transmit the data stream of the decompressed RAW video data to the serializer 322. Alternatively, the second video transmission unit 321 may be provided with or connected to a second DVP output interface, the serializer 322 may be provided with or connected to a second DVP input interface, and the second video transmission unit 321 is connected to the serializer 322 through the second DVP output interface and the second DVP input interface, so that the second video transmission unit 321 can transmit the data stream of the decompressed RAW video data to the serializer 322. In another example, a second LVDS output interface may be provided on the second video transmission unit 321, and a second LVDS input interface may be provided on the serializer 322. The second video transmission unit 321 is connected to the serializer 322 through the second LVDS output interface and the second LVDS input interface, so that the second video transmission unit 321 can transmit the data stream of the decompressed RAW video data to the serializer 322.
[0089] In one embodiment, a serial adding device may be provided on the RAW decompression circuit 31, and a deserializing device may be provided on the second data transmission circuit 32. The serial adding device is connected to the third data sharing connection part P3, and the deserializing device is respectively connected to the second video transmission unit and the serializer.
[0090] After the RAW decompression circuit 31 decompresses the compressed RAW video data, it transmits the decompressed RAW video data to the serialization device, which performs serialization on the decompressed RAW video data, and then transmits the serialized RAW video data to the second data transmission circuit 32 through the third data sharing connection part P3 and the fourth data sharing connection part P4. After that, the deserialization device can deserialize the RAW video data after the serialization process, and then transmit the deserialized RAW video data to the second video transmission unit, which can analyze the deserialized RAW video data and then transmit the analyzed RAW video data to the serializer. Exemplarily, the second video transmission unit can perform fault injection, synchronization processing and other analysis processing on the RAW video data after deserialization. The specific steps for the second video transmission unit to perform synchronization processing on the RAW video data after deserialization are as follows: the second video transmission unit compares the current clock with the timestamp, and when the current time reaches the timestamp, the deserialized RAW video data is output to the serializer, and the serializer performs serialization processing on the deserialized RAW video data.
[0091] In one embodiment, as shown in FIG7 , the data re-injection device 3 includes a RAW decompression circuit 31 and a second data transmission circuit 32. The RAW decompression circuit 31 is provided with a video stream output interface H1, and the RAW decompression circuit 31 is used to decompress the compressed RAW video data to obtain the decompressed RAW video data. The second data transmission circuit 32 is provided with a video stream input interface H2 and a data stream output unit G2. The second data transmission circuit 32 is connected to the video stream output interface H1 of the RAW decompression circuit 31 through the video stream input interface H2 and the video connection line to receive the video stream of the decompressed RAW video data. The second data transmission circuit 32 is also connected to the first domain controller 4 through the data stream output unit G2, and is used to re-inject the data stream of the decompressed RAW video data to the first domain controller 4 through the data stream output unit G2.
[0092] In the application, the RAW decompression circuit 31 may further include a video stream output interface H1, and a video stream input interface H2 may further be provided on the second data transmission circuit 32. In the data reinjection device 3, the decompressed RAW video data may also be transmitted from the RAW decompression circuit 31 to the second video transmission unit 321. The video stream output interface H1 and the video stream input interface H2 can be HDMI interfaces. Although the HDMI interface generally does not accept data in RAW format, the RAW decompression circuit 31 can, after decompressing the compressed RAW video data, send the decompressed RAW video data to the second data transmission circuit 32, divide the RAW video data into three parts according to a preset segmentation method to obtain a first part of data, a second part of data and a third part of data, and put them into the red channel, green channel and blue channel of the red, green and blue color mode respectively, and based on the red channel, green channel and blue channel, transmit the first part of data, the second part of data and the third part of data of the RAW video data to the second video transmission unit 321 in the second data transmission circuit 32 through the HDMI line (i.e., the video connection line) The specific data transmission method can be found in the record in patent CN115665357B, which will not be repeated here.
[0093] The second data transmission circuit may include a converter and a serializer. The converter is connected between the serializer and the video stream input interface H2, and is used to convert the video stream of the decompressed RAW video data (such as a video stream transmitted via HDMI) into the required data stream (such as a MIPI data stream), and then send it to the serializer. The serializer can add strings to obtain the data stream of the decompressed RAW video data (such as a GMSL data stream), and then inject it back to the first domain controller.
[0094] In one example, the RAW decompression circuit 31 may include a processing chip for video decompression, and may also include its peripheral circuits, such as Hi3510, etc. In other examples, the RAW decompression circuit 31 may also be implemented based on FPGA or other processors, and any decompression implementation scheme that can correspond to the compression implementation scheme of the RAW compression circuit 12 can be applied to the RAW decompression circuit 31 to constitute an implementation method of the present application. In addition, a fault simulation circuit may be integrated or connected in the second video transmission unit to inject faults into the RAW video data. For example, part of the data in the RAW video data may be deleted and replaced with black block data to simulate an occlusion fault. For another example, part of the RAW video data may be discarded to simulate a frame loss fault.
[0095] In one embodiment, the present application further provides a test system, which includes the data acquisition system described in any of the above embodiments and the data reinjection device described in any of the above embodiments.
[0096] The test system of the present application, when performing data acquisition work, connects the data acquisition device 1 to the camera device 21 in the vehicle 2, so that the RAW video data is directly transmitted through the data stream receiving unit G1, the deserializer 111, the first video transmission unit 112, the second data sharing connection unit P1, the first data sharing connection unit P2 and the RAW compression circuit 12, and the RAW video data is directly compressed in the RAW compression circuit 12, and the compressed RAW video data is stored in the storage component 13. ; When performing data reinjection, first connect the data acquisition device 1 with the data reinjection device 3 so that the RAW decompression circuit 31 in the data reinjection device 3 can receive the compressed RAW video data in the storage component 13, and then connect the data reinjection device 3 with the first domain controller 4, so that the RAW video data can be directly transmitted to the first domain controller 4 through the third data sharing connection part P3, the RAW decompression circuit 31, the second video transmission unit 321, the fourth data sharing connection part P4, the serializer 322, and the data stream output part G2. By improving the RAW-YUV conversion device, YUV data compression device, YUV data decompression device, and YUV-RAW conversion device in the related technology into a RAW compression circuit 12 and a RAW decompression circuit 31, the format conversion of the RAW video data is avoided, thereby avoiding the information loss that occurs during the data conversion process, and ensuring that in the process of developing the first domain controller 4, the RAW video data received by the first domain controller 4 can more realistically restore the scene captured by the camera device 21, thereby ensuring that the development of the first domain controller 4 is more accurate.
[0097] In one embodiment, as shown in Figure 8, the data acquisition system may also include a video bypass device 5, which is suitable for being detachably connected to the vehicle 2, and is respectively connected to the camera device 21 of the vehicle 2, the second domain controller 22 of the vehicle 2, and the data acquisition device 1, for receiving the data stream of RAW video data sent by the camera device 21, and transmitting the data stream of RAW video data to the data acquisition device 1 and the second domain controller 22, respectively.
[0098] During the driving process of the vehicle 2, the video data captured by the camera device 21 needs to be transmitted not only to the data acquisition device 1 for subsequent development of the first domain controller 4, but also to the second domain controller 22 of the vehicle 2. The second domain controller 22 is a domain controller that has been developed and can be used normally in the vehicle (of course, it may also be a domain controller that has not been fully developed but has been installed in the vehicle). Only by transmitting the RAW video data to the second domain controller 22 can the normal operation of the vehicle 2 be guaranteed. Therefore, a video bypass device 5 can be connected between the vehicle 2 and the data acquisition device 1. The video bypass device 5 can receive each channel of RAW video data in the camera device 21 (for example, a data stream of n channels of RAW video data), and copy the RAW video data into two identical RAW video data (for example, a data stream of 2n channels of RAW video data), and transmit the copied RAW video data to the data acquisition device 1 and the second domain controller 22 respectively, so as to meet the data acquisition requirements of the data acquisition device 1 and the normal operation requirements of the vehicle 2 at the same time.
[0099] In one embodiment, as shown in FIG. 9 , the video bypass device 5 includes: a third board card, and the third board card includes: a deserialization unit 51 , a third video transmission unit 52 , a first serial unit 53 and a second serial unit 54 .
[0100] The deserialization unit 51 is connected to the camera device 21, and is used to receive the data stream of the RAW video data sent by the camera device 21, and deserialize the data stream of the RAW video data to obtain the data stream of the deserialized RAW video data. The third video transmission unit 52 is respectively connected to the deserialization unit 51, the first serial unit 53 and the second serial unit 54, and is used to transmit the data stream of the deserialized RAW video data to the first serial unit 53 and the second serial unit 54 respectively.
[0101] The first serial unit 53 is connected to the second domain controller 22, and is used to perform string addition processing on the data stream of the received RAW video data after deserialization processing, obtain the data stream of the RAW video data after string addition processing, and transmit the data stream of the RAW video data after string addition processing to the second domain controller. The second serial unit 54 is connected to the data acquisition device 1, and is used to perform string addition processing on the data stream of the received RAW video data after deserialization processing, obtain the data stream of the RAW video data after string addition processing, and transmit the data stream of the RAW video data after string addition processing to the data acquisition device 1.
[0102] It can be understood that the video bypass device 5 is detachably installed on the vehicle 2. In order to ensure stable transmission of the signal, the RAW video data after serialization output by the camera device 21 is provided with a GMSL interface on the deserialization unit 51 of the video bypass device 5, so that the deserialization component 51 can stably receive the RAW video data after serialization sent by the camera device 21, and deserialize the RAW video data after serialization to obtain the deserialized RAW video data.
[0103] Since both the second domain controller 22 and the data acquisition component 1 need to receive RAW video data, the function of the third video transmission unit 52 is to receive one path of deserialized RAW video data, and copy the one path of deserialized RAW video data into two paths of completely identical deserialized RAW video data, and then transmit the two paths of deserialized RAW video data to the first serial unit 53 and the second serial unit 54 respectively.
[0104] The first serial unit 53 is connected to the second domain controller 22, and is used to perform string processing on the RAW video data after deserialization processing, so as to transmit the RAW video data after string processing to the second domain controller 22. The second serial unit 54 is connected to the data acquisition device 1, and is used to perform string processing on the RAW video data after deserialization processing, so as to transmit the RAW video data after string processing to the data acquisition device 1. In the application, the deserialization unit 51, the third video transmission unit 52, the first serial unit 53 and the second serial unit 54 can be a circuit structure set on the third board, and the third board can be a single circuit board or a combination of multiple circuit boards.
[0105] In the application, the RAW video data of the camera device in the test vehicle is obtained through the test system of the present application, and the RAW video data is directly compressed and decompressed without format conversion, so that the first domain controller can receive RAW video data with higher information integrity, thereby achieving better development effect of the first domain controller.
[0106] In one example, the first board, the second board, and the third board mentioned in the above embodiments can be implemented by a combination board; specifically, the combination board may include a plurality of main boards and a plurality of sub-boards, and the functions of each main board are the same. Since the functions of the first video transmission unit, the second video transmission unit, and the third video transmission unit are mainly to realize the transmission of RAW video data, they can all be realized by the main board. Specifically, the main board is provided with a video transmission unit, which can be switched in three states, namely:
[0107] The first state is used to implement the function of the first video transmission unit;
[0108] The second state is used to implement the function of the second video transmission unit;
[0109] The third state is used to implement the function of the third video transmission unit;
[0110] In addition, the mainboard may be provided with a PCIe interface, which can be used to realize the function of a data sharing connection unit in various states.
[0111] Since the first video transmission unit, the second video transmission unit, and the third video transmission unit are all used to realize the transmission of video data, the internal circuits often have a certain degree of duplication. Furthermore, by realizing three different states of the same video transmission unit, it is convenient to realize circuit reuse and avoid designing and producing a mainboard circuit for each purpose, which facilitates large-scale manufacturing and helps reduce costs.
[0112] Several daughter boards include:
[0113] A first daughter board is provided with a serializer;
[0114] The second daughter board is provided with a deserializer;
[0115] The third sub-board is provided with a converter for realizing format conversion of the video stream, for example, converting the HDMI video stream into the MIPI data stream.
[0116] The main board and any sub-board can be assembled and connected in a detachable manner.
[0117] Furthermore, the first board can be realized by assembling a main board (provided with a video transmission unit in a first state) and a second sub-board (provided with a deserializer in the first board); the second board can be realized by assembling a main board (provided with a video transmission unit in a second state) and a first sub-board (provided with a serializer in the second board); or it can be realized by assembling a main board (provided with a video transmission unit in a second state), a first sub-board (provided with a serializer in the second board), and a third sub-board (provided with a converter); the third board can be realized by assembling a main board (provided with a video transmission unit in a third state), a second sub-board (provided with a deserializer in the third board), and multiple first sub-boards (respectively provided with different serializers in the third board).
[0118] Through the above scheme, the main board and various sub-boards can be mass-produced first, and then, when producing and assembling data acquisition devices, data bypass devices, and data reinjection devices, suitable main boards and sub-boards can be selected for assembly. This helps to simplify production and reduce costs, and can help take into account other more diverse application scenarios. For example, in the scenario of reinjection of simulated video data, the required main board and sub-board combination can also be used to meet the needs.
[0119] In the description of this specification, the description with reference to the terms "some embodiments", "other embodiments", "ideal embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example.
[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0121] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A data acquisition system, characterized in that: It includes a data acquisition device, the data acquisition device is suitable for being detachably mounted on a vehicle, and the vehicle is equipped with a camera device capable of outputting a data stream of RAW video data; The data acquisition device comprises: a first data transmission circuit, and a RAW compression circuit capable of compressing the RAW video data; The first data transmission circuit is provided with a data stream receiving unit, and the data stream receiving unit of the first data transmission circuit is used to receive the data stream of the RAW video data. The first data transmission circuit is connected to the RAW compression circuit to transmit the RAW video data to the RAW compression circuit.
2. The data acquisition system according to claim 1, characterized in that: The data acquisition device also includes: The storage component is connected to the RAW compression circuit and is used to store the compressed RAW video data.
3. The data acquisition system according to claim 1, characterized in that: The RAW compression circuit is provided with a first data sharing connection part; the first data transmission circuit further comprises: a second data sharing connection portion; a deserializer, connected to the data stream receiving unit, configured to receive the data stream of the RAW video data through the data stream receiving unit, and perform deserialization processing on the data stream of the RAW video data; The first video transmission unit is connected to the deserializer and the second data sharing connection part respectively, and is used to receive the data stream of the RAW video data after deserialization, and transmit the deserialized RAW video data to the RAW compression circuit through the second data sharing connection part and the first data sharing connection part.
4. The data acquisition system according to claim 1, characterized in that: The data acquisition system also includes: A video bypass device, the video bypass device is suitable for being detachably installed on the vehicle, and is respectively connected to the camera device, the second domain controller of the vehicle, and the data acquisition device, for receiving the data stream of RAW video data sent by the camera device, and transmitting the data stream of RAW video data to the data acquisition device and the second domain controller respectively.
5. The data acquisition system according to claim 4, characterized in that: The video bypass device comprises: a deserialization unit, a first serialization unit, a second serialization unit and a third video transmission unit; The deserialization unit is connected to the camera device, and is used to receive the data stream of the RAW video data sent by the camera device, and perform deserialization processing on the data stream of the RAW video data to obtain the data stream of the RAW video data after the deserialization processing; The third video transmission unit is connected to the deserialization unit, the first serial unit and the second serial unit respectively, and is used to transmit the data stream of the RAW video data after deserialization processing to the first serial unit and the second serial unit respectively; The first serial unit is connected to the second domain controller, and is used to perform serialization processing on the received data stream of the RAW video data after the deserialization processing to obtain the data stream of the RAW video data after the serialization processing, and transmit the data stream of the RAW video data after the serialization processing to the second domain controller; The second serial unit is connected to the data acquisition device, and is used to perform string addition processing on the data stream of the received RAW video data after deserialization processing to obtain the data stream of the RAW video data after string addition processing, and transmit the data stream of the RAW video data after string addition processing to the data acquisition device.
6. A data re-injection device, characterized in that: The data reinjection device is connected to the first domain controller, and is used to decompress the compressed RAW video data obtained by the data acquisition system according to any one of claims 1 to 5, and reinject the decompressed RAW video data to the first domain controller.
7. The data re-injection device according to claim 6, characterized in that: The data re-injection device comprises: A RAW decompression circuit, wherein the RAW decompression circuit is provided with a third data sharing connection part, and the RAW decompression circuit is used to decompress the compressed RAW video data to obtain the decompressed RAW video data; A second data transmission circuit, wherein a data stream output unit and a fourth data sharing connection unit are provided on the second data transmission circuit, wherein the second data transmission circuit is connected to the third data sharing connection unit of the RAW decompression circuit via the fourth data sharing connection unit to obtain the decompressed RAW video data, and the second data transmission circuit is also connected to the first domain controller via the data stream output unit to inject the data stream of the decompressed RAW video data back to the first domain controller via the data stream output unit.
8. The data re-injection device according to claim 7, characterized in that: The second data transmission circuit further includes: a second video transmission unit, the second video transmission unit being connected to the fourth data sharing connection portion to receive the decompressed RAW video data through the fourth data sharing connection portion and the third data sharing connection portion; A serializer is respectively connected to the second video transmission unit and the data stream output unit, and is used to receive the decompressed RAW video data, and perform string addition processing on the decompressed RAW video data to obtain a data stream of the string-added RAW video data, and inject the data stream of the string-added RAW video data back to the first domain controller through the data stream output unit.
9. The data re-injection device according to claim 6, characterized in that: The data re-injection device also includes: A RAW decompression circuit, wherein the RAW decompression circuit is provided with a video stream output interface, and the RAW decompression circuit is used to decompress the compressed RAW video data to obtain the decompressed RAW video data; The second data transmission circuit is provided with a video stream input interface and a data stream output unit. The second data transmission circuit is connected to the video stream output interface of the RAW decompression circuit via the video stream input interface and a video connection line to receive the video stream of the RAW video data. The second data transmission circuit is also connected to the first domain controller via the data stream output unit to inject the data stream of the decompressed RAW video data back to the first domain controller via the data stream output unit.
10. A testing system, characterized in that: The test system comprises the data acquisition system described in any one of claims 1-5 and the data reinjection device described in any one of claims 6-9.
