Sensing detection device, sensing detection system and sensing detection method

Abstract

The invention discloses a sensing detection device, a sensing detection system and a sensing detection method. The sensing detection device comprises a laser radar module, an image sensing module and a serializer; the serializer is in communication connection with the laser radar module and the image sensing module, and the serializer is used for transmitting a data acquisition command to the laser radar module and the image sensing module; the image sensing module is used for outputting image data of a target object according to the data acquisition command, and the laser radar module is used for outputting depth data of the target object according to the data acquisition command; and the serializer is also used for packaging the image data and the depth data and sending the image data and the depth data through the communication end. According to the sensing detection device provided by the invention, fusion of image data and depth data is realized through the serializer, and meanwhile, the connection structure of the sensing detection device and the domain controller is simplified.

Description

Technical Field

[0001] The present invention relates to the technical field of sensing detection, and particularly to a sensing detection device, a sensing detection system and a sensing detection method. Background Art

[0002] With the development of vehicle intelligence and networking, the popularity of end-to-end autonomous driving is getting higher and higher, and the detection accuracy of vehicle sensors is getting higher and higher. In the parking scenario, multiple sensors work together to provide environmental information around the vehicle, which plays an important role in enabling the driver to complete the parking operation safely and accurately.

[0003] Currently, parking scenario perception is achieved through a camera + ultrasonic sensor. The camera outputs image information through a coaxial cable interface, and the image information is directly given to the domain controller for perception after being decoded by a deserializer. The ultrasonic sensor is connected to the domain controller through a distributed system interface. The camera and the ultrasonic sensor are connected to the domain controller through different communication interfaces, so that connection cables need to be separately set between the vehicle sensors and the domain controller, resulting in a complex connection structure between the vehicle sensors and the domain controller. Summary of the Invention

[0004] The present invention provides a sensing detection device, a sensing detection system and a sensing detection method to simplify the connection between vehicle sensors and the domain controller.

[0005] According to one aspect of the present invention, a sensing detection device is provided, including a lidar module, an image sensing module and a serializer;

[0006] The serializer is respectively communicatively connected to the lidar module and the image sensing module, and the serializer is used to transmit data acquisition commands to the lidar module and the image sensing module;

[0007] The image sensing module is used to output image data of a target object according to the data acquisition command, and the lidar module is used to output depth data of the target object according to the data acquisition command; the serializer is also used to pack the image data and the depth data and send them through a communication port.

[0008] Optionally, the data acquisition command includes an image data acquisition command; the serializer and the image sensing module are respectively communicatively connected to the image sensing module through a first transmission channel and a second transmission channel; the serializer is used to transmit the image data acquisition command to the image sensing module through the first transmission channel, and the image sensing module is used to transmit the acquired image data to the serializer through the second transmission channel according to the image data acquisition command; the interface types corresponding to the first transmission channel and the second transmission channel are different.

[0009] Optionally, the data acquisition command includes a depth data acquisition command; the sensing detection device further includes a data conversion module, the data conversion module is communicatively connected to the serializer through a third transmission channel, and the data conversion module is connected to the lidar module through a fourth transmission channel; the serializer is configured to transmit a depth data acquisition instruction to the data conversion module through the third transmission channel, and the data conversion module is configured to convert the communication protocol of the depth data acquisition instruction and then transmit it to the lidar module through the fourth transmission channel; the lidar module is configured to acquire depth data according to the depth data acquisition instruction after communication protocol conversion, and transmit the acquired depth data to the data conversion module through the fourth transmission channel, and the data conversion module is further configured to convert the communication protocol of the depth data and then transmit it to the serializer through the third transmission channel; the interface types corresponding to the third transmission channel and the fourth transmission channel are different.

[0010] Optionally, the sensing detection device includes a camera, and the image sensing module and the serializer are integrated in the camera.

[0011] Optionally, the camera includes a surround view camera, and / or, the lidar module includes a time-of-flight lidar.

[0012] Optionally, the sensing detection device further includes a first power supply module, a first isolation module, and a second isolation module, the first power supply module is connected to the lidar module, the image sensing module, and the serializer; the first isolation module is connected between the first power supply module and the communication terminal, the second isolation module is connected between the serializer and the communication terminal, the first isolation module is configured to isolate communication data signals, and the communication data signals include the data acquisition command, the image data, and the depth data, and the second isolation module is configured to isolate the power supply signal of the power supply.

[0013] Optionally, the first isolation module includes a first inductor, a second inductor, and a first resistor, a first end of the first inductor is connected to the communication terminal, a second end of the first inductor is connected to a first end of the second inductor and a first end of the first resistor, and a second end of the second inductor and a second end of the first resistor are connected to the first power supply module; the second isolation module includes a first capacitor, a first end of the first capacitor is connected to the communication terminal, and a second end of the first capacitor is connected to the serializer.

[0014] According to another aspect of the present invention, there is provided a sensing detection system, including at least one of the sensing detection devices, and further including a domain controller, the domain controller is communicatively connected to the sensing detection device;

[0015] The domain controller is configured to send the data acquisition command to the serializer according to a set data frame frequency, and receive the packed image data and depth data.

[0016] Optionally, the domain controller includes a system-on-chip and a deserialiser. The system-on-chip is communicatively connected to the deserialiser, and the deserialiser is communicatively connected to the serializer. The deserialiser is configured to deserialize the packed image data and depth data, and then transmit the depth data to the system-on-chip through a fifth transmission channel and transmit the image data to the system-on-chip through a sixth transmission channel. The system-on-chip transmits a control command to the deserialiser through a seventh transmission channel. The interface types corresponding to the fifth transmission channel, the sixth transmission channel, and the seventh transmission channel are different.

[0017] Optionally, the sensing detection system further includes a coaxial cable, which connects the deserialiser and the serializer and is configured to transmit communication data signals and power supply signals.

[0018] According to another aspect of the present invention, there is provided a sensing detection method, which is executed by the sensing detection device. The sensing detection method includes:

[0019] The serializer receives a data acquisition command sent by the domain controller;

[0020] The serializer transmits the data acquisition command to the lidar module and the image sensing module;

[0021] The image sensing module outputs image data of a target object according to the data acquisition command;

[0022] The lidar module outputs depth data of the target object according to the data acquisition command;

[0023] The serializer packs the image data and the depth data and sends them through a communication port.

[0024] Optionally, before the image sensing module outputs image data of the target object according to the data acquisition command, it further includes:

[0025] The serializer receives a frame synchronization signal sent by the domain controller;

[0026] The serializer copies the frame synchronization signal and outputs two paths of the frame synchronization signal, which are respectively transmitted to the lidar module and the image sensing module;

[0027] The image sensing module outputs image data of the target object according to the data acquisition command, including:

[0028] The image sensing module outputs the image data of the target object according to the data acquisition command and the frame synchronization signal;

[0029] The lidar module outputs the depth data of the target object according to the data acquisition command, including:

[0030] The lidar module outputs the depth data of the target object according to the data acquisition command and the frame synchronization signal;

[0031] The serializer packs the image data and the depth data and sends them through the communication port, including:

[0032] The serializer packs the image data and the depth data according to the frame synchronization signal and sends them through the communication port.

[0033] In the technical solution of the embodiment of the present invention, the sensing detection device includes a lidar module, an image sensing module and a serializer. The serializer is respectively communicatively connected to the lidar module and the image sensing module. The serializer can receive and transmit the data acquisition command to the lidar module and the image sensing module. The image sensing module outputs the image data of the target object according to the data acquisition command, and the lidar module outputs the depth data of the target object according to the data acquisition command. The serializer packs the image data and the depth data to realize the fusion of the image data and the depth data, and sends the fused data through the communication port. In this way, in the sensing detection device, the lidar module and the image sensing module can be connected to the communication port through the serializer, and connected to the domain controller through the communication port, and the fused data can be packed and sent through the communication port. Since the information output by different sensors is output through the same communication port, the connection structure between the sensing detection device and the domain controller can be simplified. Moreover, by fusing the image data and the depth data in the serializer, the problem that the domain controller has complex fusion of the image data and the depth data caused by directly transmitting the image data and the depth data to the domain controller through different communication interfaces can be reduced.

[0034] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Description of the Drawings

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0036] Figure 1 is a sensing detection structure diagram in the related art of the present invention;

[0037] Figure 2 is a schematic structural diagram of a sensing detection device provided by an embodiment of the present invention;

[0038] Figure 3 is a schematic structural diagram of another sensing detection device provided by an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of serializer data fusion provided by an embodiment of the present invention;

[0040] Figure 5 is a layout comparison diagram of the sensing detection device provided by an embodiment of the present invention;

[0041] Figure 6 is a schematic structural diagram of another sensing detection device provided by an embodiment of the present invention;

[0042] Figure 7 is a schematic structural diagram of a sensing detection system provided by an embodiment of the present invention;

[0043] Figure 8 is a schematic structural diagram of another sensing detection system provided by an embodiment of the present invention;

[0044] Figure 9 is a schematic diagram of deserialization by a deserializer provided by an embodiment of the present invention;

[0045] Figure 10 is a flowchart of a sensing detection method provided by an embodiment of the present invention;

[0046] Figure 11 is a flowchart of another sensing detection method provided by an embodiment of the present invention. Detailed implementation manners

[0047] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0048] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0049] The existing parking scene perception is realized by a camera + ultrasonic sensor. Figure 1 It is a sensing detection structure diagram in the related technology of the present invention, as Figure 1 shown. The connection interface of the camera is a coaxial cable interface. The image of the camera is directly given to the system on chip (SOC) of the domain controller for perception after being decoded by the deserialization device. The communication protocol between the camera and the deserialization device is the Mobile Industry Processor Interface (MIPI). The connection interface of the ultrasonic radar is the Distributed System Interface 3 (DSI3). The microcontroller unit (MCU) in the domain controller analyzes the three-dimensional depth data according to the ultrasonic data and then provides it to the SOC for sensor fusion. The communication protocols between the MCU, the DSI3 deserialization device and the SOC are the Serial Peripheral Interface (SPI). The camera and the domain controller are communicatively connected through a coaxial cable, and the ultrasonic radar and the domain controller are communicatively connected through DSI3. The camera and the ultrasonic sensor are connected to the domain controller through different communication interfaces, so that connection cables need to be separately set between the camera and the domain controller, and between the ultrasonic sensor and the domain controller, resulting in a complex connection structure between the vehicle sensors and the domain controller. At the same time, the camera image data and the ultrasonic data come from different communication interfaces, and the data fusion is complicated due to the data parsing by heterogeneous processors.

[0050] To solve the above problems, an embodiment of the present invention provides a sensing detection device, Figure 2 which is a schematic structural diagram of a sensing detection device provided by an embodiment of the present invention. As Figure 2As shown in the figure, the sensing and detection device 100 includes a lidar module 110, an image sensing module 120, and a serializer 130. The serializer 130 is communicatively connected to the lidar module 110 and the image sensing module 120 respectively. The serializer 130 is used to transmit data acquisition commands to the lidar module 110 and the image sensing module 120. The image sensing module 120 is used to output image data of the target object according to the data acquisition command, and the lidar module 110 is used to output depth data of the target object according to the data acquisition command. The serializer 130 is also used to pack the image data and the depth data and send them through communication port a.

[0051] In an embodiment of the present invention, the lidar module 110 is a module that senses the target object based on the propagation characteristics of light. Among them, the sensing of the target object includes distance measurement, three-dimensional imaging, etc. The image sensing module 120 is a module that acquires, processes, and outputs image information. For example, the image sensing module 120 can be integrated in a camera. The serializer 130 is a device that converts parallel data into serial data, and the serializer can perform parallel-to-serial conversion according to a preset rule.

[0052] In an embodiment of the present invention, the data acquisition command is issued by the domain controller. The serializer 130 receives the data acquisition instruction issued by the domain controller and transmits the data acquisition instruction to the lidar module 110 and the image sensing module 120. The image sensing module 120 outputs image data of the target object according to the data acquisition command, and the lidar module 110 outputs depth data of the target object according to the data acquisition command. The serializer 130 packs the image data and the depth data. Among them, the serializer 130 packs the image data and the depth data, including data selection, data splicing, encoding processing, etc. The serializer 130 realizes data fusion by packing the image data and the depth data. The serializer 130 sends the packed data to the domain controller through the communication port, so that the lidar module 110, the image sensing module 120, and the domain controller perform data transmission through the same communication interface.

[0053] In the technical solution of the embodiment of the present invention, the sensing detection device includes a lidar module, an image sensing module, and a serializer. The serializer is communicatively connected to the lidar module and the image sensing module respectively. The serializer can receive and transmit data acquisition commands to the lidar module and the image sensing module. The image sensing module outputs image data of the target object according to the data acquisition command, and the lidar module outputs depth data of the target object according to the data acquisition command. The serializer packs the image data and the depth data to realize the fusion of the image data and the depth data, and sends the fused data through the communication terminal. In this way, in the sensing detection device, the lidar module and the image sensing module can be connected to the communication terminal through the serializer, and connected to the domain controller through the communication terminal, and the fused data can be packed and sent through the communication terminal. Since the information output by different sensors is output through the same communication terminal, the connection structure between the sensing detection device and the domain controller can be simplified. Moreover, by fusing the image data and the depth data in the serializer, the problem that the domain controller has complex fusion of the image data and the depth data caused by directly transmitting the image data and the depth data to the domain controller through different communication interfaces can be reduced.

[0054] Figure 3 is a schematic structural diagram of another sensing detection device provided by the embodiment of the present invention, as Figure 3 shown, the data acquisition command includes an image data acquisition command; the serializer 130 and the image sensing module 120 are communicatively connected to the image sensing module 120 through a first transmission channel and a second transmission channel respectively; the serializer 130 is used to transmit the image data acquisition command to the image sensing module 120 through the first transmission channel, and the image sensing module 120 is used to transmit the acquired image data to the serializer 130 through the second transmission channel according to the image data acquisition command; the interface types corresponding to the first transmission channel and the second transmission channel are different.

[0055] In the embodiment of the present invention, the first transmission channel adopts the Inter-Integrated Circuit (IIC) protocol, the second transmission channel adopts the MIPI protocol, the interface corresponding to the first transmission channel is an IIC interface, and the interface corresponding to the second transmission channel is a MIPI interface. On the basis of the above embodiment, the serializer 130 receives the frame synchronization signal sent by the domain controller, copies the frame synchronization signal, and outputs two identical frame synchronization signals, which are respectively transmitted to the lidar module 110 and the image sensing module 120. The serializer 130 transmits the image data acquisition command and the frame synchronization signal to the image sensing module 120 through the first transmission channel, and the image sensing module 120 acquires image data according to the image data acquisition command and the frame synchronization signal, and transmits the image data to the serializer 130 through the second transmission channel.

[0056] Continue to refer to Figure 3, the data acquisition command includes a depth data acquisition command; the sensing detection device further includes a data conversion module 210. The data conversion module 210 is communicatively connected to the serializer 130 through a third transmission channel, and the data conversion module 210 is connected to the lidar module 110 through a fourth transmission channel; the serializer 130 is configured to transmit a depth data acquisition instruction to the data conversion module 210 through the third transmission channel. The data conversion module 210 is configured to perform communication protocol conversion on the depth data acquisition instruction and then transmit it to the lidar module 110 through the fourth transmission channel; the lidar module 110 is configured to perform depth data acquisition according to the depth data acquisition instruction after communication protocol conversion, and transmit the acquired depth data to the data conversion module 210 through the fourth transmission channel. The data conversion module 210 is further configured to perform communication protocol conversion on the depth data and then transmit it to the serializer 130 through the third transmission channel; the interface types corresponding to the third transmission channel and the fourth transmission channel are different.

[0057] In the embodiment of the present invention, the third transmission channel adopts the SPI protocol, the fourth transmission channel adopts the IIC protocol, the interface corresponding to the third transmission channel is the SPI interface, and the interface corresponding to the fourth transmission channel is the IIC interface. The data conversion module 210 is a module for performing communication protocol conversion. The process of communication protocol conversion includes data reception, protocol parsing, data processing and conversion, etc. For example, the data conversion module 210 adopts a microcontroller MCU. On the basis of the above embodiment, the serializer performs data fusion according to the received image data and depth data. Figure 4 is a schematic diagram of serializer data fusion provided by the embodiment of the present invention. As Figure 4 shown, the serializer 130 transmits a depth data acquisition command and a frame synchronization signal to the data conversion module 210 through the third transmission channel. The data conversion module 210 performs communication protocol conversion on the depth data acquisition instruction and the frame synchronization signal and then transmits them to the lidar module 110 through the fourth transmission channel. The lidar module 110 performs depth data acquisition according to the depth data acquisition instruction and the frame synchronization signal after communication protocol conversion, and transmits the acquired depth data to the data conversion module 210 through the fourth transmission channel. The data conversion module 210 performs communication protocol conversion on the depth data and then transmits it to the serializer 130 through the third transmission channel. The serializer packs the image data and the depth data to achieve the fusion of the image data and the depth data.

[0058] Continue to refer to Figure 3 , the sensing detection device includes a camera 220, and the image sensing module 120 and the serializer 130 are integrated in the camera. The camera 220 is a device for collecting image information around the vehicle. The camera 220 further includes an image processor, which can perform noise reduction, correction, etc. on the image data output by the image sensing module 120.

[0059] Specifically, the camera 220 includes a surround view camera, and / or the lidar module 110 includes a time-of-flight lidar. The surround view camera usually consists of multiple cameras, which are generally installed at the front, rear, left, and right positions of the vehicle. A TOF (Time-of-Flight) lidar is a device that determines the distance between the lidar and the target object based on measuring the time of flight of a laser pulse from emission to being reflected back by the target object. Figure 5 is a layout comparison diagram of the sensing detection device provided by an embodiment of the present invention. As Figure 5 shown, on the left is the layout of the sensing detection device in the prior art, which consists of 4 cameras + 12 ultrasonic sensors, and the physical interfaces include 4 coaxial cables + 4 DSI3 buses. On the right is the layout of the sensing detection device in the embodiment of the present invention, which only requires 4 coaxial cable physical interfaces, reducing 4 DSI3 buses and wiring harnesses, and simplifying the structure of the sensing detection device.

[0060] Continuing to refer to Figure 3 , the TOF lidar includes a laser emitter and a SPAD (Single Photon Avalanche Diode) array sensor. The laser emitter emits a laser signal. The SPAD array sensor is connected to the laser emitter and the data conversion module 210. The SPAD array sensor calculates the depth data of the target object based on the laser signal reflected by the target object. The laser emitter is connected to the data conversion module 210, and the laser emitter emits a laser signal according to the control signal output by the data conversion module 210. Since the SPDA array sensor has a high sensitivity to photons, the detection accuracy and efficiency can be improved. Based on the above embodiment, the SPAD array sensor collects depth data according to the depth data acquisition command and frame synchronization signal output by the data conversion module 210 and transmits the depth data to the data conversion module 210. The data conversion module 210 converts the communication protocol of the depth data and then transmits it to the serializer.

[0061] Figure 6 is a schematic structural diagram of another sensing detection device provided by an embodiment of the present invention. As Figure 6 shown, the sensing detection device further includes a first power module 410, a first isolation module 420, and a second isolation module 430. The first power module 410 is connected to the lidar module 110, the image sensing module 120, and the serializer 130. The first isolation module 420 is connected between the first power module 410 and the communication terminal a, and the second isolation module 420 is connected between the serializer 130 and the communication terminal a. The first isolation module 420 is used to isolate communication data signals, and the communication data signals include data acquisition commands, image data, and depth data. The second isolation module 430 is used to isolate the power supply signal.

[0062] In an embodiment of the present invention, the first power supply module 410 is a module that supplies power to the lidar module 110, the image sensing module 120, and the serializer 130. For example, the first power supply module 410 includes a power supply device such as a storage battery. The first isolation module 420 is a module that isolates communication data signals. The frequency of the communication data signals is generally high. The first isolation module 420 can use a device that blocks the passage of high-frequency signals to prevent high-frequency signals from interfering with the first power supply module 410. The second isolation module 430 is a module that isolates the power supply signal of the power supply. The power supply signal is a DC signal. The second isolation module can use a device that blocks the passage of DC signals to prevent DC signals from interfering with the transmission of communication data signals.

[0063] Specifically, the first isolation module 420 includes a first inductor L1, a second inductor L2, and a first resistor R1. The first end of the first inductor L1 is connected to the communication terminal a. The second end of the first inductor L1 is connected to the first end of the second inductor L2 and the first end of the first resistor R1. The second end of the second inductor L2 and the second end of the first resistor R1 are connected to the first power supply module 410. The second isolation module 430 includes a first capacitor C1. The first end of the first capacitor C1 is connected to the communication terminal a. The second end of the first capacitor C1 is connected to the serializer 130.

[0064] In an embodiment of the present invention, the first isolation module 420 uses the first inductor L1 and the second inductor L2 to isolate the communication data signals with high-frequency characteristics by utilizing the characteristics of inductors to pass low-frequency signals and block high-frequency signals. The second isolation module 430 uses the first capacitor C1 to isolate the power supply signal of the power supply by utilizing the characteristics of capacitors to block DC signals and pass AC signals. When the sensing detection device is working, the power supply signal and the communication data signal do not interfere with each other, improving the stability of signal transmission.

[0065] In an embodiment of the present invention, the serializer transmits a depth data acquisition command and a frame synchronization signal to the lidar module, and transmits an image data acquisition command and a frame synchronization signal to the image sensing module. The image sensing module outputs the image data of the target object according to the image data acquisition command and the frame synchronization signal. The lidar module outputs the depth data of the target object according to the depth data acquisition command and the frame synchronization signal, ensuring the synchronization of the image data and the depth data transmitted to the serializer. The serializer packs the image data and the depth data to achieve the fusion of the image data and the depth data, and sends the fused data to the domain controller through the communication terminal. By combining the communication interfaces of different sensors into one communication terminal, the structure of the sensing detection device is simplified, the integration difficulty is reduced, and the fusion efficiency is improved.

[0066] An embodiment of the present invention further provides a sensing detection system. Figure 7 It is a schematic structural diagram of a sensing detection system provided by an embodiment of the present invention, as Figure 7As shown, the sensing detection system 10 includes at least one sensing detection device 100 in any of the above embodiments, and further includes a domain controller 11, which is communicatively connected to the sensing detection device 100; the domain controller 11 is configured to send a data acquisition command to the serializer according to a set data frame frequency, and receive the packed image data and depth data.

[0067] In an embodiment of the present invention, the sensing detection system 10 is a system capable of sensing the external environment state and physical quantity changes and processing the sensed signals, which can ensure the safe driving of the vehicle and realize various intelligent functions. The domain controller 11 is a centralized electronic control unit, which can integrate multiple control functions related to a specific functional area in the vehicle, uniformly manage and coordinate the interaction and cooperation among numerous sensors, actuators, and related subsystems in this area, and achieve efficient function control and data processing, etc. On the basis of the above embodiment, the domain controller 11 sends an image data acquisition command and a depth data acquisition command to the serializer according to a set data frame frequency, and the domain controller 11 also sends a frame synchronization signal to the serializer. The domain control 11 receives the fused data of the image data and depth data output by the serializer. The domain controller 11 and the sensing detection device 100 perform data transmission through communication port a. Compared with the prior art, the domain controller and multiple sensors perform data transmission through multiple communication interfaces, reducing the complexity of data fusion and ensuring the synchronization of data transmission at the same time.

[0068] Figure 8 It is a schematic structural diagram of another sensing detection system provided by an embodiment of the present invention. As Figure 8 shown, the domain controller 11 includes a system on chip (SOC) and a deserialization unit 710. The system on chip (SOC) is communicatively connected to the deserialization unit 710, and the deserialization unit 710 is communicatively connected to the serializer 130. The deserialization unit is configured to perform deserialization processing on the packed image data and depth data, and then transmit the depth data to the system on chip (SOC) through the fifth transmission channel, and transmit the image data to the system on chip (SOC) through the sixth transmission channel. The system on chip (SOC) transmits a control command to the deserialization unit through the seventh transmission channel. The interface types corresponding to the fifth transmission channel, the sixth transmission channel, and the seventh transmission channel are different.

[0069] In an embodiment of the present invention, a system on chip (SOC) is a highly integrated integrated circuit chip that has functions such as executing instructions, data processing, data storage, and image processing. A deserialiser 710 is a device that converts received serial data into parallel data. By deserialising the serial data output by a serializer, the deserialiser 710 enables the SOC to conveniently process the data. The fifth transmission channel uses the SPI protocol, the sixth transmission channel uses the MIPI protocol, and the seventh transmission channel uses the IIC protocol. The interface corresponding to the fifth transmission channel is an SPI interface, the interface corresponding to the sixth transmission channel is a MIPI interface, and the interface corresponding to the seventh transmission channel is an IIC interface. The domain controller 11 further includes a second power module 720, a third inductor L3, a fourth inductor L4, and a second resistor R2. The third inductor L3 and the fourth inductor L4 provide isolation protection for the second power module 720.

[0070] Continuing to refer to Figure 8 , the sensing detection system further includes a coaxial cable 12. The coaxial cable 12 connects the deserialiser 710 and the serializer 130 and is used to transmit communication data signals and power supply signals. A coaxial cable is a signal transmission wire with a double concentric conductor structure that can transmit communication data signals and power supply signals simultaneously.

[0071] Exemplarily, the sensing detection system includes four sensing detection devices 100 (one sensing detection device is shown in the figure). Correspondingly, four coaxial cables need to be set to connect to the domain controller. The deserialiser is respectively connected to the four coaxial cable interfaces b through four capacitors C2, and the deserialiser deserialises the fusion data output by the four sensing detection devices 100. For example, the depth data is an SPI signal and the image data is a MIPI signal. The deserialiser extracts the four SPI signals from the fusion data and then transmits them to the SOC through one SPI interface. The deserialiser extracts the four MIPI signals from the fusion data and then transmits them to the SOC through one MIPI interface. The transmission rate of the depth data of one TOF lidar is 72 * 24 * 64 bit * 25 frames = 2.8 Mbit / S, and the transmission rate of four TOF lidars is 2.8 * 4 = 11.2 Mbit / S. The SPI channel bandwidth of the deserialiser is 25 Mbit / S, which can meet the bandwidth requirements of the four-channel fusion sensing depth data. Figure 9 is a schematic diagram of the deserialiser deserialisation provided by an embodiment of the present invention. The coaxial interfaces 1-4 of the deserialiser receive the fusion data output by the four sensing detection devices. After deserialising the fusion data, the depth data is transmitted to the SOC through one SPI interface, and the image data is transmitted to the SOC through one MIPI interface.

[0072] An embodiment of the present invention provides a sensing detection method. Figure 10The following is a flowchart of a sensing detection method provided by an embodiment of the present invention. As shown in Figure 10 the figure, the sensing detection method includes:

[0073] S10. The serializer receives a data acquisition command sent by the domain controller.

[0074] S20. The serializer transmits the data acquisition command to the lidar module and the image sensing module.

[0075] S30. The image sensing module outputs image data of the target object according to the data acquisition command.

[0076] S40. The lidar module outputs depth data of the target object according to the data acquisition command.

[0077] S50. The serializer packs the image data and the depth data and sends them through the communication port.

[0078] Based on the above embodiment, the embodiment of the present invention provides another sensing detection method. Figure 11 The following is a flowchart of another sensing detection method provided by an embodiment of the present invention. As shown in Figure 11 the figure, before the image sensing module outputs image data of the target object according to the data acquisition command, it further includes:

[0079] S11. The serializer receives a frame synchronization signal sent by the domain controller.

[0080] S21. The serializer copies the frame synchronization signal and outputs two paths of frame synchronization signals, which are respectively transmitted to the lidar module and the image sensing module.

[0081] S31. The image sensing module outputs image data of the target object according to the data acquisition command and the frame synchronization signal.

[0082] S41. The lidar module outputs depth data of the target object according to the data acquisition command and the frame synchronization signal.

[0083] S51. The serializer packs the image data and the depth data according to the frame synchronization signal and sends them through the communication port.

[0084] In an embodiment of the present invention, the serializer transmits data acquisition commands and frame synchronization signals to the lidar module and the image sensing module. The image sensing module outputs image data of the target object according to the data acquisition commands and the frame synchronization signals, and the lidar module outputs depth data of the target object according to the data acquisition commands and the frame synchronization signals. The serializer packs the image data and the depth data according to the frame synchronization signals and sends them through the communication port. By sending the frame synchronization signals to the lidar module and the image sensing module, the serializer can ensure that the depth data output by the lidar module and the image data output by the image sensing module are data of the same frame. The serializer fuses the two data of the same frame to achieve frame synchronization of the fused data and further reduces the fusion complexity.

[0085] The sensing detection method provided by the embodiment of the present invention is executed by the sensing detection device in any of the above embodiments. The serializer transmits depth data acquisition commands and frame synchronization signals to the lidar module, and transmits image data acquisition commands and frame synchronization signals to the image sensing module. The image sensing module outputs image data of the target object according to the image data acquisition commands and the frame synchronization signals, and the lidar module outputs depth data of the target object according to the depth data acquisition commands and the frame synchronization signals, ensuring the synchronization of the image data and the depth data transmitted to the serializer. The serializer packs the image data and the depth data to achieve the fusion of the image data and the depth data, and sends the fused data to the domain controller through the communication port. By combining the communication interfaces of different sensors into one communication port, the connection structure between the sensing detection device and the domain controller can be simplified. By fusing the image data and the depth data in the serializer, the problem that the image data and the depth data are directly transmitted to the domain controller through different communication interfaces, resulting in complex fusion of the image data and the depth data by the domain controller, can be reduced.

[0086] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solution of the present invention can be achieved. No limitation is imposed herein.

[0087] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A sensing detection device, characterized in that, It includes a lidar module, an image sensing module, and a serializer; The serializer is respectively communicatively connected to the lidar module and the image sensing module, and the serializer is used to transmit data acquisition commands to the lidar module and the image sensing module; The image sensing module is used to output image data of a target object according to the data acquisition command, and the lidar module is used to output depth data of the target object according to the data acquisition command; the serializer is also used to package the image data and the depth data and send them through a communication port.

2. The sensing and detecting device according to claim 1, wherein The data acquisition command includes an image data acquisition command; the serializer and the image sensing module are respectively communicatively connected to the image sensing module through a first transmission channel and a second transmission channel; The serializer is used to transmit the image data acquisition command to the image sensing module through the first transmission channel, and the image sensing module is used to transmit the acquired image data to the serializer through the second transmission channel according to the image data acquisition command; The interface types corresponding to the first transmission channel and the second transmission channel are different.

3. The sensing detection device according to claim 2, wherein The data acquisition command includes a depth data acquisition command; The sensing detection device further includes a data conversion module, the data conversion module is communicatively connected to the serializer through a third transmission channel, and the data conversion module is connected to the lidar module through a fourth transmission channel; The serializer is used to transmit a depth data acquisition instruction to the data conversion module through the third transmission channel, and the data conversion module is used to perform communication protocol conversion on the depth data acquisition instruction and then transmit it to the lidar module through the fourth transmission channel; The lidar module is used to perform depth data acquisition according to the depth data acquisition instruction after communication protocol conversion, and transmit the acquired depth data to the data conversion module through the fourth transmission channel, and the data conversion module is also used to perform communication protocol conversion on the depth data and then transmit it to the serializer through the third transmission channel; The interface types corresponding to the third transmission channel and the fourth transmission channel are different.

4. The sensing and detecting device according to claim 1, characterized in that, The sensing detection device includes a camera, and the image sensing module and the serializer are integrated in the camera.

5. The sensing and detecting device according to claim 4, characterized in that, The camera includes a surround view camera, and / or, the lidar module includes a time-of-flight lidar.

6. The sensing and detecting device according to claim 1, wherein, The sensing detection device further includes a first power supply module, a first isolation module, and a second isolation module, and the first power supply module is connected to the lidar module, the image sensing module, and the serializer; The first isolation module is connected between the first power supply module and the communication port, the second isolation module is connected between the serializer and the communication port, the first isolation module is used to isolate communication data signals, and the communication data signals include the data acquisition command, the image data, and the depth data, and the second isolation module is used to isolate the power supply signal of the power supply.

7. The sensing and detecting device according to claim 6, wherein The first isolation module includes a first inductor, a second inductor and a first resistor. The first end of the first inductor is connected to the communication terminal. The second end of the first inductor is connected to the first end of the second inductor and the first end of the first resistor. The second end of the second inductor and the second end of the first resistor are connected to the first power supply module. The second isolation module includes a first capacitor. The first end of the first capacitor is connected to the communication terminal. The second end of the first capacitor is connected to the serializer.

8. A sensing detection system, characterized in that, It includes at least one sensing detection device according to any one of claims 1-7, and further includes a domain controller, and the domain controller is communicatively connected to the sensing detection device; The domain controller is configured to send the data acquisition command to the serializer according to a set data frame frequency, and receive the packed image data and depth data.

9. The sensing and detecting system according to claim 8, wherein, The domain controller includes a system-on-chip and a deserialiser. The system-on-chip is communicatively connected to the deserialiser. The deserialiser is communicatively connected to the serializer. The deserialiser is configured to deserialize the packed image data and depth data, and then transmit the depth data to the system-on-chip through a fifth transmission channel, and transmit the image data to the system-on-chip through a sixth transmission channel. The system-on-chip transmits a control command to the deserialiser through a seventh transmission channel. The interface types corresponding to the fifth transmission channel, the sixth transmission channel and the seventh transmission channel are different.

10. The sensing and detecting system according to claim 9, characterized in that, The sensing detection system further includes a coaxial cable, and the coaxial cable connects the deserialiser and the serializer, and is configured to transmit communication data signals and power supply signals.

11. A sensing detection method, characterized in that, Performed by the sensing detection device according to any one of claims 1-7, the sensing detection method includes: The serializer receives the data acquisition command sent by the domain controller; The serializer transmits the data acquisition command to the lidar module and the image sensing module; The image sensing module outputs image data of the target object according to the data acquisition command; The lidar module outputs depth data of the target object according to the data acquisition command; The serializer packs the image data and the depth data and sends them through the communication terminal.

12. The sensing and detecting method according to claim 11, wherein Before the image sensing module outputs image data of the target object according to the data acquisition command, it further includes: The serializer receives the frame synchronization signal sent by the domain controller; The serializer copies the frame synchronization signal and outputs two paths of the frame synchronization signal, which are respectively transmitted to the lidar module and the image sensing module; The image sensing module outputs image data of the target object according to the data acquisition command, including: The image sensing module outputs image data of the target object according to the data acquisition command and the frame synchronization signal; The lidar module outputs depth data of the target object according to the data acquisition command, including: The lidar module outputs depth data of the target object according to the data acquisition command and the frame synchronization signal; The serializer packs the image data and the depth data and sends them through the communication terminal, including: The serializer packs the image data and the depth data according to the frame synchronization signal and sends them through the communication port.

Images