Vehicle-mounted camera synchronization system, method and device and electronic equipment

Abstract

The invention discloses a vehicle-mounted camera synchronization system, method and device and electronic equipment, and relates to the technical field of data communication, and the system comprises the steps that an ADAS camera is connected with an ADAS domain controller, and the ADAS camera is used for receiving a frame synchronization signal and transmitting ADAS image data; the intelligent cabin domain controller is connected with the ADAS domain controller, and the intelligent cabin domain controller is used for receiving a frame synchronization signal; the look-around camera is connected with the intelligent cabin domain controller, and the look-around camera is used for receiving the frame synchronization signal and transmitting look-around image data; the ADAS domain controller is used for synchronously sending frame synchronization signals to the plurality of ADAS cameras and the plurality of all-round cameras, and receiving ADAS image data and all-round image data. According to the invention, the time for the image data captured by the ADAS camera and the surround view camera to reach the sensing operation unit is consistent, and the accuracy of the sensing result of the BEV sensing algorithm is improved.

Description

Technical Field

[0001] The present application relates to the field of data communication technology, and in particular to a vehicle-mounted camera synchronization system, method, device and electronic equipment. Background Art

[0002] With the rapid development of advanced intelligent driving technology, in-vehicle cameras have become indispensable sensors. Currently, a vehicle with intelligent driving is generally equipped with up to 11 cameras, including front-view, side-view, rear-view, and surround-view cameras, which are used for different functions. For example, the seven ADAS cameras (i.e., two front-view cameras (telephoto and wide-angle), four side-view cameras, and one rear-view camera) are mainly responsible for Advanced Driver Assistance Systems (ADAS) driving and Autonomous Emergency Braking (AEB) functions, while the four surround-view cameras are mainly responsible for automatic parking, driving records, and assisted parking functions.

[0003] ADAS cameras and surround-view cameras may not be designed to directly synchronize image frames, primarily because they typically consist of multiple independent cameras, each with its own image sensor, processor, and output interface. These cameras may be installed in different locations within the vehicle. Due to these different physical locations, when these cameras are activated, the image data they capture may arrive at the perception processing unit at different times.

[0004] Bird's Eye View (BEV) perception algorithms require fusing image information from multiple cameras into a unified 3D scene representation. Temporal offsets between image data can cause errors in image fusion and 3D reconstruction. These errors can affect the accuracy of the BEV perception algorithm's perception results, impacting subsequent planning and control. Summary of the Invention

[0005] In view of this, the present application provides a vehicle-mounted camera synchronization system, method, device and electronic equipment, the main purpose of which is to solve the problem that the image data captured by the current ADAS camera and the surround-view camera arrive at the perception operation unit at an inconsistent time, which will affect the accuracy of the perception results of the BEV perception algorithm and thus affect the subsequent planning and control.

[0006] According to a first aspect of the present application, there is provided a vehicle-mounted camera synchronization system, comprising: a plurality of ADAS cameras, a plurality of surround-view cameras, an ADAS domain controller, and an intelligent cockpit domain controller;

[0007] The multiple ADAS cameras are connected to the ADAS domain controller via a coaxial cable harness, and the multiple ADAS cameras are used to receive a frame synchronization signal sent by the ADAS domain controller. The multiple ADAS cameras simultaneously start exposure based on the frame synchronization signal and transmit the exposed ADAS image data to the ADAS domain controller;

[0008] The smart cockpit domain controller is connected to the ADAS domain controller via a coaxial cable. The smart cockpit domain controller is configured to receive the frame synchronization signal sent by the ADAS domain controller and synchronously send the frame synchronization signal to the multiple surround view cameras based on the frame synchronization signal.

[0009] The multiple surround-view cameras are connected to the smart cockpit domain controller via a coaxial cable, and the multiple surround-view cameras are used to receive the frame synchronization signal. The multiple surround-view cameras simultaneously start exposure based on the frame synchronization signal and transmit the exposed surround-view image data to the ADAS domain controller;

[0010] The ADAS domain controller is used to synchronously send the frame synchronization signal to the multiple ADAS cameras and the multiple surround view cameras, and synchronously receive the ADAS image data and the surround view image data.

[0011] According to a second aspect of the present disclosure, a vehicle-mounted camera synchronization method is provided. The method is applied to the above-mentioned vehicle-mounted camera synchronization system, comprising:

[0012] Synchronously sending a frame synchronization signal to the plurality of ADAS cameras and the plurality of surround view cameras, wherein the frame synchronization signal is used to ensure that the plurality of ADAS cameras and the plurality of surround view cameras start capturing image data at the same time point;

[0013] When the multiple ADAS cameras and the multiple surround view cameras receive the frame synchronization signal, the ADAS image data exposed by the multiple ADAS cameras and the surround view image data exposed by the multiple surround view cameras are synchronously received.

[0014] According to a third aspect of the present application, a vehicle-mounted camera synchronization device is provided, which is applied to the above-mentioned vehicle-mounted camera synchronization system, including:

[0015] a sending module, configured to synchronously send a frame synchronization signal to the plurality of ADAS cameras and the plurality of surround-view cameras, wherein the frame synchronization signal is configured to ensure that the plurality of ADAS cameras and the plurality of surround-view cameras start capturing image data at the same time point;

[0016] A receiving module is used to synchronously receive the ADAS image data exposed by the multiple ADAS cameras and the surround view image data exposed by the multiple surround view cameras when the multiple ADAS cameras and the multiple surround view cameras receive the frame synchronization signal.

[0017] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the vehicle-mounted camera synchronization method described in the second aspect is implemented.

[0018] According to the fifth aspect of the present application, an electronic device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the vehicle-mounted camera synchronization method described in the second aspect when executing the computer program.

[0019] According to a sixth aspect of the present disclosure, a vehicle is provided, comprising: the system as described in the first aspect, the device as described in the third aspect, the computer-readable storage medium as described in the fourth aspect, or the electronic device as described in the fifth aspect.

[0020] By means of the above-mentioned technical solution, the present application provides a vehicle-mounted camera synchronization system, method, device and electronic device, which can be connected to an ADAS domain controller through a coaxial cable through multiple ADAS cameras, and the multiple ADAS cameras are used to receive frame synchronization signals sent by the ADAS domain controller. The multiple ADAS cameras start exposure at the same time based on the frame synchronization signal, and transmit the exposed ADAS image data to the ADAS domain controller; the smart cockpit domain controller is connected to the ADAS domain controller through a coaxial cable, and the smart cockpit domain controller is used to receive the frame synchronization signal sent by the ADAS domain controller, and based on the frame synchronization signal, synchronously send the frame synchronization signal to multiple surround-view cameras; multiple surround-view cameras are connected to the smart cockpit domain controller through a coaxial cable, and the multiple surround-view cameras are used to receive frame synchronization signals. The multiple surround-view cameras start exposure at the same time based on the frame synchronization signal, and transmit the exposed surround-view image data to the ADAS domain controller; the ADAS domain controller is used to synchronously send frame synchronization signals to multiple ADAS cameras and multiple surround-view cameras, and synchronously receive ADAS image data and surround-view image data. For the disclosed embodiment, the ADAS domain controller first generates a frame synchronization signal and transmits it to each ADAS camera and surround-view camera through a coaxial cable. After receiving the synchronization signal, each ADAS camera and surround-view camera will start exposure according to the time point indicated by the signal. After the exposure is completed, each camera transmits the image data back to the ADAS domain controller, thereby achieving the consistency of time when the image data captured by the ADAS camera and the surround-view camera arrives at the perception operation unit, thereby improving the accuracy of the perception results of the BEV perception algorithm.

[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, which can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A schematic structural diagram of a vehicle-mounted camera synchronization system provided by an embodiment of the present disclosure;

[0025] Figure 2 A schematic structural diagram of an ADAS control system provided in an embodiment of the present disclosure;

[0026] Figure 3 A schematic structural diagram of an intelligent cockpit control system provided by an embodiment of the present disclosure;

[0027] Figure 4 An oscilloscope screenshot provided for an embodiment of the present disclosure;

[0028] Figure 5 Another oscilloscope screenshot provided for an embodiment of the present disclosure;

[0029] Figure 6 A schematic diagram of a process for synchronizing a vehicle-mounted camera according to an embodiment of the present disclosure;

[0030] Figure 7 A schematic structural diagram of a vehicle-mounted camera synchronization device provided by an embodiment of the present disclosure;

[0031] Figure 1 middle:

[0032] 1- Multiple ADAS cameras;

[0033] 2- Multiple surround view cameras;

[0034] 3-ADAS domain controller, 31-first deserializer;

[0035] 4-smart cockpit domain controller, 41-third serializer, 42-second deserializer;

[0036] Figure 2 middle:

[0037] 11-first optical sensor, 12-first serializer, 13-first optical lens;

[0038] 32-GPIO pins;

[0039] 5- First Lpoc inductor;

[0040] Figure 3 middle:

[0041] 21-second optical sensor, 22-second serializer, 23-second optical lens;

[0042] 6- Second Lpoc inductor. DETAILED DESCRIPTION

[0043] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other unless there is a conflict.

[0044] To address the current issue of inconsistent arrival times of image data captured by ADAS cameras and surround-view cameras at the perception processing unit, which affects the accuracy of the BEV perception algorithm and, in turn, impacts subsequent planning and control, this embodiment provides a vehicle-mounted camera synchronization system, method, device, and electronic device.

[0045] like Figure 1 As shown, an embodiment of the present disclosure provides a vehicle-mounted camera synchronization system, which includes: multiple ADAS cameras 1, multiple surround-view cameras 2, an ADAS domain controller 3, and an intelligent cockpit domain controller 4;

[0046] Among them, the Advanced Driver-Assistance Systems (ADAS) camera 1 is a sensor that can be installed on a vehicle to provide visual information about the vehicle's surroundings to assist the driver in performing driving tasks. The ADAS camera can monitor the vehicle's surroundings in different driving environments, provide warnings, and assist in performing functions such as adaptive cruise control (ACC), lane keeping assist (LKA), forward collision warning (FCW), and blind spot monitoring (BSM).

[0047] Surround-view cameras, also known as 360-degree cameras, provide a comprehensive 360-degree visual overview of the vehicle's surroundings. Surround-view camera systems typically consist of multiple cameras installed at different locations on the vehicle. These cameras work together, using image processing technology to fuse images from multiple perspectives to create a complete view of the vehicle's surroundings. The driver can view this panoramic view on the display screen and perform operations such as parking, reversing, and automated parking.

[0048] The ADAS domain controller 3 (Advanced Driver-Assistance Systems Controller) is part of the advanced driver assistance system and is responsible for receiving and processing data from various vehicle sensors. These sensors can include cameras, radar, LiDAR, and ultrasonic sensors, and are used to monitor the vehicle's surroundings, such as pedestrians and vehicles, lane markings, traffic signs, and the vehicle's driving status.

[0049] The Intelligent Cockpit Domain Controller 4 can be used to manage and coordinate various electronic systems and functions within the vehicle.

[0050] In a specific application scenario, multiple ADAS cameras 1 are connected to the ADAS domain controller 3 via a coaxial cable. The multiple ADAS cameras 1 are used to receive the frame synchronization signal sent by the ADAS domain controller 3. The multiple ADAS cameras 1 start exposure at the same time based on the frame synchronization signal, and transmit the exposed ADAS image data to the ADAS domain controller 3; the smart cockpit domain controller 4 is connected to the ADAS domain controller 3 via a coaxial cable. The smart cockpit domain controller 4 is used to receive the frame synchronization signal sent by the ADAS domain controller 3, and based on the frame synchronization signal, synchronously send the frame synchronization signal to multiple surround-view cameras 2; multiple surround-view cameras 2 are connected to the smart cockpit domain controller 4 via a coaxial cable. The multiple surround-view cameras 2 are used to receive the frame synchronization signal. The multiple surround-view cameras 2 start exposure at the same time based on the frame synchronization signal, and transmit the exposed surround-view image data to the ADAS domain controller 3; the ADAS domain controller 3 is used to synchronously send frame synchronization signals to multiple ADAS cameras 1 and multiple surround-view cameras 2, and synchronously receive ADAS image data and surround-view image data.

[0051] The coaxial cable can be used to transmit high-frequency signals, ensuring signal stability and interference resistance during long-distance transmission. The frame synchronization signal is used to ensure that multiple ADAS cameras 1 and multiple surround-view cameras 2 begin capturing image data at the same time.

[0052] In the disclosed embodiment, the ADAS domain controller 3 is responsible for sending frame synchronization signals to all ADAS cameras 1 and surround-view cameras 2. After receiving the synchronization signal, all ADAS cameras 1 and surround-view cameras 2 simultaneously begin capturing images and transmit the captured image data to the ADAS domain controller 3. The captured image data is then fused and processed using the BEV perception algorithm to obtain a more accurate perception of the vehicle's surrounding environment. Based on this information, the system can then make more accurate decisions, such as lane keeping and adaptive cruise control.

[0053] Bird's Eye View (BEV) perception algorithms are computer vision technologies used to process information about a vehicle's surroundings. They generate a bird's-eye view from the vehicle's perspective, displaying information such as roads, obstacles, and other vehicles. BEV perception algorithms are crucial in autonomous driving and advanced driver assistance systems (ADAS), helping them better understand the vehicle's environment.

[0054] In specific application scenarios, such as Figure 2 As shown, the multiple ADAS cameras 1 may include a first optical sensor 11 and a first serializer 12 , and the ADAS domain controller 3 may include a first deserializer 31 and a GPIO pin 32 ;

[0055] Among them, the first optical sensor 11 can be part of the ADAS camera 1, responsible for capturing light and converting it into electrical signals to form ADAS image data; the first serializer 12 can be a hardware device used to receive the image data captured by the first optical sensor 11 and convert it into a signal format suitable for long-distance transmission; the first deserializer 31 is used to receive serial data and convert it into a digital signal that can be processed by a computer; the general purpose input / output (GPIO) pin for input or output, the GPIO pin 32 is used to transmit frame synchronization signals.

[0056] In a specific application scenario, the first optical sensor 11 is connected to the first serializer 12 through the GPIO pin 32. The first optical sensor 11 is used to receive the frame synchronization signal sent by the ADAS domain controller 3 through the first deserializer 31 and the first serializer 12, and perform exposure based on the frame synchronization signal, and convert the exposed ADAS image data into a first MIPI digital signal;

[0057] The first serializer 12 is connected to the first deserializer 31 via a coaxial cable. The first serializer 12 is used to receive a first MIPI digital signal and convert the first MIPI digital signal into a first LVDS signal for long-distance transmission, as well as transmit a frame synchronization signal.

[0058] The first deserializer 31 is connected to the GPIO pin 32 . The first deserializer 31 is used to transmit a frame synchronization signal using the GPIO pin 32 and receive a first LVDS signal through a coaxial cable. The GPIO pin 32 is used to directly transmit the frame synchronization signal.

[0059] Among them, MIPI (Mobile Industry Processor Interface) can be a standardized communication interface for mobile devices and other portable devices. The first MIPI digital signal can be a digital video signal format of ADAS image data, used to transmit the ADAS image data captured by the ADAS camera 1; LVDS (Low Voltage Differential Signaling) is a differential signal transmission technology used for high-speed data transmission. The first LVDS signal can be used to transmit the ADAS image data captured by the ADAS camera 1, especially in long-distance transmission, which can provide better signal quality and anti-interference capability.

[0060] In the disclosed embodiment, the vehicle camera synchronization system generally includes multiple ADAS cameras 1 to provide a comprehensive view of the vehicle's surrounding environment. These ADAS cameras 1 can be front, rear, left, right, or interior cameras of the vehicle, without specific limitation.

[0061] The first optical sensor 11 receives the frame synchronization signal from the ADAS domain controller 3, then exposes and converts the ADAS image data into a first MIPI digital signal. The first serializer 12 receives these first MIPI digital signals and converts them into a first LVDS signal for long-distance transmission, while also transmitting the frame synchronization signal. The first deserializer 31 receives the first LVDS signal and converts it back into a digital signal for processing by the ADAS domain controller 3.

[0062] In specific application scenarios, such as Figure 2 As shown, the system also includes: a first Lpoc inductor 5; the first Lpoc inductor 5 is used to transmit the first DC power signal on the ADAS domain controller 3 side, and superimpose the first DC power signal and the first LVDS signal on the coaxial cable harness at the same time, and extract the first DC power signal in the coaxial cable harness to power multiple ADAS cameras 1.

[0063] The first DC power signal can be a signal specifically used to provide energy to the ADAS domain controller 3, ensuring that the ADAS domain controller 3 can function properly. The first LPOC inductor 5 can be used to transmit the first DC power signal required by the ADAS domain controller 3. In modern vehicles, to improve space efficiency and simplify wiring, a shared cable (such as a coaxial cable) is often used to transmit multiple signals, including power and data signals. Therefore, the first LPOC inductor 5 can superimpose the first DC power signal and the first LVDS signal on the coaxial cable, and identify and separate the first DC power signal for power supply from the coaxial cable, thereby ensuring that the DC power required by the ADAS domain controller 3 can be effectively transmitted through the coaxial cable and provide power to the multiple ADAS cameras 1.

[0064] For the embodiments of the present disclosure, Figure 2As shown, the camera module of each ADAS camera 1 is composed of a first optical lens 13 (lens), a first optical sensor 11 (CMOS sensor) and a first serializer 12 (Serilizer). On the control link, the system chip (SoC) can control the first serializer 12 and the first optical sensor 11 through the I2C interface. After configuring the relevant registers, the first optical sensor 11 can realize image output. Among them, I2C (Inter-Integrated Circuit) can be a common serial communication protocol for connecting low-speed peripheral devices. The first optical sensor 11 converts the ADAS image data into a digital signal of the MIPI interface (i.e., the first MIPI digital signal), and the first serializer 12 can convert the digital signal of the MIPI interface (i.e., the first MIPI digital signal) into a first LVDS signal that can be transmitted over a long distance, and then transmit it through a low-cost coaxial cable to give the ADAS image data to the ADAS controller. In terms of power supply, PoC (Power over Cable) technology is used to simultaneously superimpose the first DC power signal and the first LVDS signal on the coaxial cable, without the need for additional cables. By utilizing the low-pass characteristics of LPOC, the first deserializer 31 side loads the DC power supply onto the coaxial cable harness through the first LPOC inductor 5, and the first serializer 12 side then extracts the DC power supply through LPOC to power the camera module of the ADAS camera 1.

[0065] At the same time, the first serializer 12-first deserializer 31 (SerDes) channel can realize the reverse GPIO transparent transmission function. The external pin of the first deserializer 31 can be connected to the input IO and transmitted to the first serializer 12 side via the coaxial cable. The first serializer 12 is connected to the first optical sensor 11 (CMOS sensor) through GPIO. Using the GPIO transparent transmission function, the system chip (SoC) can send a frame synchronization signal to the first optical sensor 11 (CMOS sensor) through the first deserializer 31 and the first serializer 12. After receiving the frame synchronization signal, multiple first optical sensors 11 (CMOS sensors) start exposure at the same time and transmit the image to the system chip (SoC), thereby realizing the frame synchronization function of multiple ADAS cameras 1.

[0066] In specific application scenarios, such as Figure 3 As shown, the multiple surround view cameras 2 include a second optical sensor 21 and a second serializer 22 , and the smart cockpit domain controller 4 includes a third serializer 41 and a second deserializer 42 ;

[0067] The second optical sensor 21 is connected to the second serializer 22 via the GPIO pin 32. The second optical sensor 21 is used to receive the frame synchronization signal sent by the ADAS domain controller 3 through the first deserializer 31 and the third serializer 41, perform exposure based on the frame synchronization signal, and convert the exposed surround view image data into a second MIPI digital signal;

[0068] The second serializer 22 is connected to the second deserializer 42 via a coaxial cable. The second serializer 22 is used to receive the second MIPI digital signal and transmit the frame synchronization signal.

[0069] The second deserializer 42 and the third serializer 41 are connected via the GPIO pin 32 , and the second deserializer 42 is used to transmit the frame synchronization signal via the GPIO pin 32 and receive the second MIPI digital signal via the coaxial cable.

[0070] The third serializer 41 is connected to the first deserializer 31 via a coaxial cable. The third serializer 41 is used to receive the second MIPI digital signal, convert the second MIPI digital signal into a second LVDS signal for long-distance transmission, and transmit a frame synchronization signal.

[0071] The first deserializer 31 is configured to transmit a frame synchronization signal using the GPIO pin 32 and receive a second LVDS signal through a coaxial cable.

[0072] Among them, the second optical sensor 21 can be part of the surround-view camera 2, used to receive the frame synchronization signal from the ADAS domain controller 3, and expose it accordingly, and convert the captured surround-view image data into a second MIPI digital signal; the second serializer 22 can be used to receive the second MIPI digital signal converted by the optical sensor, and can be used to transmit the data to other components; the second deserializer 42 can be used to receive the second MIPI digital signal transmitted by the second serializer 22, and may be used to further process or transmit these signals; the second MIPI digital signal can be a digital video signal format of the surround-view image data, used to transmit the surround-view image data captured by the surround-view camera 2; the second LVDS signal can be used to transmit the surround-view image data captured by the surround-view camera 2, especially when transmitted over long distances, it can provide better signal quality and anti-interference capability.

[0073] In the embodiment of the present disclosure, the vehicle-mounted camera synchronization system generally includes a plurality of surround-view cameras 2 , which may be front, rear, left, right, or interior cameras of the vehicle, without specific limitation.

[0074] The second optical sensor 21 can be used to receive the frame synchronization signal from the ADAS domain controller 3, and then expose and convert the surround view image data into a second MIPI digital signal. The second serializer 22 can receive these second MIPI digital signals, and the second deserializer 42 can be used to receive the second MIPI digital signal transmitted by the second serializer 22 while transmitting the frame synchronization signal. The third serializer 41 is used to receive the second MIPI digital signal and convert the second MIPI digital signal into a second LVDS signal for long-distance transmission, as well as transmit the frame synchronization signal. The first deserializer 31 receives the second LVDS signal and converts it back into a digital signal for processing by the ADAS domain controller 3.

[0075] In specific application scenarios, such as Figure 3 As shown, the system further includes: a second Lpoc inductor 6;

[0076] The second LPOC inductor 6 is used to transmit the second DC power signal on the intelligent cockpit domain controller 4 side, and superimpose the second DC power signal and the second LVDS signal on the coaxial cable harness at the same time, and extract the second DC power signal in the coaxial cable harness to power multiple surround-view cameras 2.

[0077] The second DC power signal can be a signal specifically used to provide energy to the intelligent cockpit domain controller 4, ensuring the normal operation of the intelligent cockpit domain controller 4. The second LPOC inductor 6 can be used to transmit the second DC power signal required by the intelligent cockpit domain controller 4. In modern vehicles, to improve space efficiency and simplify wiring, a shared cable (such as a coaxial cable) is often used to transmit multiple signals, including power and data signals. Therefore, the second LPOC inductor 6 can superimpose the second DC power signal and the second LVDS signal on the coaxial cable, and identify and separate the second DC power signal from the coaxial cable for power supply, thereby ensuring that the DC power required by the intelligent cockpit domain controller 4 can be effectively transmitted through the coaxial cable and provide power to the multiple surround view cameras 2.

[0078] For the embodiments of the present disclosure, Figure 3 As shown, the camera module of each surround view camera 2 is composed of a second optical sensor 21 (CMOS sensor), a second serializer 22 (Serilizer) and a second optical lens 23 (lens).

[0079] For the embodiments of the present disclosure, Figure 2 and Figure 3 As shown, the surround-view camera 2 can be connected to the system chip (SoC2) of the smart cockpit domain controller 4 via the second serializer 22 and the second deserializer 42. The system chip (SoC2) performs configuration management such as initialization on the surround-view camera 2 and simultaneously receives the surround-view image data from the second deserializer 42.

[0080] like Figure 3 As shown, in order to synchronize the data between surround view camera 2 and ADAS camera 1, surround view camera 2 needs to be triggered by the same frame synchronization signal. This can be achieved by:

[0081] The surround view image data of the surround view camera 2 is sent to the SoC2 of the smart cockpit domain controller 4 through the mipi interface. At the same time, a copy of the same surround view image data is sent to the third serializer 41. The third serializer 41 converts the surround view image data into a second LVDS signal and sends it to the first deserializer 31 inside the ADAS domain controller 3. The first deserializer 31 parses the surround view image data and sends it to the SoC1 of the ADAS domain controller 3 through the mipi interface. As mentioned above, the GPIO transparent transmission function can be realized between the serializer and the deserializer. Therefore, when the SoC of the ADAS domain controller 3 sends a frame synchronization signal to the ADAS camera 1, it also connects this frame synchronization signal to the second deserializer 42 that receives the surround-view camera 2. By utilizing the GPIO transparent transmission function between the second deserializer 42 and the third serializer 41, the frame synchronization signal is sent to the third serializer 41 of the smart cockpit domain controller 4. A GPIO is connected between the third serializer 41 of the smart cockpit domain controller 4 and the second deserializer 42 of the smart cockpit domain controller 4 to send the frame synchronization signal to the second deserializer 42, and then to the surround-view camera 2, thereby realizing the synchronous triggering of the surround-view camera 2 and the ADAS camera 1.

[0082] According to actual measurement, Figure 4 and Figure 5 As shown, the delay error between the synchronous triggering of surround view camera 2 and ADAS camera 1 is approximately 5.5µs, and the jitter of the frame synchronization signal is approximately 120ns, resulting in a total error of less than 6µs. Assuming a vehicle speed of 120km / h, the impact of this error is estimated to be 33.33m / s*6µs=200µm. Therefore, the impact of adding a serializer-deserializer stage is negligible.

[0083] like Figure 6 As shown, an embodiment of the present disclosure provides a vehicle-mounted camera synchronization method, which can be applied to the above-mentioned vehicle-mounted camera synchronization system and executed by an ADAS domain controller in the vehicle-mounted camera synchronization system. The vehicle-mounted camera synchronization method may include:

[0084] Step 101: synchronously send a frame synchronization signal to multiple ADAS cameras and multiple surround view cameras.

[0085] Among them, Advanced Driver-Assistance Systems (ADAS) cameras are sensors that can be installed on cars to provide visual information about the vehicle's surroundings to assist the driver in performing driving tasks. ADAS cameras can monitor the vehicle's surroundings in different driving environments, provide warnings, and assist in performing functions such as adaptive cruise control (ACC), lane keeping assist (LKA), forward collision warning (FCW), and blind spot monitoring (BSM).

[0086] Surround-view cameras, also known as 360-degree cameras, provide a comprehensive 360-degree visual overview of the vehicle's surroundings. These cameras can be mounted on multiple sides of the vehicle to provide a comprehensive view of the vehicle's surroundings. These cameras work together, using image processing technology to fuse images from multiple perspectives to create a complete view of the vehicle's surroundings. The driver can view this panoramic view on the display screen and perform various maneuvers, such as parking, reversing, and automated parking.

[0087] For the embodiments of the present disclosure, the executing entity may be an ADAS domain controller, which sends a frame synchronization signal to all ADAS cameras and surround-view cameras. The frame synchronization signal is used to ensure that multiple ADAS cameras and multiple surround-view cameras start capturing image data at the same time point, and synchronously receive image data captured by multiple ADAS cameras and multiple surround-view cameras.

[0088] Step 102 : When the multiple ADAS cameras and the multiple surround view cameras receive a frame synchronization signal, synchronously receive ADAS image data exposed by the multiple ADAS cameras and surround view image data exposed by the multiple surround view cameras.

[0089] Among them, ADAS image data is image information captured by the ADAS camera, which may include information about the road in front of the vehicle and the surrounding environment, and can be used to implement functions such as adaptive cruise control and collision warning; surround view image data is image information captured by the visual camera, which may include visual information around the vehicle, and can be used to assist parking and monitor the conditions around the vehicle.

[0090] For the embodiments of the present disclosure, Figure 1As shown, the frame synchronization signal is sent by the ADAS domain controller and is sent to all ADAS cameras and the smart cockpit domain controller through the deserializer on the ADAS domain controller. After receiving the frame synchronization signal, the smart cockpit domain controller sends it to all surround-view cameras through the serializer and deserializer to achieve frame synchronization of all cameras. Compared with the original solution that did not achieve synchronization of all cameras, this application has minimal impact on system performance because the delay and jitter of the added first-level serializer and deserializer are controllable, and only two signal lines are added, so the hardware cost is extremely low. Through the GPIO transparent transmission function between the serializer and deserializer, the function of triggering all cameras with the same source frame synchronization signal is realized.

[0091] This application realizes the frame synchronization of the surround-view camera and the ADAS camera, which can ensure that all cameras are exposed at the same time. After the transmission link delay, the SoC of the ADAS domain controller can obtain all camera data at the same time, and use the BEV perception algorithm to fuse all the captured camera data to obtain more accurate perception of the vehicle's surrounding environment.

[0092] In summary, according to a vehicle-mounted camera synchronization method disclosed in the present invention, multiple ADAS cameras can be connected to an ADAS domain controller through a coaxial cable, and the multiple ADAS cameras are used to receive a frame synchronization signal sent by the ADAS domain controller. The multiple ADAS cameras start exposure at the same time based on the frame synchronization signal, and transmit the exposed ADAS image data to the ADAS domain controller; the smart cockpit domain controller is connected to the ADAS domain controller through a coaxial cable, and the smart cockpit domain controller is used to receive a frame synchronization signal sent by the ADAS domain controller, and based on the frame synchronization signal, synchronously send the frame synchronization signal to multiple surround-view cameras; multiple surround-view cameras are connected to the smart cockpit domain controller through a coaxial cable, and the multiple surround-view cameras are used to receive a frame synchronization signal. The multiple surround-view cameras start exposure at the same time based on the frame synchronization signal, and transmit the exposed surround-view image data to the ADAS domain controller; the ADAS domain controller is used to synchronously send frame synchronization signals to the multiple ADAS cameras and the multiple surround-view cameras, and synchronously receive ADAS image data and surround-view image data. For the disclosed embodiment, the ADAS domain controller first generates a frame synchronization signal and transmits it to each ADAS camera and surround-view camera through a coaxial cable. After receiving the synchronization signal, each ADAS camera and surround-view camera will start exposure according to the time point indicated by the signal. After the exposure is completed, each camera transmits the image data back to the ADAS domain controller, thereby achieving the consistency of time when the image data captured by the ADAS camera and the surround-view camera arrives at the perception operation unit, thereby improving the accuracy of the perception results of the BEV perception algorithm.

[0093] Based on the above Figure 6The specific implementation of the method shown in this embodiment provides a vehicle-mounted camera synchronization device, such as Figure 7 As shown, the device includes: a sending module 31, a receiving module 32;

[0094] a sending module 31, configured to synchronously send a frame synchronization signal to the plurality of ADAS cameras and the plurality of surround view cameras, wherein the frame synchronization signal is configured to ensure that the plurality of ADAS cameras and the plurality of surround view cameras start capturing image data at the same time point;

[0095] The receiving module 32 is configured to synchronously receive the ADAS image data exposed by the multiple ADAS cameras and the surround view image data exposed by the multiple surround view cameras when the multiple ADAS cameras and the multiple surround view cameras receive the frame synchronization signal.

[0096] It should be noted that for other corresponding descriptions of the functional units involved in the vehicle-mounted camera synchronization device provided in this embodiment, please refer to Figure 6 The corresponding description of the method in will not be repeated here.

[0097] Based on the above Figure 6 The method shown in FIG. 1 is a method for performing the above-mentioned steps. Accordingly, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program can realize the above-mentioned steps. Figure 6 The method shown.

[0098] Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present disclosure.

[0099] Based on the above Figure 6 The method shown, and Figure 7 In order to achieve the above-mentioned purpose, the embodiment of the present disclosure further provides an electronic device that can be configured on the vehicle (such as an electric vehicle) side, the device including a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 6 The method shown.

[0100] Based on the above electronic device, an embodiment of the present application further provides a vehicle, which may specifically include: the above electronic device. The vehicle may specifically be an electric vehicle, etc.

[0101] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, and the like. The user interface may include a display, an input unit such as a keyboard, and the like. The optional user interface may also include a USB interface, a card reader interface, and the like. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), and the like.

[0102] Those skilled in the art will understand that the above-mentioned physical device structure provided by the present disclosure does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0103] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.

[0104] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that the present disclosure can be implemented by means of software plus the necessary general hardware platform, or by means of hardware. Compared with the prior art, the present disclosure provides a vehicle-mounted camera synchronization system, method, device, and electronic device. The present disclosure connects multiple ADAS cameras to an ADAS domain controller via a coaxial cable, wherein the multiple ADAS cameras are configured to receive a frame synchronization signal sent by the ADAS domain controller, and the multiple ADAS cameras simultaneously start exposure based on the frame synchronization signal and transmit the exposed ADAS image data to the ADAS domain controller; the smart cockpit domain controller is connected to the ADAS domain controller via a coaxial cable, wherein the smart cockpit domain controller is configured to receive a frame synchronization signal sent by the ADAS domain controller and synchronously send the frame synchronization signal to multiple surround view cameras based on the frame synchronization signal; the multiple surround view cameras are connected to the smart cockpit domain controller via a coaxial cable, wherein the multiple surround view cameras are configured to receive a frame synchronization signal, and the multiple surround view cameras simultaneously start exposure based on the frame synchronization signal and transmit the exposed surround view image data to the ADAS domain controller; the ADAS domain controller is configured to synchronously send frame synchronization signals to the multiple ADAS cameras and the multiple surround view cameras, and synchronously receive the ADAS image data and the surround view image data. For the disclosed embodiment, the ADAS domain controller first generates a frame synchronization signal and transmits it to each ADAS camera and surround-view camera through a coaxial cable. After receiving the synchronization signal, each ADAS camera and surround-view camera will start exposure according to the time point indicated by the signal. After the exposure is completed, each camera transmits the image data back to the ADAS domain controller, thereby achieving the consistency of time when the image data captured by the ADAS camera and the surround-view camera arrives at the perception operation unit, thereby improving the accuracy of the perception results of the BEV perception algorithm.

[0105] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0106] The above are merely specific embodiments of the present application, which are intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments herein, but rather is intended to conform to the widest scope consistent with the principles and novel features of the present application.

Claims

1. A vehicle-mounted camera synchronization system, characterized in that: The system includes: multiple ADAS cameras, multiple surround view cameras, an ADAS domain controller, and an intelligent cockpit domain controller; The multiple ADAS cameras are connected to the ADAS domain controller via a coaxial cable harness, and the multiple ADAS cameras are used to receive a frame synchronization signal sent by the ADAS domain controller. The multiple ADAS cameras simultaneously start exposure based on the frame synchronization signal and transmit the exposed ADAS image data to the ADAS domain controller; The smart cockpit domain controller is connected to the ADAS domain controller via a coaxial cable. The smart cockpit domain controller is configured to receive the frame synchronization signal sent by the ADAS domain controller and synchronously send the frame synchronization signal to the multiple surround view cameras based on the frame synchronization signal. The multiple surround-view cameras are connected to the smart cockpit domain controller via a coaxial cable, and the multiple surround-view cameras are used to receive the frame synchronization signal. The multiple surround-view cameras simultaneously start exposure based on the frame synchronization signal and transmit the exposed surround-view image data to the ADAS domain controller; The ADAS domain controller is used to synchronously send the frame synchronization signal to the multiple ADAS cameras and the multiple surround view cameras, and synchronously receive the ADAS image data and the surround view image data.

2. The vehicle-mounted camera synchronization system according to claim 1, characterized in that: The multiple ADAS cameras include a first optical sensor and a first serializer, and the ADAS domain controller includes a first deserializer and a GPIO pin; The first optical sensor is connected to the first serializer via a GPIO pin. The first optical sensor is configured to receive the frame synchronization signal sent by the ADAS domain controller via the first deserializer and the first serializer, perform exposure based on the frame synchronization signal, and convert the exposed ADAS image data into a first MIPI digital signal. The first serializer is connected to the first deserializer via a coaxial cable harness, and the first serializer is configured to receive the first MIPI digital signal, convert the first MIPI digital signal into a first LVDS signal for long-distance transmission, and transmit the frame synchronization signal; The first deserializer is connected to the GPIO pin, and the first deserializer is used to transmit the frame synchronization signal using the GPIO pin and receive the first LVDS signal through a coaxial cable; The GPIO pin is used to directly transmit the frame synchronization signal.

3. The vehicle-mounted camera synchronization system according to claim 2, characterized in that: The system further includes: a first Lpoc inductor; The first Lpoc inductor is used to transmit the first DC power signal on the ADAS domain controller side, and to simultaneously superimpose the first DC power signal and the first LVDS signal on the coaxial cable harness, and to extract the first DC power signal in the coaxial cable harness to power the multiple ADAS cameras.

4. The vehicle-mounted camera synchronization system according to claim 1, characterized in that: The plurality of surround view cameras include a second optical sensor and a second serializer, and the smart cockpit domain controller includes a third serializer and a second deserializer; The second optical sensor is connected to the second serializer via a GPIO pin. The second optical sensor is configured to receive the frame synchronization signal sent by the ADAS domain controller via the first deserializer and the third serializer, perform exposure based on the frame synchronization signal, and convert the exposed surround view image data into a second MIPI digital signal. The second serializer is connected to the second deserializer via a coaxial cable, and the second serializer is used to receive the second MIPI digital signal and transmit the frame synchronization signal; The second deserializer and the third serializer are connected via a GPIO pin, and the second deserializer is configured to transmit the frame synchronization signal via the GPIO pin and receive the second MIPI digital signal via a coaxial cable. The third serializer is connected to the first deserializer via a coaxial cable harness, and the third serializer is used to receive the second MIPI digital signal, convert the second MIPI digital signal into a second LVDS signal for long-distance transmission, and transmit the frame synchronization signal; The first deserializer is configured to transmit the frame synchronization signal using the GPIO pin and receive the second LVDS signal through a coaxial cable.

5. The vehicle-mounted camera synchronization system according to claim 4, characterized in that: The system further includes: a second Lpoc inductor; The second LPOC inductor is used to transmit the second DC power signal on the smart cockpit domain controller side, and simultaneously superimpose the second DC power signal and the second LVDS signal on the coaxial cable harness, and extract the second DC power signal in the coaxial cable harness to power the multiple surround-view cameras.

6. A vehicle-mounted camera synchronization method, characterized in that: The method is applied to the vehicle-mounted camera synchronization system according to any one of claims 1 to 5, and the method includes: Synchronously sending a frame synchronization signal to the plurality of ADAS cameras and the plurality of surround view cameras, wherein the frame synchronization signal is used to ensure that the plurality of ADAS cameras and the plurality of surround view cameras start capturing image data at the same time point; When the multiple ADAS cameras and the multiple surround view cameras receive the frame synchronization signal, the ADAS image data exposed by the multiple ADAS cameras and the surround view image data exposed by the multiple surround view cameras are synchronously received.

7. A vehicle-mounted camera synchronization device, characterized in that: The device comprises: a sending module, configured to synchronously send a frame synchronization signal to the plurality of ADAS cameras and the plurality of surround-view cameras, wherein the frame synchronization signal is configured to ensure that the plurality of ADAS cameras and the plurality of surround-view cameras start capturing image data at the same time point; A receiving module is used to synchronously receive the ADAS image data exposed by the multiple ADAS cameras and the surround view image data exposed by the multiple surround view cameras when the multiple ADAS cameras and the multiple surround view cameras receive the frame synchronization signal.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to claim 6 is implemented.

9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to claim 6 is implemented.

10. A vehicle, characterized in that: include: The vehicle-mounted camera synchronization system according to any one of claims 1 to 5, or the device according to claim 7, or the computer-readable storage medium according to claim 8, or the electronic device according to claim 9.

Images