Intelligent driving domain control fused door opening early warning method and device and automobile

Abstract

The invention relates to the technical field of automobile door opening early warning, and provides an intelligent driving domain control fused door opening early warning method and device and an automobile. The method comprises the following steps: when a micro-control unit is powered on and awakened to start working, receiving a first target alarm signal sent by a radar, and forwarding the first target alarm signal to a domain controller; after the on-chip system connected with the multi-view camera is powered on and started, radar target detection data are obtained from the micro-control unit, an image collected by the multi-view camera is recognized, image target detection data are obtained, a second target alarm signal is determined according to the radar target detection data and the image target detection data, and the second target alarm signal is sent to the micro-control unit. A second target alarm signal is periodically sent to the micro-control unit; according to the invention, target fusion is carried out through the radar and the camera, so that the tracking alarm of the door opening early warning function on special-shaped targets and general targets is expanded, and the usability of the door opening early warning function is improved.

Description

Technical Field

[0001] The present application relates to the technical field of automobile door opening warning, and in particular to a door opening warning method, device and automobile integrated with intelligent driving and domain control. Background Art

[0002] The door opening warning system, or DOW (short for Door Open Warning), is a technology that works when the door of a passenger car is about to be opened when it is parked. It can monitor other road users on the side and rear of the car, and promptly issue warning information to the driver and passengers when there is a risk of collision due to the opening of the door. For example, when the driver and passengers open the car door when the car is parked, there is a risk of collision between the car door and the vehicle coming from the rear and the person getting off the car. At this time, the system will issue a risk warning to the driver and passengers, and the warning information can indicate the side where the risk occurs. The door opening warning system can help the driver and passengers perceive the potential risk of collision in advance and take effective risk avoidance measures.

[0003] In the prior art, vehicle-mounted perception sensors can be used to monitor obstacles on the side and rear of the car. For example, millimeter-wave radars (left and right rear millimeter-wave radars are respectively arranged on the left and right sides of the rear of the car) can be used to detect rear targets whose distance from the car is less than the preset collision distance. When the target is detected to be moving close, the relative speed of the target is evaluated, and when a potential risk is detected, an early warning will be issued. Alternatively, left and right surround cameras can be used to detect the blind spot behind the car, and the early warning signal is sent to the body controller. The body controller lights up the red warning sign on the corresponding side and locks the four doors at the same time. In addition, lateral laser radars and cameras can be used to generate car environment information based on image data and point cloud information, so as to improve the accuracy and timeliness of the car's perception of lateral environmental targets. However, these existing solutions all have some shortcomings. Among them, pure millimeter-wave detection can only process specific targets; and the use of cameras for surround detection has slow startup and a small recognition range, and can identify few types of targets; in addition, the use of laser radars is too expensive and lacks practicality.

[0004] In addition, the existing technology also has a solution that uses a combination of radar and camera, but because the camera and radar perceive data differently, the corresponding controllers are also different, so only time-sharing detection can be performed, and there is a lack of target fusion, so the detection effect is the same as that of a single perception sensor, that is, the types of detected targets are small.

[0005] In summary, in view of the shortcomings of the existing door opening warning scheme, how to expand the tracking and alarm of the door opening warning system's detection targets at a low cost is a technical problem that needs to be solved in the actual application of the current door opening warning system. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a door opening warning method, device and automobile with integrated intelligent driving and domain control to solve the problem of how to expand the tracking and alarm of the door opening warning system's detection targets at a low cost.

[0007] In a first aspect of an embodiment of the present application, a door opening warning method of intelligent driving domain control fusion is provided, which is applied to a car including a domain controller, an intelligent driving controller, a millimeter wave radar and a camera, wherein the domain controller is connected to the intelligent driving controller, the intelligent driving controller includes a micro control unit and a system on chip, the millimeter wave radar includes a left rear corner radar and a right rear corner radar, and is connected to the micro control unit, the camera includes at least one of a left rear side camera, a right rear side camera and a rear camera, and is connected to the system on chip, and the door opening warning method of intelligent driving domain control fusion includes: when the micro control unit in the intelligent driving controller is powered on and wakes up to start working, the micro control unit receives first target alarm signals sent by the left rear corner radar and the right rear corner radar respectively, and forwards the first target alarm signal to the domain controller; After the domain controller is powered on and started up by the on-chip system connected to the multi-view camera in the intelligent driving controller, the on-chip system obtains the radar target detection data reported by the left rear corner radar and the right rear corner radar from the microcontroller unit through preset subscription rules, and the on-chip system recognizes the image collected by the multi-view camera to obtain image target detection data; the on-chip system determines the second target alarm signal based on the radar target detection data and the image target detection data; the on-chip system periodically sends the second target alarm signal to the microcontroller unit; when receiving the second target alarm signal, the microcontroller unit stops forwarding the first target alarm signal to the domain controller, and broadcasts the second target alarm signal to the domain controller; the domain controller responds to the first target alarm signal and the second target alarm signal, and executes the preset door opening warning measures.

[0008] According to a second aspect of an embodiment of the present application, a door opening warning device integrating intelligent driving and domain control is provided, which is applied to a car including a domain controller, an intelligent driving controller, a millimeter-wave radar and a camera. The domain controller is connected to the intelligent driving controller, the intelligent driving controller includes a microcontroller unit and a system on a chip, the millimeter-wave radar includes a left rear corner radar and a right rear corner radar, and is connected to the microcontroller unit, the camera includes at least one of a left rear side camera, a right rear side camera and a rear camera, and is connected to the system on a chip, and the door opening warning device integrating intelligent driving and domain control includes: a first warning processing module, configured to receive first target alarm signals sent by the left rear corner radar and the right rear corner radar respectively when the microcontroller unit in the intelligent driving controller is powered on and wakes up and starts working, and forward the first target alarm signal to the domain controller; a target data fusion module, configured to be connected in the intelligent driving controller After the system on chip of the multi-view camera is powered on and started, the system on chip obtains the radar target detection data reported by the left rear corner radar and the right rear corner radar from the micro control unit through the preset subscription rules, and the system on chip recognizes the image collected by the multi-view camera to obtain the image target detection data; the second warning processing module is configured so that the system on chip determines the second target alarm signal according to the radar target detection data and the image target detection data; the warning signal sending module is configured so that the system on chip periodically sends the second target alarm signal to the micro control unit; the signal reporting control module is configured so that when the micro control unit receives the second target alarm signal, it stops forwarding the first target alarm signal to the domain controller and broadcasts the second target alarm signal to the domain controller; the car door opening warning module is configured so that the domain controller responds to the first target alarm signal and the second target alarm signal and executes the preset door opening warning measures.

[0009] According to a third aspect of an embodiment of the present application, a car is provided, comprising a door opening warning system, the door opening warning system comprising a millimeter wave radar, a camera, an intelligent driving controller, and a domain controller, the intelligent driving controller comprising at least a microcontroller unit, a system on a chip, a memory, and a computer program stored in the memory and capable of correspondingly running on the microcontroller unit and the system on a chip, the microcontroller unit being connected to the millimeter wave radar, the system on a chip being connected to the camera, the microcontroller unit and the system on a chip being communicatively connected, and the steps of the above method are implemented when the microcontroller unit and the system on a chip execute the computer program.

[0010] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the door opening warning method integrated with intelligent driving and domain control starts working when the microcontroller unit in the intelligent driving controller is powered on and awakened, and the microcontroller unit receives the first target alarm signals sent by the left rear corner radar and the right rear corner radar respectively, and forwards the first target alarm signal to the domain controller; after the domain controller is powered on and started up with the system-on-chip connected to the multi-view camera in the intelligent driving controller, the system-on-chip obtains the radar target detection data reported by the left rear corner radar and the right rear corner radar from the microcontroller unit through the preset subscription rules, and the system-on-chip recognizes the images collected by the multi-view camera to obtain the image target detection data; the on-chip system determines the second target alarm signal according to the radar target detection data and the image target detection data; the on-chip system periodically sends the second target alarm signal to the microcontroller unit; when the microcontroller unit receives the second target alarm signal, it stops forwarding the first target alarm signal to the domain controller, and broadcasts the second target alarm signal to the domain controller; the domain controller responds to the first target alarm signal and the second target alarm signal, and executes the preset door opening warning measures to integrate the radar and the camera to realize the DOW function, thereby expanding the tracking and alarm of the DOW function to special-shaped targets and general targets, and improving the availability and intelligence of the DOW function. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 It is a flowchart of a door opening warning method for intelligent driving domain control fusion provided in an embodiment of the present application;

[0013] Figure 2 It is a schematic diagram of the architecture of a door opening warning system provided in an embodiment of the present application;

[0014] Figure 3 It is a left / right rear and rear-view camera coverage and millimeter-wave radar warning area map provided in an embodiment of the present application;

[0015] Figure 4 It is a structural schematic diagram of a door opening warning device integrating intelligent driving and domain control provided in an embodiment of the present application;

[0016] Figure 5 It is a partial structural schematic diagram of a car provided in an embodiment of the present application;

[0017] Figure 6The embodiment of this application provides Figure 2 A structural diagram of MCU in FIG. DETAILED DESCRIPTION

[0018] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0019] See also Figure 1 In the first embodiment of the present application, a door opening warning method for intelligent driving domain control fusion is provided, comprising the steps of:

[0020] S101, when the microcontroller connected to the left rear corner radar and the right rear corner radar in the intelligent driving controller is powered on and wakes up to start working, the microcontroller receives the first target alarm signal sent by the left rear corner radar and the right rear corner radar respectively, and forwards the first target alarm signal to the domain controller;

[0021] S102, after the chip system connected to the multi-view camera in the intelligent driving controller is powered on and started, the chip system obtains radar target detection data reported by the left rear corner radar and the right rear corner radar from the microcontroller unit according to a preset subscription rule, and the chip system recognizes the image collected by the multi-view camera to obtain image target detection data;

[0022] S103, the system on chip determines a second target alarm signal according to the radar target detection data and the image target detection data;

[0023] S104, the system on chip periodically sends a second target alarm signal to the micro control unit;

[0024] S105, when receiving the second target alarm signal, the micro control unit stops forwarding the first target alarm signal to the domain controller, and broadcasts the second target alarm signal to the domain controller;

[0025] S106: The domain controller responds to the first target alarm signal and the second target alarm signal and executes preset door opening warning measures.

[0026] In practical applications, the above-mentioned intelligent driving domain control fusion door opening warning method is applied to Figure 2In the car of the system architecture shown, the system architecture includes a domain controller, an intelligent driving controller, a millimeter-wave radar and a camera, wherein the domain controller includes at least one of a BCM (abbreviation of Body Control Module, translated as a body controller) and a HUT (abbreviation of Head Unit, translated as a vehicle controller or a cockpit controller), and an ADC (abbreviation of Automotive Domain Controller, translated as an intelligent driving controller) includes an MCU (abbreviation of Microcontroller Unit, translated as a microcontroller unit or a microcontroller unit) and a SOC (abbreviation of System on Chip, translated as a system on chip or a system-on-chip), and the ADC, BCM and HUT are communicatively connected in sequence, the MCU is respectively connected to CR_RL (representing a left rear corner radar) and CR_RR (representing a right rear corner radar), and the SOC is respectively connected to the cameras at the left rear view, right rear view and rear view positions, that is, the SOC is connected to at least one of the left rear side camera, the right rear side camera and the rear camera.

[0027] This embodiment utilizes the structural characteristics that the radar and the camera are respectively controlled by the MCU and the SoC in the intelligent driving controller. Since the MCU is powered on and woken up before the SoC, the radar first sends the first target alarm signal to the MCU before the SoC is started up, and then the MCU forwards it to the BCM / HUT to report the warning signal; after the SoC is started up, the SoC will subscribe to the target information reported by the radar from the MCU side, and after completing the target fusion of the radar and the camera, the alarm signal (such as the second target alarm signal) for entering the preset alarm area with a collision risk is sent to the MCU, and the MCU broadcasts the alarm signal to the BCM / HUT. It can be seen that the embodiment of the present application realizes the DOW (abbreviation of Door open Warning, translated as door opening warning) function by combining radar and camera for target fusion, expands the tracking alarm of the DOW function for special-shaped targets and general targets, and improves the availability and intelligence of the DOW function.

[0028] Specifically, combined Figure 2As shown in the figure, MCU and SoC are two modules integrated in the intelligent driving controller, and the intelligent driving controller, BCM and HUT are connected in sequence. Among them, MCU is connected to at least the left rear corner radar and the right rear corner radar on the vehicle, and SoC is connected to the left rear side camera, the right rear side camera and the rear camera respectively. In actual applications, the power-on startup of MCU is faster than that of SoC. Therefore, when MCU and SoC are powered on at the same time, MCU will enter the working state before SoC. Then, in the gap between the MCU powering on and starting to work and the SoC powering on and starting, the left rear corner radar and the right rear corner radar detect the door opening warning, and then send the corresponding first target alarm signal to the MCU, and the MCU forwards the first target alarm signal to the BCM / HUT, etc. After the SoC is started, the SoC subscribes to the target information reported by the MCU side rear corner radars (i.e., the left rear corner radar and the right rear corner radar). At this time, the MCU no longer directly forwards the first target alarm signals of the two left rear corner radars and the right rear corner. After the SoC completes the target fusion, the second target alarm signal for entering the alarm area with a collision risk is sent to the MCU, and the MCU broadcasts the second target alarm signal to the BCM / HUT, thereby completing the reporting of the door opening warning signal. Finally, the BCM / HUT can execute corresponding warning measures according to the contents of the first target alarm signal and the second target alarm signal, such as lighting up the blind spot light, sending a vibration alarm signal, etc., to realize the door opening warning function.

[0029] The first target alarm signal can be a status identifier. For example, 0 and 1 are used to represent the detection result of the radar. When the radar does not detect the risk of door opening collision, the status identifier is set to "0" and the first target alarm signal with the status identifier of "0" is sent to the MCU; when the radar detects the risk of door opening collision, the status identifier is set to "1" and the first target alarm signal with the status identifier of "1" is sent to the MCU.

[0030] In practice, the radar can generate the first target alarm signal, and the MCU is responsible for receiving and forwarding the first target alarm signal. The MCU can also calculate and generate the first target alarm signal based on the radar detection data, that is, the radar sends the radar target detection data to the MCU, the MCU generates the first target alarm signal based on the radar target detection data, and reports the first target alarm signal to the BCM / HUT.

[0031] In an optional embodiment, before the MCU receives the first target warning signal sent by the left rear corner radar and the right rear corner radar respectively, the process includes:

[0032] The left rear corner radar and the right rear corner radar obtain radar target detection data by sending radar signals, and judge whether there is a collision risk when opening the door based on the radar target detection data: if not, a first alarm status mark is generated; if so, a second alarm status mark is generated;

[0033] The left rear corner radar and the right rear corner radar use the generated first alarm state identifier or the second alarm state identifier as the first target alarm signal, and send the first target alarm signal to the MCU.

[0034] The left rear corner radar and the right rear corner radar are installed at the rear corners of the car on both sides to detect obstacles in the surrounding environment and measure their position and speed. The left rear corner radar and the right rear corner radar can use ultrasonic radar, millimeter wave radar or infrared radar. For example, taking millimeter wave radar as an example, the basic principle of millimeter wave radar to detect the position and speed of obstacles is that millimeter wave radar emits high-frequency electromagnetic waves, which propagate in space. When these electromagnetic waves encounter obstacles, they will be reflected. The electromagnetic waves reflected back by obstacles are called echoes. The receiving antenna of the millimeter wave radar will capture these echo signals, and the radar system will analyze these echo signals to extract information such as the distance and speed of the target. Specifically, the radar system calculates the distance of the target by measuring the time delay from the emission of the electromagnetic wave to the reception of the reflected signal, thereby determining the position of the target. When the target moves relative to the radar, the frequency of the reflected wave changes. This phenomenon is called the Doppler effect. By measuring the frequency shift of the reflected wave, the radar can calculate the relative speed of the target. It can be seen that the radar target detection data in this embodiment includes the detected target (i.e., obstacle), as well as the position and speed of the target, wherein the number of targets detected by the radar can be one or more. When there are multiple targets, the radar target detection data contains the position and speed of each target.

[0035] Among them, when the radar detects a target, there is no unique specific implementation method for determining whether opening the car door will collide with the target.

[0036] In an optional embodiment, when the radar target detection data includes the position and speed of a first target obstacle identified by the radar, it is determined whether there is a collision risk when opening the vehicle door based on the radar target detection data, including: determining whether the position of the first target obstacle is within a preset alarm area; if the first target obstacle is not within the alarm area, determining that there is no collision risk; if the first target obstacle is within the preset alarm area, estimating whether the vehicle door will collide with the first target obstacle when open based on the speed of the first target obstacle: if the vehicle door will collide with the first target obstacle, determining that there is a collision risk; if the vehicle door will not collide with the first target obstacle, determining that there is no collision risk.

[0037] Specifically, the warning area is a virtual detection range, which is pre-defined as a specific area around the car. When the target object enters this area, the system will activate the corresponding early warning mechanism.

[0038] For example, combined with Figure 3 For example, a rectangular alarm area is set on each side of the car, which can be 1-2 meters in size and covers the range extending from the door to the outside. Any object that quickly approaches or enters the area will trigger the generation of an alarm signal. The left rear corner radar and the right rear corner radar continuously scan the environment around the car, identify and track the position and speed of obstacles. In this scenario, when an obstacle enters the preset alarm area, the system immediately analyzes its position, speed and movement trajectory in detail, and predicts the movement trajectory of the obstacle in the next few seconds. If the predicted trajectory intersects with the range of the door opening and the time coincides, the system will determine that there is a high risk of collision, and thus generate a second alarm status mark; otherwise, it is determined that there is no collision risk and a first alarm status mark is generated.

[0039] It should be noted that the two radars work independently, and the two radars send the first target alarm signal to the MCU respectively, and the MCU forwards the first target alarm signals sent by the two radars.

[0040] This embodiment uses radar to perform target monitoring for door opening warning. When a target is detected that will not cause a door opening collision, a first alarm status identifier is generated as a first target alarm signal. When a target is detected that will cause a door opening collision, a second alarm status identifier is generated as a first target alarm signal to reduce the risk of a collision before the SoC startup is completed.

[0041] After the SoC is started, it will detect targets through the camera. By combining the two perception sensors, radar and camera, it can improve the accuracy of target detection and make up for the shortcomings of both radar and camera.

[0042] In an optional embodiment, the SoC recognizes the image captured by the multi-view camera to obtain image target detection data, including: inputting the image captured by the multi-view camera into a pre-trained deep learning model for recognition to obtain image target detection data of whether there is a second target obstacle in the image; when there is a second target obstacle in the image, based on the position change of the obstacle feature points in the continuous images captured by the multi-view camera, determining the position and speed of the second target obstacle, wherein the image target detection data includes the position and speed of the second target obstacle.

[0043] The pre-trained deep learning model is used to identify the images captured by the multi-view camera and determine whether there is a target (i.e., obstacle) in the image. If an obstacle is identified in the image, the features of the target in the image are used to calculate the position and speed of the target, thereby obtaining the image target detection data. Of course, the image target detection data also includes the detection results of no obstacles in the image.

[0044] It should be noted that existing technical means can be used to implement deep learning models and determine the position and speed of target obstacles in images. For example, neural network models such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) or Faster R-CNN are selected. After training with a large number of annotated image data sets (such as COCO, Cityscapes, etc.), the model can effectively identify pedestrians, cars, obstacles and other targets in the image. That is, after the training is completed, the model can identify the images collected by multi-view cameras to obtain image target detection data of whether there is a second target obstacle in the image. In addition, after the target is detected using the deep learning model, the system will extract the feature points of the target in continuous image frames (such as the target is a person, the target's feature points can be the head, feet, etc.). These feature points can be tracked by algorithms such as optical flow, SIFT (Scale-Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF) to determine the speed and movement trend of the target.

[0045] This embodiment uses SoC to analyze and process images collected by multi-view cameras to obtain image target detection data, thereby achieving early warning monitoring of obstacles within the camera monitoring range, so as to combine radar target detection data for target fusion.

[0046] After the SoC is started, the SoC will subscribe to the target information reported by the radar from the MCU, and then perform target fusion of the radar and camera to generate a second target alarm signal for door opening warning.

[0047] In an optional embodiment, a second target alarm signal is determined based on radar target detection data and image target detection data, including: judging whether there is a collision risk when opening the car door based on the radar target detection data: if not, generating a first alarm status identifier; if so, generating a second alarm status identifier; judging whether there is a collision risk when opening the car door based on the image target detection data, if not, generating a first alarm status identifier; if so, generating a second alarm status identifier; fusing the first alarm status identifier or the second alarm status identifier generated by the radar target detection data and the image target detection data to obtain a second target alarm signal.

[0048] Specifically, the SoC obtains the obstacle detection data of the radar, i.e., the radar target detection data, by subscribing to the radar target detection data, including but not limited to the first target obstacle detected by the radar, and the position and speed of the first target obstacle. Then, the first target obstacle, the position and speed of the first target obstacle, and the preset alarm area are used to estimate whether the door will collide with the first target obstacle when the door is opened, thereby generating a corresponding alarm state mark. At the same time, the SoC analyzes and processes the image collected by the camera to obtain the obstacle detection data of the camera, i.e., the image target detection data, including but not limited to the second target obstacle detected by the camera, and the position and speed of the second target obstacle. Similarly, the second target obstacle, the position and speed of the second target obstacle, and the preset alarm area are used to estimate whether the door will collide with the second target obstacle when the door is opened, thereby generating a corresponding alarm state mark. Finally, the alarm state mark generated by the collision risk judgment of the radar target detection data and the image target detection data is fused to generate a second target alarm signal.

[0049] It is understood that the first target obstacle detected by the radar and the second target obstacle detected by the camera may be the same target or different targets. For example, assuming that the first target obstacle detected by the radar includes targets A and B, and the second target obstacle detected by the camera may also include targets A and B, or the second target obstacle detected by the camera is targets B and C, wherein target B is a target that can be detected by both the radar and the camera, and A is a target that can be detected by the radar but not by the camera, and C is a target that cannot be detected by the radar but can be detected by the camera.

[0050] When the first target obstacle detected by the radar and the second target obstacle detected by the camera are the same target, perception data fusion optimization can be performed on the same target based on the radar target detection data and the image target detection data, including fusing the position and / or speed of the same target.

[0051] For example, suppose the obstacle recognition results of the millimeter-wave radar and camera are as follows:

[0052] The first target obstacle recognition result of the millimeter wave radar is:

[0053] Obstacle ID: 1;

[0054] Distance: 10.5 meters;

[0055] Angle: 15 degrees.

[0056] The second target obstacle recognition result of the rearview mirror camera:

[0057] Obstacle ID: 1 (i.e. same target);

[0058] Distance: 10.3 meters;

[0059] Bounding box: [100,50,200,150].

[0060] The results after fusion of the same target:

[0061] Obstacle ID: 1;

[0062] Distance: 10.4 meters (average value), or use simple weighted summation method for fusion. Assuming the weights are 0.6 and 0.4 respectively, the distance is: 10.5*0.6+10.3*0.4=10.42;

[0063] Angle: 15 degrees;

[0064] Bounding box: [100,50,200,150].

[0065] It can be seen that when the position data of the same target in the radar target detection data and the image target detection data are inconsistent, the average value or weighted sum of the position data of the same target in the radar target detection data and the image target detection data is calculated and used as the position information of the same target to achieve target fusion.

[0066] Among them, the specific implementation method of judging whether there is a collision risk when opening the car door based on the radar target detection data can refer to the above embodiment and will not be repeated here.

[0067] In addition, there is no unique way to determine whether there is a risk of collision when opening a vehicle door based on radar target detection data.

[0068] For example, in an optional embodiment, based on the image target detection data, it is determined whether there is a collision risk when opening the vehicle door, including: determining whether the position of the second target obstacle is within a preset alarm area; if the second target obstacle is not within the alarm area, determining that there is no collision risk; if the second target obstacle is within the preset alarm area, estimating whether the vehicle door will collide with the second target obstacle when open based on the speed of the second target obstacle: if the vehicle door will collide with the second target obstacle, determining that there is a collision risk; if the vehicle door will not collide with the second target obstacle, determining that there is no collision risk.

[0069] It can be seen that judging whether there is a collision risk when opening the car door based on the image target detection data is the same as judging whether there is a collision risk when opening the car door based on the radar target detection data. The specific working principle can be found in the specific implementation method of the radar target detection data, which will not be repeated here.

[0070] The embodiment of the present application realizes target fusion by performing collision risk judgment on the target information of the radar and the camera respectively, and generating a second target alarm signal by combining the collision risk judgment results of the two. Such target fusion can make up for the shortcomings of the radar and the camera in target detection, thereby achieving a stronger effect of target early warning monitoring and improving the accuracy of early warning.

[0071] In actual applications, the MCU needs to switch the alarm signal before and after the SoC is started, which may easily lead to signal loss and chaotic broadcasting. In order to avoid signal loss and chaotic broadcasting caused by the switch between MCU direct forwarding and MCU sending fusion alarm signals, the following optional embodiments are provided to solve the problem that the MCU needs to switch the alarm signal sending, which may easily lead to signal loss and chaotic broadcasting.

[0072] In an optional embodiment, the SoC periodically sends a second target alarm signal to the MCU, including: when the SoC obtains the second target alarm signal through the first fusion after power-on, sending the second target alarm signal according to the first cycle; when the SoC obtains a new second target alarm signal through the fusion again, sending a new second target alarm signal according to the second cycle, wherein the second cycle is greater than the first cycle.

[0073] Specifically, before the SoC side sends the second target alarm signal for the first time, the MCU still directly forwards the first target alarm signal. When the SoC completes the first target fusion processing after power-on, the second target alarm signal is obtained. At this time, the SoC first sends a small cycle (i.e., the first cycle) interception signal to the MCU quickly, which is used to characterize that the SoC can report the alarm signal, so that the MCU stops forwarding the first target alarm signal, and quickly sends a small cycle interception signal, which can avoid the situation where the MCU cannot receive the interception signal in time and continues to forward the first target alarm signal due to frame drops and other situations. After that, the SoC sends the alarm signal to the MCU with a normal signal cycle (i.e., the second cycle), and the MCU forwards the fused alarm signal. It can be understood that the first cycle and the second cycle are both signal transmission cycles. The SoC sends the second target alarm signal obtained by the target fusion through the cycle, which can avoid the situation where the signal is not sent successfully due to factors such as frame loss, and ensures the stability of signal transmission. In addition, the interception signal can be a separate signal, or it can be the second target alarm signal obtained by the first fusion of the SoC. In the embodiment of the present application, the interception signal is preferably the second target alarm signal obtained by the first fusion of the SoC.

[0074] In addition, in view of regulatory requirements, the car must also support the DOW function for 3 minutes after the KL15 is powered off. Therefore, the normal operation of the door opening warning function must still be guaranteed after the car is powered off, which will obviously increase the energy consumption of the car. Therefore, how to reduce energy consumption while ensuring the availability of the DOW function while meeting regulatory requirements is a technical problem that needs to be faced in practical applications.

[0075] In an optional embodiment, the above Figure 1The method shown also includes: after the car is started, detecting whether the KL15 signal of the car is in a power-off state; when the KL15 signal is in a power-off state, detecting whether the car has turned on the door opening warning function; when the door opening warning function is turned on, the intelligent driving controller powers off and sleeps according to the AUTOSAR process; when the door opening warning function is not turned on, the intelligent driving controller continues to maintain the normal working state (i.e., Normal Operation State) in the AUTOSAR NM message, and sends the NM message with PN (abbreviation of Partial Network, translated as partial network) set to act as a holding source to continue to wake up the BCM and HUT to maintain work; and the intelligent driving controller only retains the underlying OS (abbreviation of Operating System, translated as operating system) and DOW functional modules and turns off other functional modules, and starts the internal timing for N minutes: if a locking state signal is received within the N-minute timing, the intelligent driving controller terminates the timing and performs the power-off sleep process according to the AUTOSAR NM process; if no locking state signal is received within N minutes, after the timing of N minutes, the intelligent driving controller performs the power-off sleep process according to the AUTOSAR NM process. Among them, N≥3.

[0076] In automotive electrical systems, KL30B, KL30, and KL15 refer to specific electrical connection points or power states. These symbols are electrical terms in the German automotive industry standard (DIN standard) used to describe the electrical systems in automobiles. KL30 refers to the power point connected directly to the positive terminal of the car battery. KL30 usually means always on, that is, it is always powered under any circumstances (regardless of the state of the ignition switch). This connection point is usually used for circuits that need to be continuously powered, such as in-car clocks, alarm systems, and memory functions of ECUs (engine control units). KL30B is a controlled version of KL30, which is also connected to the positive terminal of the battery, but the control of the power supply is managed by the car's control module or electronic control unit (ECU). KL30B is usually used for circuits that need to be powered off under certain specific conditions. For example, when the car is stationary, KL30B can be powered off to avoid excessive battery discharge, but when the engine is running or under certain conditions, KL30B will re-power. This connection point is commonly used in circuits where non-critical loads (such as some in-car entertainment systems or charging ports) need to prevent battery power from being consumed when parked for a long time. KL15 refers to the power line controlled by the ignition switch, which is energized when the ignition switch is in the "ON" or "IGN" (ignition) position. The KL15 power point is usually used for circuits that need to be energized when the engine is started or the car is running, such as the engine control unit (ECU), fuel pump, instrument panel, and other equipment related to the operation of the engine. When the ignition switch is turned off (OFF), KL15 will be powered off to prevent unnecessary power consumption. In this embodiment, it is detected whether the KL15 signal of the car is in the power-off state, that is, it is detected whether the position of the ignition switch of the car is in the power-on position. If the position of the ignition switch is in the power-on position, it means that the car is currently in the starting power-on state; if the position of the ignition switch is not in the power-on position, it means that the car is currently in the power-off state. Users generally turn off the power and lock the car after parking. Therefore, the KL15 signal is detected to determine whether to continue to maintain the DOW function after powering off.

[0077] Specifically, the two corner radars do not perform AUTOSAR direct network management, but only connect to the vehicle's KL30B relay (which can delay disconnection of the KL30B relay after power-off). They can be powered on and work in the ACC position after the car is unlocked. The intelligent driving controller is connected to the KL30 normal power and the vehicle's KL15 as a wake-up signal. After KL30B is powered on, the user opens the door or turns the key to the ACC position, and the MCU of the intelligent driving controller can be powered on and wake up to start working.

[0078] Among them, AUTOSAR (the abbreviation of AUTomotive Open System ARchitecture) is an open and standardized automotive software architecture platform for developing and managing embedded system software in automobiles. In combination with this embodiment, the AUTOSAR network manages the power-off process: when the ECU (the abbreviation of Electronic Control Unit, translated as electronic control unit or electronic control unit) is working normally, it is in a normal operating state. After KL15 OFF (this embodiment refers to after the DOW function is enabled or after the car is locked), since there is no longer a demand for communication, it enters the ready sleep state, waits for the timer to overflow in the ready sleep state, enters the ready longitudinal sleep stage, waits for the timer to overflow in the bus sleep stage, and formally enters the bus sleep state, and the ECU completes the power-off sleep.

[0079] In addition, the "NM" in the AUTOSAR NM message stands for Network Management. The NM message is a message type used in the AUTOSAR architecture to manage and coordinate the ECU nodes in the automotive network. Setting refers to setting a state or flag bit to "1" or an active state.

[0080] In practical applications, ADC, BCM and HUT are all divided into a PN function group. ADC controls the above steps according to the switch status of the DOW function sent by the HUT, thereby reducing energy consumption and ensuring the availability of the DOW function while meeting regulatory requirements.

[0081] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0082] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0083] See also Figure 4 In a second embodiment of the present application, a door opening warning device integrating intelligent driving and domain control is provided, which is applied to at least Figure 2 In the car with the system architecture shown, the door opening warning device of the intelligent driving domain control fusion includes:

[0084] The first warning processing module 401 is configured to receive the first target warning signal sent by the left rear corner radar and the right rear corner radar respectively when the micro control unit in the intelligent driving controller is powered on and wakes up to start working, and forward the first target warning signal to the domain controller;

[0085] The target data fusion module 402 is configured to, after the system on chip connected to the multi-view camera in the intelligent driving controller is powered on and started, obtain the radar target detection data reported by the left rear corner radar and the right rear corner radar from the micro control unit through a preset subscription rule, and the system on chip recognizes the image collected by the multi-view camera to obtain the image target detection data;

[0086] The second warning processing module 403 is configured as a system on a chip to determine a second target alarm signal according to the radar target detection data and the image target detection data;

[0087] The warning signal sending module 404 is configured as the system on chip to periodically send a second target warning signal to the micro control unit;

[0088] The signal reporting control module 405 is configured so that when the micro control unit receives the second target alarm signal, it stops forwarding the first target alarm signal to the domain controller and broadcasts the second target alarm signal to the domain controller;

[0089] The automobile door opening warning module 406 is configured as a domain controller to respond to the first target alarm signal and the second target alarm signal and execute a preset door opening warning measure.

[0090] in, Figure 2 The system architecture shown has been described in detail above and will not be repeated here. According to the technical solution provided in the embodiment of the present application, the radar and the camera are controlled by the MCU and SoC in the intelligent driving controller respectively. Since the MCU is powered on and wakes up before the SoC, the radar sends the first target alarm signal to the MCU before the SoC is started, and then the MCU forwards it to the BCM / HUT to report the warning signal; and after the SoC is started, the SoC will subscribe to the target information reported by the radar from the MCU side, and after completing the target fusion of the radar and the camera, the alarm signal (such as the second target alarm signal) for entering the preset alarm area with a risk of collision is sent to the MCU, and the MCU broadcasts the alarm signal to the BCM / HUT. It can be seen that the embodiment of the present application realizes the DOW function by combining the radar and the camera for target fusion, which can expand the tracking alarm of special-shaped targets and general targets, and improve the availability and intelligence of the DOW function.

[0091] In some optional embodiments, the door opening warning device of the intelligent driving domain control fusion further includes:

[0092] The power-off function control module 407 is configured to detect whether the position of the ignition switch of the automobile is in the power-on position after the automobile is started; if the position of the ignition switch is not in the power-on position, detect whether the door opening warning function of the automobile is turned on; when the door opening warning function is turned on, the ADC is powered off and hibernated according to the AUTOSAR process; when the door opening warning function is not turned on, the ADC continues to maintain the normal operation state in the AUTOSAR NM message, and sends the NM message with PN set to serve as a holding source to continue to wake up the BCM and HUT to maintain work; and the ADC only retains the underlying OS and DOW functional modules and closes other functional modules, and starts the internal timing for 3 minutes: if a locking state signal is received within the 3-minute timing, the ADC terminates the timing and goes through the power-off hibernation process according to the AUTOSAR NM process; if no locking state signal is received within 3 minutes, after the 3-minute timing, the ADC goes through the power-off hibernation process according to the AUTOSAR NM process.

[0093] In some optional embodiments, the above-mentioned first warning processing module is configured to, before the MCU respectively receives the first target alarm signals sent by the left rear corner radar and the right rear corner radar, the left rear corner radar and the right rear corner radar obtain radar target detection data by sending radar signals, and judge whether there is a collision risk when opening the vehicle door based on the radar target detection data: if not, a first alarm status identifier is generated; if so, a second alarm status identifier is generated; the left rear corner radar and the right rear corner radar use the generated first alarm status identifier or the second alarm status identifier as the first target alarm signal, and send the first target alarm signal to the MCU.

[0094] In some optional embodiments, the target data fusion module 402 is configured to input the image captured by the multi-view camera into a pre-trained deep learning model for recognition, and obtain image target detection data of whether there is a second target obstacle in the image; when there is a second target obstacle in the image, the position and speed of the second target obstacle are determined based on the position changes of the obstacle feature points in the continuous images captured by the multi-view camera, wherein the image target detection data includes the position and speed of the second target obstacle.

[0095] In some optional embodiments, the above-mentioned second warning processing module 403 is configured to determine whether there is a collision risk when opening the car door based on the radar target detection data: if not, a first alarm status identification is generated; if so, a second alarm status identification is generated; based on the image target detection data, determine whether there is a collision risk when opening the car door, if not, a first alarm status identification is generated; if so, a second alarm status identification is generated; and the first alarm status identification or the second alarm status identification generated by the radar target detection data and the image target detection data are integrated to obtain a second target alarm signal.

[0096] In some optional embodiments, the radar target detection data includes the position and velocity of the first target obstacle identified by the radar;

[0097] The above-mentioned first warning processing module is configured to determine whether the position of the first target obstacle is within a preset alarm area; if the first target obstacle is not within the alarm area, it is determined that there is no collision risk; if the first target obstacle is within the preset alarm area, based on the speed of the first target obstacle, it is estimated whether the vehicle door will collide with the first target obstacle when it is open: if the vehicle door will collide with the first target obstacle, it is determined that there is a collision risk; if the vehicle door will not collide with the first target obstacle, it is determined that there is no collision risk.

[0098] In some optional embodiments, the second warning processing module 403 is configured to determine whether the position of the second target obstacle is within a preset alarm area; if the second target obstacle is not within the alarm area, it is determined that there is no collision risk; if the second target obstacle is within the preset alarm area, based on the speed of the second target obstacle, it is estimated whether the vehicle door will collide with the second target obstacle when open: if the vehicle door will collide with the second target obstacle, it is determined that there is a collision risk; if the vehicle door will not collide with the second target obstacle, it is determined that there is no collision risk.

[0099] In some optional embodiments, the above-mentioned early warning signal sending module 404 is configured to send the second target alarm signal according to the first period when the second target alarm signal is obtained by the first fusion after the SoC is powered on; when the SoC fuses again to obtain a new second target alarm signal, a new second target alarm signal is sent according to the second period, wherein the second period is greater than the first period.

[0100] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0101] See also Figure 2 and Figure 5In the third embodiment of the present application, a car is provided, including a door opening warning system, the door opening warning system includes a millimeter wave radar, a camera, an ADC, and a BCM / HUT. The ADC includes at least an MCU and a SoC. The MCU is connected to the millimeter wave radar, the SoC is connected to the camera, and the MCU and the SoC are communicatively connected.

[0102] exist Figure 2 In the figure, CR_RL represents the left rear corner radar, and CR_RR represents the right rear corner radar; the two rear corner radars are connected to the MCU of ADC, and SoC subscribes to the radar target, left rear view camera, right rear view camera and rear view camera on the MCU side through ETH using SOME / IP to connect to SoC; the ECUs communicate with each other using CAN bus, BCM is used to control the lighting of blind spot lights, and HUT is responsible for sound graded alarm prompts.

[0103] Among them, the two corner radars do not do direct AUTOSAR network management, but only connect to the vehicle's KL30B relay (the KL30B relay can be delayed to disconnect after power off), and can be powered on and work in the ACC position after the car is unlocked. The ADC is connected to the KL30 normal power and connected to the vehicle's KL15 as a wake-up signal. After the KL30B is powered on, the user opens the door or turns the key to the ACC position, and the ADC's MCU can be powered on and wake up to start working. The rear corner radar sends the target and alarm signals (blind spot lights, sound classification, vibration alarm signals) to the MCU, and the MCU forwards the alarm signals to the BCM / HUT, etc. Usually, the ADC's MCU wakes up quickly and can work quickly, while the SoC cold start is slow (because it takes time to load the underlying driver + OS startup, and it takes about 30 to 60 seconds to start). Before the SoC fails to work, the MCU directly forwards the blind spot lights / alarm signals of the two rear corner radars. When the SoC is started, the SoC subscribes to the target information reported by the MCU's side rear corner radar. At this time, the MCU no longer directly forwards the alarm signals of the two rear corner radars. After the SoC completes the target fusion, the alarm signal of the risk of collision entering the alarm area is sent to the MCU, and the MCU broadcasts the alarm signal to the BCM / HUT.

[0104] In addition, since the SoC is connected to more than one camera, including but not limited to two side rear cameras and a rear-view camera, the images captured by the two side rear cameras and the rear-view camera can be fused by BEV (Bird's Eye View, usually called a bird's eye view or a panoramic view), and then the camera combined with the detection data of the millimeter-wave radar can be used to perform target fusion. Specifically, after the camera captures the image, the SoC extracts feature points from the image or uses a deep learning model to identify obstacles, and analyzes the position changes of the obstacle feature points in the continuous image (optical flow method) to estimate its position and speed, and then tracks the target, which can well identify pedestrians, special-shaped obstacles or special sports cars.

[0105] ADC is the abbreviation of Automotive Domain Controller, which is generally translated as intelligent driving controller. Intelligent driving controllers usually integrate the functions of multiple electronic control units (ECUs). This means that it can process data from multiple sensors, such as cameras, radars, lidars, ultrasonic sensors, etc., and make complex driving decisions. In addition, in order to process large amounts of sensor data in real time, intelligent driving controllers usually contain high-performance computing units such as multi-core processors (MCUs) and system-on-chips (SoCs), and sometimes integrate dedicated accelerators or AI processors to support deep learning algorithms and complex environmental perception tasks.

[0106] Among them, MCU in ADC is the abbreviation of Microcontroller Unit, which is generally translated as micro control unit. SoC in ADC is the abbreviation of System on Chip, which is generally translated as system-level chip. SoC is a highly integrated chip that integrates multiple components in a computer system into a single chip, thereby achieving low power consumption, high performance and small size of complex functions. In ADC, MCU and SoC can store and run programs separately and work independently of each other. They can also work together to realize the complex functions of intelligent driving, for example, to realize the steps in the above method embodiment, or to realize the functions of each module in the above device embodiment.

[0107] See also Figure 6 , the MCU in the ADC includes a central processing unit, a memory, and a computer program stored in the memory and executable on the central processing unit. When the central processing unit executes the computer program stored in the memory, some steps in the above-mentioned method embodiments are implemented. Alternatively, when the central processing unit executes the computer program, the functions of each module in the above-mentioned device embodiments are implemented. Of course, in practice, the MCU may also include at least one structure of an I / O port, a timer / counter, an analog-to-digital converter / digital-to-analog converter, a communication interface, and a power management.

[0108] Similarly, the SoC in the ADC may also include a central processing unit, a memory, and a computer program stored in the memory and executable on the central processing unit. When the central processing unit executes the computer program, some steps in the above-mentioned method embodiments are implemented. Alternatively, when the central processing unit executes the computer program, the functions of each module in the above-mentioned device embodiments are implemented. In addition, in practice, the SoC may also include structures such as a graphics processing unit, a digital signal processor, a memory controller, an I / O interface, a hardware accelerator, a power management unit, and a communication module.

[0109] ETH is the abbreviation of Ethernet. In automobiles, Ethernet is used for high-speed data communication between MCU and SoC in ADC.

[0110] SOME / IP (short for Scalable service-Oriented MiddlewarE over IP) is a communication protocol used in automotive electronic systems. In a car, when the SoC sends a subscription request through the SOME / IP protocol, it obtains the detection results reported by the radar to the MCU, including but not limited to radar target detection data.

[0111] BCM is the abbreviation of Body Control Module, which is generally translated as body controller. It is a control unit responsible for managing and controlling the internal electrical system of the car and also has gateway function.

[0112] HUT is the abbreviation of Head Unit, which is generally translated as cockpit controller or in-vehicle infotainment system host.

[0113] The automobile provided in this embodiment combines the camera and the radar for target fusion to realize the DOW function, which can expand the tracking and alarm of special-shaped targets and general targets, and improve the availability and intelligence of the DOW function.

[0114] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional modules in the embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. Computer-readable storage media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0116] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A door opening warning method for intelligent driving and domain control integration is applied to a car including a domain controller, an intelligent driving controller, a millimeter wave radar and a camera, wherein the domain controller is connected to the intelligent driving controller, the intelligent driving controller includes a micro control unit and a system on chip, the millimeter wave radar includes a left rear corner radar and a right rear corner radar, and is connected to the micro control unit, the camera includes at least one of a left rear side camera, a right rear side camera and a rear camera, and is connected to the system on chip, It is characterized in that The door opening warning method of intelligent driving domain control fusion includes: When the microcontroller unit in the intelligent driving controller is powered on and wakes up to start working, the microcontroller unit receives the first target alarm signals sent by the left rear corner radar and the right rear corner radar respectively, and forwards the first target alarm signals to the domain controller; After the domain controller is powered on and started up by the system-on-chip connected to the multi-view camera in the intelligent driving controller, the system-on-chip obtains radar target detection data reported by the left rear corner radar and the right rear corner radar from the microcontroller unit through a preset subscription rule, and the system-on-chip recognizes the image collected by the multi-view camera to obtain image target detection data; The system on chip determines a second target alarm signal according to the radar target detection data and the image target detection data; The system on chip periodically sends the second target alarm signal to the micro control unit; When receiving the second target alarm signal, the micro control unit stops forwarding the first target alarm signal to the domain controller and broadcasts the second target alarm signal to the domain controller; The domain controller responds to the first target alarm signal and the second target alarm signal to execute preset door opening warning measures.

2. The method according to claim 1, It is characterized in that Also includes: After the car is started, check whether the ignition switch of the car is in the power-on position; If the ignition switch is not in the power-on position, check whether the car has the door opening warning function turned on; When the door opening warning function is turned on, the intelligent driving controller will be powered off and put into sleep mode according to the automotive open system architecture process; When the door opening warning function is not turned on, the intelligent driving controller continues to maintain the normal working state in the vehicle open system architecture network management message, and sends the network management message of some network settings to act as a holding source to continue to wake up the body controller and the cockpit controller to maintain work; The intelligent driving controller only retains the underlying operating system and door opening warning function modules and shuts down other function modules, and starts the internal timing for N minutes: if a vehicle locking status signal is received within the N-minute timing, the intelligent driving controller terminates the timing and goes into power-down process according to the automotive open system architecture network management process; if no vehicle locking status signal is received within N minutes, after the timing for N minutes, the intelligent driving controller goes into power-down process according to the automotive open system architecture network management process.

3. The method according to claim 1, It is characterized in that Before the micro control unit receives the first target warning signal sent by the left rear corner radar and the right rear corner radar respectively, it includes: The left rear corner radar and the right rear corner radar acquire radar target detection data by sending radar signals, and judge whether there is a collision risk when opening the door according to the radar target detection data: if not, a first alarm state mark is generated; if so, a second alarm state mark is generated; The left rear corner radar and the right rear corner radar use the generated first alarm status identifier or the second alarm status identifier as a first target alarm signal, and send the first target alarm signal to the micro control unit.

4. The method according to claim 1, It is characterized in that The system-on-chip recognizes the images collected by the multi-view cameras and obtains image target detection data, including: The image captured by the multi-view camera is input into the pre-trained deep learning model for recognition to obtain image target detection data of whether there is a second target obstacle in the image; When there is a second target obstacle in the image, the position and speed of the second target obstacle are determined based on the position changes of the obstacle feature points in the continuous images captured by the multi-view camera, wherein the image target detection data includes the position and speed of the second target obstacle.

5. The method according to claim 4, It is characterized in that The system on chip determines a second target alarm signal according to the radar target detection data and the image target detection data, including: According to the radar target detection data, determine whether there is a collision risk when opening the door: if not, generate a first alarm state mark; if so, generate a second alarm state mark; According to the image target detection data, determine whether there is a collision risk when opening the door, and if not, generate a first alarm state mark; if so, generate a second alarm state mark; The first alarm status identifier or the second alarm status identifier generated by the radar target detection data and the image target detection data is integrated to obtain a second target alarm signal.

6. The method according to claim 3 or 5, It is characterized in that The radar target detection data includes the position and speed of the first target obstacle identified by the radar; The determining, based on the radar target detection data, whether there is a collision risk when opening the door includes: Determining whether the position of the first target obstacle is within a preset alarm area; If the first target obstacle is not within the warning area, it is determined that there is no collision risk; If the first target obstacle is within the preset warning area, based on the speed of the first target obstacle, it is estimated whether the vehicle door will collide with the first target obstacle when it is opened: If the vehicle door will collide with the first target obstacle, it is determined that there is a collision risk; If the vehicle door will not collide with the first target obstacle, it is determined that there is no collision risk.

7. The method according to claim 5, It is characterized in that The determining, based on the image target detection data, whether there is a collision risk when opening the door includes: Determining whether the position of the second target obstacle is within a preset alarm area; If the second target obstacle is not within the warning area, it is determined that there is no collision risk; If the second target obstacle is within the preset warning area, based on the speed of the second target obstacle, it is estimated whether the door will collide with the second target obstacle when it is opened: If the vehicle door will collide with the second target obstacle, it is determined that there is a collision risk; If the vehicle door will not collide with the second target obstacle, it is determined that there is no collision risk.

8. The method according to any one of claims 1 to 5, It is characterized in that The system on chip periodically sends the second target alarm signal to the micro control unit, including: When the second target alarm signal is obtained by the first fusion after the system on chip is powered on, the second target alarm signal is sent according to the first cycle; When the on-chip system is integrated again to obtain a new second target alarm signal, the new second target alarm signal is sent according to a second period, wherein the second period is greater than the first period.

9. A door opening warning device integrating intelligent driving and domain control is applied to a car including a domain controller, an intelligent driving controller, a millimeter-wave radar and a camera, wherein the domain controller is connected to the intelligent driving controller, the intelligent driving controller includes a microcontroller unit and a system on a chip, the millimeter-wave radar includes a left rear corner radar and a right rear corner radar, and is connected to the microcontroller unit, the camera includes at least one of a left rear side camera, a right rear side camera and a rear camera, and is connected to the system on a chip, It is characterized in that The door opening warning device of the intelligent driving domain control fusion includes: The first warning processing module is configured to receive the first target warning signals sent by the left rear corner radar and the right rear corner radar respectively by the micro control unit when the micro control unit in the intelligent driving controller is powered on and wakes up to start working, and forward the first target warning signals to the domain controller; The target data fusion module is configured to, after the system on chip connected to the multi-view camera in the intelligent driving controller is powered on and started, obtain the radar target detection data reported by the left rear corner radar and the right rear corner radar from the microcontroller unit through a preset subscription rule, and the system on chip recognizes the image collected by the multi-view camera to obtain the image target detection data; A second warning processing module is configured as a system on a chip to determine a second target alarm signal according to the radar target detection data and the image target detection data; An early warning signal sending module is configured to periodically send the second target alarm signal to the micro control unit through the system on chip; The signal reporting control module is configured so that when the micro control unit receives the second target alarm signal, it stops forwarding the first target alarm signal to the domain controller and broadcasts the second target alarm signal to the domain controller; The automobile door opening warning module is configured so that the domain controller responds to the first target alarm signal and the second target alarm signal and executes a preset door opening warning measure.

10. A car, comprising a door opening warning system, Features: The door opening warning system includes a millimeter-wave radar, a camera, an intelligent driving controller, and a domain controller. The intelligent driving controller includes at least a microcontroller unit, a system on a chip, a memory, and a computer program stored in the memory and correspondingly executable on the microcontroller unit and the system on a chip. The microcontroller unit is connected to the millimeter-wave radar, the system on a chip is connected to the camera, the microcontroller unit and the system on a chip are communicatively connected, and the microcontroller unit and the system on a chip implement the steps of the method as claimed in any one of claims 1 to 8 when executing the computer program.

Images