Target detection method and system for vehicle, vehicle and equipment

Abstract

The invention discloses a target detection method and system for a vehicle, the vehicle and equipment. The target detection method for the vehicle comprises the following steps: when a target exists in a dangerous target area, preliminarily identifying whether the target is an accident vehicle or not according to sensing data of a camera; if yes, state information and position information of the accident vehicle are determined based on scanning data of the radar and sensing data of the camera; if the preliminarily identified target is an obstacle of other types except the accident vehicle, a danger coefficient of the target is obtained according to the scanning data and perception data of a camera, and otherwise, the danger coefficient of the target is obtained according to the scanning data; and when the target is judged to be a dangerous target according to the danger coefficient, state information and position information of the accident vehicle are determined. According to the embodiment of the invention, the method has the advantages of high recognition speed, accuracy and reliability of the accident vehicle (such as the side vehicle), and guarantees the intelligent driving safety of the vehicle.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and specifically to a vehicle target detection method, system, vehicle and equipment. Background Art

[0002] Current intelligent driving technology can only cover common typical scenarios, such as following a car, congested roads, and vehicles cutting in from the front, but it is difficult to respond appropriately to some uncommon dangerous conditions. For drivers, dangerous conditions with a small probability are also unbearable. The rollover accident scenario is one of the dangerous conditions that is difficult to cover with current intelligent driving technology. These dangerous conditions urgently need effective intelligent driving technology to solve them.

[0003] Rollover accident vehicles are not a common scene, and there are many types of rollover accident vehicles and multiple rollover angles, and it is difficult to collect image data for similar dangerous working conditions. At present, the mainstream technology for cameras to detect and identify rollover accident vehicles is achieved through intelligent algorithm models based on neural networks. However, the current perception model of the camera lacks a training set of rollover vehicles, which results in the poor recognition ability of current smart cameras for rollover accident vehicles. The distance at which the camera can identify the target is relatively close, basically within 100 meters. If there is light interference, such as backlight or dim light, the distance index of the camera's target recognition will be greatly reduced. In addition, if you encounter a white box truck that has overturned in front of you, the white roof is facing the on-board camera of the main vehicle. It is easy to identify it as the sky and miss the inspection.

[0004] Millimeter-wave radar is an active detection sensor that mainly obtains information such as the distance, speed, and acceleration of the target based on the difference frequency information between the target echo and the transmitted wave. At the same time, millimeter-wave radar can also identify objects based on the distribution characteristics of the target's radar backscatter cross-section (RCS), but the recognition ability is relatively weak. Due to the transmission characteristics of electromagnetic waves, millimeter-wave radars are easily affected by multipath and roadside plant interference, resulting in more noise points. Most current millimeter-wave radars filter out stationary targets and rollover vehicles, which results in the vehicle-mounted perception system not being able to detect rollover vehicles. In addition, even if a rollover vehicle is detected, it cannot be correctly identified, and millimeter-wave radar alone cannot complete the detection of rollover accident vehicles.

[0005] Because vehicle-mounted millimeter-wave radars are prone to interference, in order to avoid false detections that cause false triggering of the control system, the weight of radar detection information is generally reduced in the camera and millimeter-wave radar fusion algorithm, while the weight of camera detection information is increased. That is, the detection and recognition capabilities of the vision and millimeter-wave radar fusion technology mainly rely on the camera, and the millimeter-wave radar only plays an auxiliary role. However, the camera is affected by light, has a short distance, and lacks training sets, which makes it difficult for the current mainstream vision and millimeter-wave radar fusion technology to cover special working conditions, especially dangerous scenes where vehicles roll over. Summary of the invention

[0006] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a multi-vehicle target detection method, system, vehicle and equipment, which have the advantages of fast, accurate and reliable recognition of accident vehicles (such as side vehicles), thereby ensuring the intelligent driving safety of vehicles.

[0007] In a first aspect, an embodiment of the present application provides a method for detecting a target of a vehicle, wherein a radar and a camera are provided on the vehicle, and the method for detecting a target comprises:

[0008] When there is a target in the dangerous target area, preliminarily identifying whether the target is an accident vehicle based on the perception data of the camera;

[0009] If yes, determining the state information and position information of the accident vehicle based on the scanning data of the radar and the perception data of the camera;

[0010] If the target is initially identified as an obstacle of another type other than the accident vehicle, a risk factor of the target is obtained according to the scanning data and the perception data of the camera; otherwise, a risk factor of the target is obtained according to the scanning data;

[0011] When the target is judged as a dangerous target according to the risk coefficient, the state information and position information of the accident vehicle are determined.

[0012] Further, when the target is judged as a dangerous target according to the risk coefficient, the state information and position information of the accident vehicle are determined, including:

[0013] Determining whether the risk factor is higher than a preset risk factor threshold;

[0014] If yes, the target is determined to be a dangerous target, and the state information and position information of the accident vehicle are obtained; otherwise, a prompt for a non-dangerous target is triggered.

[0015] Furthermore, it also includes:

[0016] When there is a target in the warning target area, preliminarily identifying whether the target is an accident vehicle based on the perception data of the camera, the warning target area is an area farther away from the vehicle than the dangerous target area;

[0017] If yes, trigger a warning prompt according to the scanning data of the radar and the perception data of the camera; otherwise, obtain the danger factor of the target according to the scanning data and the perception data of the camera;

[0018] When the target is judged as a warning target according to the risk factor, the state information and position information of the target are determined.

[0019] Further, when the target is judged as a warning target according to the risk factor, the state information and position information of the target are determined, including:

[0020] Determining whether the risk factor is higher than a preset warning factor threshold;

[0021] If so, it is determined that the target is a warning target, and the state information and position information of the target are obtained; otherwise, a prompt of a non-warning target is triggered.

[0022] Furthermore, it also includes:

[0023] When there is a target in the warning target area, obtaining a danger factor of the target according to the scanning data of the radar, wherein the warning target area is an area farther from the vehicle than the warning target area;

[0024] When the target is judged as a warning target according to the risk factor, the state information and position information of the accident vehicle are determined.

[0025] Further, obtaining the risk factor of the target according to the scanning data includes:

[0026] The target confidence L is obtained by the following formula: con , the formula is:

[0027] L con =k*e Tn / m

[0028] Among them, T n is the life cycle of the target, k is the impact factor of the life cycle of the target, and m is the threshold value of the target’s life cycle;

[0029] The target's risk factor THQ mmr It is obtained by the following formula, which is:

[0030] THQ mmr =λ1*Lcon +λ2*RCS

[0031] Wherein, RCS is the backscattering cross-sectional area of ​​the target detected by the radar, and λ1 and λ2 are the influencing factors of the target confidence and the backscattering cross-sectional area of ​​the target, respectively.

[0032] Furthermore, obtaining the risk factor of the target according to the scanning data and the perception data of the camera includes:

[0033] The hazard factor THQ of the target is obtained by the following formula fusion , the formula is:

[0034] THQ mmr =λ1*L con +λ2*RCS+λ3*(L*W*H)

[0035] Among them, λ1, λ2, and λ3 are the influencing factors of the target confidence, the backscattering cross-sectional area of ​​the target, and the length, width, and height of the target, respectively. L is the target length, W is the target width, and H is the target height.

[0036] Further, obtaining the danger factor of the target according to the scanning data of the radar includes:

[0037] The hazard factor THQ of the target is obtained by the following formula mmr , the formula is:

[0038] THQ mmr =λ1*L con +λ2*RCS+λ3*(L mmr *W mmr )

[0039] Among them, λ1, λ2, and λ3 are the influencing factors of target confidence, target backscattering cross-sectional area, and target length and width, respectively. mmr is the target length detected by the radar, W mmr is the width of the target detected by the radar.

[0040] Further, the preliminarily identifying whether the target is an accident vehicle based on the perception data of the camera includes:

[0041] The perception data of the camera is input into a recognition model to obtain a result of the recognition model identifying whether the target is an accident vehicle, wherein the perception data includes image information of the target.

[0042] Furthermore, it also includes:

[0043] Providing a collection interface to receive sample data through the collection interface, wherein the source of the sample data includes on-site photographed images and network downloaded images;

[0044] The sample data is input into an initial recognition model, and the initial recognition model is trained according to the loss between the output of the initial recognition model and the label to obtain the recognition model.

[0045] In a second aspect, an embodiment of the present application provides a target detection system for a vehicle, wherein a radar and a camera are provided on the vehicle, and the target detection system comprises:

[0046] A preliminary judgment module is used to preliminarily identify whether a target is an accident vehicle based on the perception data of the camera when there is a target in the dangerous target area;

[0047] a determination module, for determining the state information and position information of the accident vehicle based on the scanning data of the radar and the perception data of the camera when the target is initially identified as an accident vehicle, and for obtaining the danger coefficient of the target based on the scanning data and the perception data of the camera when the target is initially identified as an obstacle of another type other than the accident vehicle; otherwise, obtaining the danger coefficient of the target based on the scanning data;

[0048] The detection module is used to determine the state information and position information of the accident vehicle when the target is judged to be a dangerous target according to the risk coefficient.

[0049] In a third aspect, an embodiment of the present application provides a vehicle, comprising: a target detection system for the vehicle according to the embodiment of the second aspect.

[0050] In a fourth aspect, an embodiment of the present application provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle target detection method as described in the embodiment of the first aspect of the present application is implemented.

[0051] The vehicle target detection method, system, vehicle and equipment proposed in the embodiments of the present application, when there is a target in the dangerous target area, based on the camera's perception data, when the target is preliminarily identified as an accident vehicle, the radar scanning data and the camera's perception data can be used to determine the state information and location information of the accident vehicle. If the preliminarily identified target is another type of obstacle other than the accident vehicle, the scanning data and the camera's perception data can be fused to obtain the target's risk factor. Otherwise, the target's risk factor is obtained based on the scanning data. Finally, when the target is judged to be a dangerous target based on the risk factor, the state information and location information of the accident vehicle can be determined. As a result, the system has the advantages of fast, accurate and reliable recognition of accident vehicles (such as side vehicles), ensuring the intelligent driving safety of the vehicle.

[0052] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0054] Figure 1 A schematic diagram of a process flow of a vehicle target detection method according to an embodiment of the present application;

[0055] Figure 2 A schematic diagram of the fusion detection range of a camera and a millimeter-wave radar in a vehicle target detection method according to an embodiment of the present application;

[0056] Figure 3 A schematic diagram of region division in a vehicle target detection method according to an embodiment of the present application;

[0057] Figure 4 A detailed flowchart of dangerous target detection in a vehicle target detection method according to an embodiment of the present application;

[0058] Figure 5 A detailed flow chart of warning target detection in a vehicle target detection method according to an embodiment of the present application;

[0059] Figure 6 A detailed flow chart of warning target detection in a vehicle target detection method according to an embodiment of the present application;

[0060] Figure 7 This is an overall flow chart of a vehicle target detection method according to an embodiment of the present application;

[0061] Figure 8 A schematic diagram of target recognition based on a neural network in a vehicle target detection method according to an embodiment of the present application;

[0062] Fig. 9 A schematic diagram of training a recognition model of a vehicle target detection method according to an embodiment of the present application;

[0063] Fig.10 A schematic diagram of an upgraded solution for acquisition of a vehicle target detection method and rollover vehicle detection according to an embodiment of the present application;

[0064] Fig.11 A structural block diagram of a vehicle target detection system according to an embodiment of the present application;

[0065] Fig.12 A schematic diagram of the structure of a computing device suitable for implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0066] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant application, rather than to limit the application. It is also necessary to explain that, for ease of description, only the parts related to the application are shown in the accompanying drawings.

[0067] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0068] The following is combined with Figure 1 A target detection method for a vehicle according to an embodiment of the present application is described. In order to address the problem that the current accident scene of a rollover vehicle cannot be detected well, and the problem that a single sensor in the related technology is difficult to detect a rollover vehicle and the millimeter-wave radar has a low weight, the present invention proposes a solution for fusion detection of a forward-looking camera and a forward millimeter-wave radar to detect a rollover vehicle; in addition, in order to address the problem that the millimeter-wave radar is easily interfered with and the camera detection is inaccurate during the sensor fusion process, the present invention adopts a detection solution based on confidence and risk factor; finally, in order to address the problem that it is difficult to establish a rollover vehicle data set, the present invention also proposes a rollover vehicle data set construction solution based on network collection of pictures and a rollover vehicle detection algorithm upgrade solution based on real vehicle collection.

[0069] Figure 1 FIG. 1 is a flow chart of a method for detecting a target vehicle according to an embodiment of the present application. Figure 1 As shown, a vehicle target detection method according to an embodiment of the present application includes the following steps:

[0070] S101: When there is a target in the dangerous target area, preliminarily identify whether the target is an accident vehicle based on the perception data of the camera.

[0071] During the driving process, the vehicle mainly focuses on the area of ​​the forward view. That is, the accident vehicle appears in the forward view of the vehicle, which usually affects the safe driving of the vehicle. Therefore, the dangerous target area is located in the area of ​​the forward view.

[0072] A vehicle-mounted radar, such as a forward-looking millimeter-wave radar installed on the vehicle, can be used to preliminarily detect whether there is a target (such as an obstacle) in the dangerous target area. When a target is detected in the dangerous target area, the target can be preliminarily identified as an accident vehicle based on the perception data such as images of the dangerous target area by a camera, such as a vehicle-mounted camera.

[0073] In one embodiment of the present invention, the accident vehicle is, for example, a rollover accident vehicle. For a rollover vehicle accident scene, the forward-looking millimeter-wave radar has a strong ability to detect the dynamic information of the target in the distance direction of the antenna array radiation, that is, in the longitudinal direction, but in the vertical direction of the antenna array radiation, that is, in the lateral direction, there is a deviation in the speed and acceleration of the target detected by the radar. In addition, due to the influence of the radar radiation gain and the antenna radiation sidelobe, the radar increases the deviation in detection accuracy of targets at the edge of the radiation angle. Taking the above factors into consideration, for a rollover vehicle accident scene, in one embodiment of the present application, a solution of multiple millimeter-wave radars is adopted, for example: a sensor fusion solution of installing three millimeter-wave radars on the front bumper of the vehicle and installing a forward-facing high-definition camera on the front windshield is adopted to achieve the purpose of vehicle target recognition.

[0074] like Figure 2 As shown in the figure, the three millimeter wave radars include: two short range radars (SRR) and one long range radar (LRR). The two SRRs are installed on both sides of the front bumper as corner radars. The normal of the radiation array of the corner radar is at an angle of 45° with the center axis of the vehicle. The two corner radars are installed symmetrically. The long range radar LRR is installed in the center of the front bumper, and the normal direction of the array is parallel to the center axis of the vehicle. The camera is installed in the center of the strong windshield to ensure that the center line of the camera's field of view (FOV) is parallel to the center axis of the vehicle.

[0075] In normal vehicle driving, generally speaking, the closer the target is to the vehicle, the greater the threat it poses to the vehicle. Therefore, the first focus is on targets within a short distance of the vehicle's lane or adjacent lanes. If such targets that are close to the vehicle threaten the vehicle's driving, the vehicle should sense such dangerous conditions in the shortest possible time, inform the control system, and leave enough time for the control system to plan and respond to avoid accidents.

[0076] Based on the above analysis, the dangerous targets in the close range are continuously tracked. Since the continuous tracking of the targets will occupy computing resources, the more targets are tracked, the slower the response. Based on this, according to the detection area perceived by the vehicle, there may be a large number of targets in the area, but most of the targets (such as those at a long distance) will not pose a threat to the vehicle, or will not pose a threat in a short time. Therefore, if Figure 3 As shown in the figure, in order to ensure the response speed, the detected targets can be divided into dangerous targets (DO), warning targets (WO), and prompt targets (PO) according to the degree of their proximity to the vehicle. The corresponding forward viewing angles can be the dangerous target area (DA), the warning target area (WA), and the prompt target area (PA). Among them:

[0077] Targets that appear in the dangerous target area, i.e. dangerous targets, are obstacles that pose a danger to moving vehicles and require continuous tracking and monitoring.

[0078] The target that appears in the warning area is at a certain distance from the vehicle, leaving enough time for decision-making and control. It may pose a potential threat to the vehicle and can be monitored. Once sensed, it can continuously track the target with a higher risk factor and provide feedback to the decision-making and control unit.

[0079] Targets that appear in the warning area are at a long distance from the vehicle, leaving a long time for decision-making and control. They usually do not pose a threat or impact on vehicle driving in a short period of time. The intelligent driving system is used to make predictions and judgments, and the perception system predicts the danger of the target based on the target status and pays attention to highly dangerous targets.

[0080] The dangerous target area, warning target area, and alert target area are determined according to conditions such as the scene. For example, the dangerous target area is determined according to the scene, the warning target area is determined according to the camera detection range and scene, and the alert area is determined according to the millimeter-wave radar detection range and scene.

[0081] In one embodiment of the present invention, preliminarily identifying whether a target is an accident vehicle based on the perception data of the camera includes: inputting the perception data of the camera into a recognition model to obtain a result of the recognition model identifying whether the target is an accident vehicle, wherein the perception data includes image information of the target. That is, using a pre-trained neural network model to identify whether the target is an accident vehicle based on the input perception data such as an image.

[0082] S102: If the target is preliminarily identified as an accident vehicle according to the perception data of the camera, the state information and position information of the accident vehicle are determined based on the scanning data of the radar and the perception data of the camera.

[0083] S103: If the target is preliminarily identified as an obstacle of another type other than the accident vehicle, the risk factor of the target is obtained based on the scanning data and the perception data of the camera; otherwise, the risk factor of the target is obtained based on the scanning data.

[0084] S104: When the target is judged to be a dangerous target according to the risk coefficient, the state information and position information of the accident vehicle are determined.

[0085] In a specific example, when the target is judged to be a dangerous target based on the danger coefficient, the status information and position information of the accident vehicle are determined, including: judging whether the danger coefficient is higher than a preset danger coefficient threshold; if so, the target is determined to be a dangerous target, and the status information and position information of the accident vehicle are obtained, otherwise, a prompt for a non-dangerous target is triggered.

[0086] For steps S102 to S104, in a specific example, after detecting the presence of a target (obstacle) in a dangerous area, the image taken by the camera is used to preliminarily determine whether the obstacle is an overturned accident vehicle. If it is preliminarily determined to be an overturned accident vehicle, the perception system will continue to track and monitor the overturned accident vehicle target, and the forward millimeter-wave radar, forward angle radar and forward-looking camera will fuse the detection results of the overturned vehicle, and send the status information of the overturned vehicle to the decision and control unit in real time.

[0087] If the obstacle target is initially judged to be other types of obstacles, it may be missed. Therefore, for other types of targets, further screening of accident vehicles in potential dangerous target areas is performed. The size of the target is calculated using the image taken by the camera, and the forward millimeter-wave radar and two forward angle radars are integrated to track and detect the target's RCS (Radar Cross Section, RCS, i.e.: the target's radar backscatter cross-sectional area) and the tracking existence life cycle. The confidence of the target is positively correlated with the target's existence life cycle, and the confidence of the target can be obtained through the life cycle calculation.

[0088] The radar detection result (scan book) is integrated with the camera measurement result (perception data) to determine the target hazard quotient (THQ). When the THQ is higher than the set threshold, it is considered a hazardous target and the location and status information of the hazardous target is given; when it is lower than the set threshold, it is considered a non-hazardous target and a non-hazardous target prompt is given. The threshold can be pre-set based on experience.

[0089] If the camera does not recognize the target or misses the target, the millimeter-wave radar is used for detection, and the target is detected and tracked based on the fusion of the forward millimeter-wave radar and the forward angle radar. The target's risk factor is determined based on the target's confidence Lcon and RCS value. If it is higher than the set risk factor threshold, it is considered a dangerous target and the target's location and status information is given; if it is lower than the set threshold, the system considers it a non-dangerous target and gives a non-dangerous target prompt. The threshold can be pre-set based on experience.

[0090] like Figure 4 As shown in Figure 1, it is a specific flow chart of target detection in dangerous areas. Figure 4 As shown, the target confidence L con Calculation formula:

[0091] L con =k*e Tn / m (1)

[0092] Among them, T n represents the target life cycle, k represents the target life cycle influencing factor, and m represents the threshold value for determining the existence of the target life cycle.

[0093] Target danger factor THQ based on radar information mmr Calculation formula:

[0094] THQ mmr = λ1 * L con + λ2 * RCS (2)

[0095] Among them, RCS represents the backscatter cross-sectional area obtained by the radar when detecting the target, and λ1 and λ2 represent the influence factors of the radar target confidence and the target radar backscatter cross-sectional area, respectively.

[0096] Target hazard coefficient THQ based on radar and camera fusion fusion Calculation formula:

[0097] THQ mmr = λ1 * L con + λ2 * RCS +λ3 *(L*W*H) (3)

[0098] Among them, λ1, λ2, and λ3 represent the target confidence, the target radar backscattering cross-sectional area, and the length, width, and height influencing factors of the object, respectively. L represents the target length, W represents the target width, and H represents the target height.

[0099] That is, the target risk factor is obtained based on the scan data, including first calculating the target confidence L according to formula 1 con, then, the target risk factor THQ mmr It is obtained by formula 2; according to the scanning data and the perception data of the camera, the method of obtaining the danger coefficient of the target is obtained by formula 3.

[0100] Combination Figure 3 As shown, the target detection method for a vehicle also includes: when there is a target in the warning target area, preliminarily identifying whether the target is an accident vehicle based on the perception data of the camera, and the warning target area is an area farther away from the vehicle than the danger target area; if so, triggering a warning prompt based on the scanning data of the radar and the perception data of the camera, otherwise, obtaining the danger coefficient of the target based on the scanning data and the perception data of the camera; when the target is judged to be a warning target based on the danger coefficient, determining the status information and position information of the target.

[0101] In this example, when the target is judged to be a warning target based on the danger factor, the status information and location information of the target are determined, including: judging whether the danger factor is higher than a preset warning factor threshold; if so, determining that the target is a warning target, and obtaining the status information and location information of the target, otherwise, triggering a prompt for a non-warning target.

[0102] Specifically, after detecting the target in the warning area, the perception system preliminarily determines whether the target is an overturned accident vehicle based on the image captured by the camera. If it is determined to be an overturned accident vehicle, the perception system integrates the detection results of the forward long-range radar, the forward angle radar and the forward-looking camera, and sends a warning message of the overturned vehicle to the rear end. If it is preliminarily determined to be a non-overturned accident vehicle, taking into account the possibility that the camera may miss the overturned vehicle based on the camera shooting, the output non-overturned accident vehicle target information is measured by the camera target size and the radar target confidence and RCS to obtain the target's danger factor. If the target danger factor is higher than the threshold set in the warning area, it is judged as a warning target, and the perception system sends the warning target's location and status information to the downstream decision domain control system. If the target danger factor is lower than the threshold set in the warning area, the perception system determines it as a non-warning target and ignores it directly. The target detection strategy process in the warning area is as follows: Figure 5 The target risk factor calculation method in the warning target area is as shown in Formula 3 in the above embodiment. No further details will be given here.

[0103] Combination Figure 3As shown, in one embodiment of the present invention, the vehicle target detection method also includes: when there is a target in the warning target area, obtaining a danger coefficient of the target based on the scanning data of the radar, wherein the warning target area is an area farther away from the vehicle than the warning target area; when the target is judged to be a warning target based on the danger coefficient, determining the status information and position information of the accident vehicle.

[0104] Specifically, if Figure 6 As shown in the figure, when the perception system detects that there is a target in the warning area, the target is beyond the detection range of the camera and is directly detected by the forward radar and the forward angle radar. The target in this area poses little danger to the main vehicle and only affects the driving plan after a period of time. The perception system only needs to pay attention to the target with a higher risk factor. The target with a lower risk factor does not affect the vehicle's path planning.

[0105] The three forward millimeter-wave radars fuse the confidence level of the detected target, the radar backscatter cross-section (RCS) and the size measured by the radar. The danger factor of the obstacle is determined by the above three parameters. If the danger factor is high and exceeds the danger factor threshold set in the warning area, the perception system will identify the target as a warning target in the warning area and send it to the downstream decision-making domain control unit; if the danger factor is lower than the danger factor threshold set in the warning area, the perception system will identify the target as a non-warning target and discard it directly, which will reduce the burden on the central computing unit.

[0106] The target risk factor calculation formula for the warning area is:

[0107] THQ mmr = λ1 * L con + λ2 * RCS +λ3 *(L mmr *W mmr ) (4)

[0108] Among them, λ1, λ2, and λ3 represent the radar target confidence, the target radar backscattering cross-sectional area, and the length and width influence factors of the object, respectively. mmr Represents the target length detected by the radar, W mmr Represents the width of the target detected by the radar. That is, the danger factor of the target is obtained according to the scanning data of the radar, which can be obtained by the above formula 4.

[0109] The vehicle target detection method of the embodiment of the present invention is as follows: Figure 7As shown in the figure, the rollover accident vehicle appears in the perception range of the perception system, and the perception system determines the area where the target is located. The first step is to determine whether it is a dangerous area. If it is in a dangerous area, the forward millimeter-wave radar, corner radar and forward-looking camera are integrated for detection, and the target is continuously monitored and tracked according to the dangerous target detection strategy; if it is not within the dangerous area, it will be determined whether the target is in the warning area or the alert area. The perception system determines whether the target is in the warning range based on the detection range of the camera. If it is in the warning range, the forward millimeter-wave radar, corner radar and forward-looking camera are integrated for perception. The perception system detects and identifies the danger of the target according to the warning target detection strategy, and makes a judgment on whether to track it. If the position of the target is not within the perception range of the camera, the forward millimeter-wave radar, corner radar, and forward-looking camera are integrated for perception. The system identifies the danger of the target according to the warning target strategy and determines whether it is a valuable warning target.

[0110] According to the vehicle target detection method of the embodiment of the present invention, when there is a target in the dangerous target area, based on the camera's perception data, if the target is preliminarily identified as an accident vehicle, the radar's scanning data and the camera's perception data can be used to determine the state information and location information of the accident vehicle. If the preliminarily identified target is another type of obstacle other than the accident vehicle, the scanning data and the camera's perception data can be fused to obtain the target's risk factor. Otherwise, the target's risk factor is obtained based on the scanning data. Finally, when the target is judged to be a dangerous target based on the risk factor, the state information and location information of the accident vehicle can be determined. As a result, the recognition speed of accident vehicles (such as side vehicles) is fast, accurate and reliable, ensuring the intelligent driving safety of the vehicle.

[0111] In one embodiment of the present invention, the vehicle target detection method further includes: providing a collection interface to receive sample data through the collection interface, wherein the source of the sample data includes on-site photographed images and network downloaded images; inputting the sample data into an initial recognition model, and training the initial recognition model according to the loss between the output of the initial recognition model and the label to obtain the recognition model. In this way, the recognition model can be continuously optimized to have a more accurate rollover vehicle recognition capability.

[0112] Specifically, in response to the problem that it is difficult to establish a rollover vehicle data set in the prior art, the present invention proposes a rollover vehicle data set construction plan based on collecting pictures from the Internet and a rollover vehicle detection algorithm upgrade plan based on real vehicle collection.

[0113] At present, intelligent cameras mainly rely on intelligent detection algorithms based on neural networks to detect and identify targets. The ability of intelligent detection algorithms to detect rollover vehicles depends on the richness of the rollover vehicle data set. The rollover vehicle data set includes training sets and test sets. The target recognition ability of the neural network-based detection model in the inference stage depends on the model parameters obtained by model training in the training stage, and the parameters obtained by the algorithm model depend on the coverage of the training data set. The more complete the data categories contained in the training data set, the stronger the detection ability of the algorithm model.

[0114] like Figure 8 As shown in the figure, given that it is difficult to collect images of rollover accident vehicles, images of rollover accident scenes can be obtained through two scene data collection methods: network collection and real vehicle collection. The database established by collecting images of rollover accident vehicles on the Internet can only cover some specific scenes and vehicle types, and the intelligent detection algorithm can be trained to achieve preliminary detection and identification of rollover accident vehicles; while the database established by collecting images of rollover accident vehicles on real vehicles is a process of continuous iteration and optimization. As the number of collected data increases, the detection capability of the trained intelligent detection algorithm model continues to improve.

[0115] like Fig. 9 As shown in the figure, we collect online images of overturned vehicles. This step mainly involves collecting on-site images of overturned accident vehicles on the Internet. The advantages of collecting images of overturned accident vehicles on the Internet are: photos of overturned accident vehicles in multiple regions or countries and multiple scenes can be obtained, and the scene categories are rich; a large number of images can be collected in a short time, greatly shortening the time for obtaining images. At the same time, as long as there is a computer connected to the Internet, the cost of image collection can be saved. There are also certain defects in the images collected on the Internet, such as the absence of continuous frames in the collected images, the single shooting angle, the different parameters of the camera used, etc., which bring difficulties to the task of establishing standard training sets and test sets.

[0116] In order to unify the image size of rollover vehicles and meet the input size requirements of the intelligent detection algorithm model based on neural networks, it is necessary to crop and fill the photos of rollover accident vehicles collected through the Internet. The images of rollover accident vehicles collected through the Internet are taken with different cameras, and the resolution parameters are difficult to unify, which is not easy to meet the input requirements of the intelligent detection algorithm model. For this reason, a data cropping and filling module is designed. For modules with higher resolution, the cropping module is used to perform pixel transformation and size transformation on the data to obtain standard rollover accident vehicle image data. For images with lower resolution, the filling module image data is used to fill pixels to improve the image resolution to meet the input requirements of the intelligent detection algorithm model.

[0117] The images of rollover accident vehicles collected from the Internet generally have a single angle. In order to enrich the scene information of the images of rollover accident vehicles, data generalization can be used to change the background of the rollover accident vehicle, or change the angle of the rollover accident vehicle, etc. to obtain more training or test images.

[0118] During the training phase of the intelligent detection algorithm model, the recognition results of the model are compared with the true value to verify the detection accuracy that can be achieved by the training model parameters. If the model recognition result is too far from the true value, then the model parameters are corrected to make the recognition result of the intelligent detection algorithm model closer to the true value, and the training is repeated until the recognition accuracy of the intelligent detection algorithm model meets the requirements. The true value of the picture needs to be annotated in advance, and data annotation provides a reference for model learning. The annotation of rollover accident vehicle pictures is completed through the annotation module, and the implementation method of annotation is mainly achieved through manual annotation.

[0119] The establishment of training and test sets, the labeled rollover accident vehicle images can be divided into training and test sets according to a certain ratio. The training data is used to tune the model parameters in the training process of the intelligent detection algorithm model. After the model parameters are finalized, they are used in the intelligent detection algorithm model to detect and identify rollover accident vehicles in the inference stage. The test data is used to verify the detection and identification accuracy of the intelligent detection algorithm model.

[0120] The rollover vehicle detection algorithm based on real vehicle data collection is being upgraded. Fig.10 As shown in the figure, real vehicle acquisition can store data in a set format, or store continuous frame data of rollover accident vehicles, and can be shot from multiple angles, which can provide the most practical road scene data for training and testing for intelligent detection algorithm models. However, real vehicle acquisition has high cost and long acquisition cycle, mainly because rollover accident scenes are difficult to encounter in actual road scenes.

[0121] To this end, a continuous iteration plan is designed. In the construction plan, a data acquisition module and a data upload interface for transmitting data are designed in the intelligent driving domain controller. Considering that the original scene data of the sensor may occupy a large transmission bandwidth, the acquisition format requirements should be designed in the data acquisition module part to minimize the data transmission bandwidth while meeting the input requirements of the intelligent detection algorithm model.

[0122] Special scene data collection nodes are deployed in the cloud to collect special scene data. The collected data includes not only the image information of the rollover accident vehicle, but also the detection information of sensors such as millimeter-wave radar, which is convenient for later annotation. Driver-operable devices are set up on the vehicle-machine side to trigger the special scene data collection and upload functions. When the intelligent driving domain controller equipped with special scene data collection function encounters a rollover vehicle accident scene, the driver triggers the scene save button, and the intelligent perception system starts the data saving function within a specific time period, obtains the rollover vehicle accident scene data information from the sensor side, and uploads it to the cloud in a pre-set format.

[0123] At the center, data aggregation, data annotation, and improvement of the rollover accident vehicle training set are completed, and the training of the intelligent detection algorithm model is completed; each data collection node in the cloud aggregates the special scene data, and the system screens the aggregated rollover vehicle image information, radar information and other data, selects data with training value, and then aligns the data time. When it is judged that the aggregated rollover accident vehicle data reaches a certain amount of preset threshold, the system deploys the data annotation module to complete the annotation of special scene data. The information of sensors such as radar is conducive to providing true values ​​in the annotation stage. After that, the construction of the training set and the test set is completed, and the optimization parameters of the intelligent detection algorithm model are obtained after the training set and the test set. Finally, the intelligent detection algorithm model in the intelligent driving domain controller is upgraded through the Over-the-Air Technology (OTA).

[0124] Fig.11 FIG. 1 is a block diagram of a vehicle target detection system according to an embodiment of the present application. Fig.11 As shown, the target detection system for a vehicle according to an embodiment of the present application includes: a preliminary judgment module 1110, a determination module 1120 and a detection module 1130, wherein:

[0125] A preliminary judgment module 1110 is used to preliminarily identify whether a target is an accident vehicle based on the perception data of the camera when there is a target in the dangerous target area;

[0126] The determination module 1120 is used to determine the state information and position information of the accident vehicle based on the scanning data of the radar and the perception data of the camera when the target is initially identified as an accident vehicle, and to obtain the danger coefficient of the target based on the scanning data and the perception data of the camera when the target is initially identified as an obstacle of another type other than the accident vehicle; otherwise, to obtain the danger coefficient of the target based on the scanning data;

[0127] The detection module 1130 is used to determine the state information and position information of the accident vehicle when the target is judged to be a dangerous target according to the risk coefficient.

[0128] According to the vehicle target detection system of the embodiment of the present invention, when there is a target in the dangerous target area, based on the camera's perception data, if the target is preliminarily identified as an accident vehicle, the radar scanning data and the camera's perception data can be used to determine the state information and location information of the accident vehicle. If the preliminarily identified target is another type of obstacle other than the accident vehicle, the scanning data and the camera's perception data can be fused to obtain the target's risk factor. Otherwise, the target's risk factor is obtained based on the scanning data. Finally, when the target is judged to be a dangerous target based on the risk factor, the state information and location information of the accident vehicle can be determined. As a result, the system has the advantages of fast, accurate and reliable recognition of accident vehicles (such as side vehicles), ensuring the intelligent driving safety of the vehicle.

[0129] It should be noted that the specific implementation method of the target detection system of the vehicle in the embodiment of the present application is similar to the specific implementation method of the target detection method of the vehicle in the embodiment of the present application. Please refer to the description of the method part for details, and no further details will be given here.

[0130] Furthermore, an embodiment of the present application provides a vehicle, comprising: a vehicle target detection system according to the above embodiment. The vehicle has the advantages of fast, accurate and reliable recognition of accident vehicles (such as side vehicles), ensuring the intelligent driving safety of the vehicle.

[0131] Reference below Fig.12 , Fig.12 A schematic diagram of the structure of a computing device suitable for implementing an embodiment of the present application is shown.

[0132] like Fig.12 As shown, the computer system includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 into the random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation instructions of the system are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0133] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0134] In particular, according to an embodiment of the present application, the above reference flow chart Figure 1 The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above-mentioned functions defined in the system of the present application are executed.

[0135] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0136] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operating instructions of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operating instruction, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0138] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the computing device described in the above embodiment, or may exist independently without being assembled into the computing device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the vehicle target detection method described in the present application.

[0139] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A vehicle target detection method, characterized in that: The vehicle is provided with a radar and a camera, and the target detection method comprises: When there is a target in the dangerous target area, preliminarily identifying whether the target is an accident vehicle based on the perception data of the camera; If yes, determining the state information and position information of the accident vehicle based on the scanning data of the radar and the perception data of the camera; If the target is initially identified as an obstacle of another type other than the accident vehicle, a risk factor of the target is obtained according to the scanning data and the perception data of the camera; otherwise, a risk factor of the target is obtained according to the scanning data; When the target is judged as a dangerous target according to the risk coefficient, the state information and position information of the accident vehicle are determined.

2. The vehicle target detection method according to claim 1, characterized in that: When the target is judged as a dangerous target according to the risk coefficient, the state information and position information of the accident vehicle are determined, including: Determining whether the risk factor is higher than a preset risk factor threshold; If yes, the target is determined to be a dangerous target, and the state information and position information of the accident vehicle are obtained; otherwise, a prompt for a non-dangerous target is triggered.

3. The vehicle target detection method according to claim 1, characterized in that: Also includes: When there is a target in the warning target area, preliminarily identifying whether the target is an accident vehicle based on the perception data of the camera, the warning target area is an area farther away from the vehicle than the dangerous target area; If yes, trigger a warning prompt according to the scanning data of the radar and the perception data of the camera; otherwise, obtain the danger factor of the target according to the scanning data and the perception data of the camera; When the target is judged as a warning target according to the risk factor, the state information and position information of the target are determined.

4. The vehicle target detection method according to claim 3, characterized in that: When the target is judged as a warning target according to the risk factor, the state information and position information of the target are determined, including: Determining whether the risk factor is higher than a preset warning factor threshold; If so, it is determined that the target is a warning target, and the state information and position information of the target are obtained; otherwise, a prompt of a non-warning target is triggered.

5. The vehicle target detection method according to claim 1, characterized in that: Also includes: When there is a target in the warning target area, obtaining a danger factor of the target according to the scanning data of the radar, wherein the warning target area is an area farther from the vehicle than the warning target area; When the target is judged as a warning target according to the risk factor, the state information and position information of the accident vehicle are determined.

6. The vehicle target detection method according to claim 1, characterized in that: The step of obtaining the risk factor of the target according to the scanning data comprises: The target confidence L is obtained by the following formula: con , the formula is: THE con =k*e Tn / m Among them, T n is the life cycle of the target, k is the impact factor of the life cycle of the target, and m is the threshold value of the target’s life cycle; The target's risk factor THQ mmr It is obtained by the following formula, which is: THQ mmr =λ1*L con +λ2*RCS Wherein, RCS is the backscattering cross-sectional area of ​​the target detected by the radar, and λ1 and λ2 are the influencing factors of the target confidence and the backscattering cross-sectional area of ​​the target, respectively.

7. The vehicle target detection method according to any one of claims 1 to 4, characterized in that: The step of obtaining a risk factor of a target according to the scanning data and the perception data of the camera includes: The hazard factor THQ of the target is obtained by the following formula fusion , the formula is: THQ mmr =λ1*L con +λ2*RCS+λ3*(L*W*H) Among them, λ1, λ2, and λ3 are the influencing factors of target confidence, target backscattering cross-sectional area, and target length, width, and height, respectively. L is the target length, W is the target width, and H is the target height.

8. The vehicle target detection method according to claim 5, characterized in that: The step of obtaining the danger factor of the target according to the scanning data of the radar comprises: The hazard factor THQ of the target is obtained by the following formula mmr , the formula is: THQ mmr =λ1*L con +λ2*RCS+λ3*(L mmr *W mmr ) Among them, λ1, λ2, and λ3 are the influencing factors of target confidence, target backscattering cross-sectional area, and target length and width, respectively. mmr is the target length detected by the radar, W mmr is the width of the target detected by the radar.

9. The vehicle target detection method according to claim 1, characterized in that: The preliminarily identifying whether the target is an accident vehicle based on the perception data of the camera includes: The perception data of the camera is input into a recognition model to obtain a result of the recognition model identifying whether the target is an accident vehicle, wherein the perception data includes image information of the target.

10. The vehicle target detection method according to claim 9, characterized in that: Also includes: Providing a collection interface to receive sample data through the collection interface, wherein the source of the sample data includes on-site photographed images and network downloaded images; The sample data is input into an initial recognition model, and the initial recognition model is trained according to the loss between the output of the initial recognition model and the label to obtain the recognition model.

11. A vehicle target detection system, characterized in that: The vehicle is equipped with a radar and a camera, and the target detection system includes: A preliminary judgment module is used to preliminarily identify whether a target is an accident vehicle based on the perception data of the camera when there is a target in the dangerous target area; a determination module, for determining the state information and position information of the accident vehicle based on the scanning data of the radar and the perception data of the camera when the target is initially identified as an accident vehicle, and for obtaining the danger coefficient of the target based on the scanning data and the perception data of the camera when the target is initially identified as an obstacle of another type other than the accident vehicle; otherwise, obtaining the danger coefficient of the target based on the scanning data; The detection module is used to determine the state information and position information of the accident vehicle when the target is judged to be a dangerous target according to the risk coefficient.

12. A vehicle, characterized in that: include: The target detection system according to claim 11.

13. A computing device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target detection method for a vehicle according to any one of claims 1 to 10 is implemented.

Images