Obstacle mismatching relation removing method and device, equipment and storage medium
By partitioning and matching obstacles, identifying and resolving the mismatch relationship, the problem of mismatch in obstacles in autonomous driving is solved, and safety and identification accuracy are improved.
Patent Information
- Application Number
- CN202410966310.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-12-06
AI Technical Summary
In the existing autonomous driving technology, the rear fusion matching solution of millimeter-wave radar and camera fails to effectively consider the logical matching of obstacles, resulting in mismatch problems and affecting the safety of autonomous driving.
By obtaining lane line detection results, partition the obstacles detected by different sensors, identify and unmatched obstacles in the matching list until there is no mismatch. The nearest neighbor search principle and preset matching conditions are used to match. The horizontal distance difference and type are used to identify the mismatch.
It avoids errors in fusion results caused by mismatch, improves the safety of autonomous driving and the accuracy of fusion results of radar and camera obstacle identification, and improves the speed and efficiency of unblocking the relationship between obstacles.
Smart Images

Figure CN119091403A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method, device, equipment and storage medium for resolving an obstacle mismatch relationship. Background Art
[0002] The existing post-fusion matching scheme of millimeter-wave radar and camera only considers various physical quantities (horizontal and vertical coordinates, horizontal and vertical speeds or horizontal and vertical accelerations, etc.) of obstacles detected by millimeter-wave radar and camera, but does not consider whether the two obstacles should really be matched logically. For example, the camera obstacle on the lane and the millimeter-wave radar obstacle on the curb may be very close, but the possibility of these two obstacles being matched together should be much lower than that of two obstacles with similar distances on the lane at the same time. However, the existing technology handles the matching in these two cases, and the matching possibility is consistent.
[0003] The existing post-fusion technology only uses the relationship between obstacles to perform matching and correlation calculations, without considering the surrounding environment of various obstacles. The disadvantage of this is that there is a high probability of mismatching, especially when the sensor has false detections. For example, when the camera occasionally detects a moving obstacle on the lane as static, the existing matching method can easily match the camera results with the static obstacles detected by the millimeter-wave radar on the curb, resulting in mismatches and erroneous fusion results, affecting the safety of autonomous driving. Summary of the invention
[0004] The main purpose of the present invention is to provide a method, device, equipment and storage medium for resolving obstacle mismatching relationships, aiming to solve the technical problem in the prior art that post-fusion matching is prone to mismatching, leading to erroneous fusion results and affecting the safety of autonomous driving.
[0005] In a first aspect, the present invention provides a method for removing an obstacle mismatching relationship, the method comprising the following steps:
[0006] Obtaining a lane line detection result of a lane to be detected, and partitioning obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions;
[0007] Perform obstacle matching on obstacles in different partitions to obtain a matching list;
[0008] Identify mismatched obstacles in the matching list, and release the matching relationship of the mismatched obstacles until no mismatches exist in the matching list.
[0009] Optionally, the acquiring a lane line detection result of a lane to be detected, and partitioning obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions, includes:
[0010] Obtain a lane line detection result of a lane to be detected, partition obstacles detected by different sensors according to the lane line detection result, and obtain obstacles on the lane and obstacles outside the lane after partitioning.
[0011] Optionally, performing obstacle matching on obstacles in different partitions to obtain a matching list includes:
[0012] Obtaining a millimeter-wave radar detection result, and setting a static obstacle near the leftmost lane line and the rightmost lane line as a curb-type obstacle according to the millimeter-wave radar detection result;
[0013] Searching for the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result closest to the obstacle of the edge type according to the nearest neighbor search principle;
[0014] The obstacle recognition result of the millimeter-wave radar and the obstacle recognition result of the camera are matched to obtain a matching list.
[0015] Optionally, matching the obstacle recognition result of the millimeter wave radar with the obstacle recognition result of the camera to obtain a matching list includes:
[0016] The millimeter-wave radar obstacle recognition result and the camera obstacle recognition result are matched according to a preset matching condition to obtain a matching list of the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result.
[0017] Optionally, the identifying the mismatched obstacle in the matching list and releasing the matching relationship of the mismatched obstacle until no mismatch exists in the matching list includes:
[0018] Traversing the matching list, obtaining a lateral distance difference between a lateral distance of a target millimeter-wave radar obstacle and a lateral distance of a target camera obstacle that are matched in pairs in the matching list;
[0019] Determine whether the lateral distance difference is greater than a preset width, and identify mismatched obstacles in the matching list according to the difference determination result;
[0020] The matching relationship of the mismatched obstacle is released until there is no mismatch in the matching list.
[0021] Optionally, the determining whether the lateral distance difference is greater than a preset width, and identifying the mismatched obstacle in the matching list according to the difference determination result, includes:
[0022] Determine whether the lateral distance difference is greater than a preset width, and obtain a difference determination result;
[0023] When the difference judgment result is that the lateral distance difference is not greater than the preset width, retaining the current matching relationship;
[0024] When the difference judgment result is that the lateral distance difference is greater than the preset width, judging whether the type of the target millimeter-wave radar obstacle is a curb type, and obtaining a type judgment result;
[0025] The mismatched obstacles in the matching list are identified according to the type determination result.
[0026] Optionally, the identifying the mismatched obstacles in the matching list according to the type determination result includes:
[0027] When the type judgment result is that the type of the target millimeter-wave radar obstacle is not a curb type, retaining the current matching relationship;
[0028] When the type determination result is that the type of the target millimeter-wave radar obstacle is a curb type, determining whether the type of the target camera obstacle is a curb type;
[0029] When the type of the target camera obstacle is not a curb type, retaining the current matching relationship;
[0030] When the type of the target camera obstacle is a curb type, determining whether the longitudinal speed of the target camera obstacle is less than a preset speed threshold;
[0031] When the longitudinal speed is less than the preset speed threshold, identifying a mismatched obstacle in the matching list and releasing a current matching relationship of the mismatched obstacle;
[0032] When the longitudinal speed is not less than the preset speed threshold, the current matching relationship is retained, and new mismatched obstacles in the matching list are continuously identified until no mismatches exist in the matching list.
[0033] In a second aspect, in order to achieve the above-mentioned object, the present invention further provides a device for removing a mismatching relationship of an obstacle, the device comprising:
[0034] A partitioning module, used to obtain a lane line detection result of a lane to be detected, and partition obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions;
[0035] The obstacle matching module is used to match obstacles in different partitions and obtain a matching list;
[0036] The release module is used to identify the mismatched obstacles in the matching list and release the matching relationship of the mismatched obstacles until there is no mismatch in the matching list.
[0037] In the third aspect, in order to achieve the above-mentioned purpose, the present invention also proposes a device for resolving an obstacle mismatching relationship, wherein the device comprises: a memory, a processor, and an obstacle mismatching relationship resolving program stored in the memory and executable on the processor, wherein the obstacle mismatching relationship resolving program is configured to implement the steps of the obstacle mismatching relationship resolving method as described above.
[0038] In a fourth aspect, in order to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which is stored a program for resolving an obstacle mismatching relationship, and when the program for resolving an obstacle mismatching relationship is executed by a processor, the steps of the method for resolving an obstacle mismatching relationship as described above are implemented.
[0039] The method for removing the mismatched obstacle relationship proposed by the present invention obtains the lane line detection result of the lane to be detected, and divides the obstacles detected by different sensors into zones according to the lane line detection result to obtain obstacles in different zones; performs obstacle matching on the obstacles in different zones to obtain a matching list; identifies the mismatched obstacles in the matching list, and removes the matching relationship of the mismatched obstacles until no mismatch exists in the matching list; the generation of fusion result errors caused by mismatching can be avoided, and possible braking or automatic emergency braking caused by mismatching can be avoided, thereby improving the safety of autonomous driving, improving the accuracy of radar and camera obstacle recognition fusion results, and improving the speed and efficiency of removing the mismatched obstacle relationship. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;
[0041] Figure 2 It is a flow chart of a first embodiment of a method for resolving mismatched obstacle relationships of the present invention;
[0042] Figure 3 It is a flow chart of a second embodiment of the method for resolving mismatched obstacle relationships of the present invention;
[0043] Figure 4 It is a flowchart of a third embodiment of the method for resolving mismatched obstacle relationships of the present invention;
[0044] Figure 5 It is a flowchart of a fourth embodiment of the method for resolving mismatched obstacle relationships of the present invention;
[0045] Figure 6 It is a schematic diagram of an improved matching process of the method for resolving an obstacle mismatching relationship of the present invention;
[0046] Figure 7 It is a functional module diagram of the first embodiment of the obstacle mismatching relationship elimination device of the present invention.
[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0049] The solution of the embodiment of the present invention is mainly: by obtaining the lane line detection result of the lane to be detected, the obstacles detected by different sensors are divided into zones according to the lane line detection result to obtain obstacles in different zones; obstacle matching is performed on the obstacles in different zones to obtain a matching list; the mismatched obstacles in the matching list are identified, and the matching relationship of the mismatched obstacles is released until there is no mismatch in the matching list; it can avoid the generation of erroneous fusion results caused by mismatches, avoid possible braking or automatic emergency braking caused by mismatches, improve the safety of autonomous driving, improve the accuracy of radar and camera obstacle recognition fusion results, improve the speed and efficiency of releasing the mismatched obstacle relationship, and solve the technical problem in the prior art that post-fusion matching is prone to mismatching, resulting in erroneous fusion results and affecting the safety of autonomous driving.
[0050] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.
[0051] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or it may be a stable memory (Non-Volatile Memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.
[0052] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0053] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating device, a network communication module, a user interface module, and an obstacle mismatching relationship removal program.
[0054] The device of the present invention calls the obstacle mismatching relationship elimination program stored in the memory 1005 through the processor 1001, and performs the following operations:
[0055] Obtaining a lane line detection result of a lane to be detected, and partitioning obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions;
[0056] Perform obstacle matching on obstacles in different partitions to obtain a matching list;
[0057] Identify mismatched obstacles in the matching list, and release the matching relationship of the mismatched obstacles until no mismatches exist in the matching list.
[0058] The device of the present invention calls the obstacle mismatching relationship elimination program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0059] Obtain a lane line detection result of a lane to be detected, partition obstacles detected by different sensors according to the lane line detection result, and obtain obstacles on the lane and obstacles outside the lane after partitioning.
[0060] The device of the present invention calls the obstacle mismatching relationship elimination program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0061] Obtaining a millimeter-wave radar detection result, and setting a static obstacle near the leftmost lane line and the rightmost lane line as a curb-type obstacle according to the millimeter-wave radar detection result;
[0062] Searching for the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result closest to the obstacle of the edge type according to the nearest neighbor search principle;
[0063] The obstacle recognition result of the millimeter-wave radar and the obstacle recognition result of the camera are matched to obtain a matching list.
[0064] The device of the present invention calls the obstacle mismatching relationship elimination program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0065] The millimeter-wave radar obstacle recognition result and the camera obstacle recognition result are matched according to a preset matching condition to obtain a matching list of the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result.
[0066] The device of the present invention calls the obstacle mismatching relationship elimination program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0067] Traversing the matching list, obtaining a lateral distance difference between a lateral distance of a target millimeter-wave radar obstacle and a lateral distance of a target camera obstacle that are matched in pairs in the matching list;
[0068] Determine whether the lateral distance difference is greater than a preset width, and identify mismatched obstacles in the matching list according to the difference determination result;
[0069] The matching relationship of the mismatched obstacle is released until there is no mismatch in the matching list.
[0070] The device of the present invention calls the obstacle mismatching relationship elimination program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0071] Determine whether the lateral distance difference is greater than a preset width, and obtain a difference determination result;
[0072] When the difference judgment result is that the lateral distance difference is not greater than the preset width, retaining the current matching relationship;
[0073] When the difference judgment result is that the lateral distance difference is greater than the preset width, judging whether the type of the target millimeter-wave radar obstacle is a curb type, and obtaining a type judgment result;
[0074] The mismatched obstacles in the matching list are identified according to the type determination result.
[0075] The device of the present invention calls the obstacle mismatching relationship elimination program stored in the memory 1005 through the processor 1001, and also performs the following operations:
[0076] When the type judgment result is that the type of the target millimeter-wave radar obstacle is not a curb type, retaining the current matching relationship;
[0077] When the type determination result is that the type of the target millimeter-wave radar obstacle is a curb type, determining whether the type of the target camera obstacle is a curb type;
[0078] When the type of the target camera obstacle is not a curb type, retaining the current matching relationship;
[0079] When the type of the target camera obstacle is a curb type, determining whether the longitudinal speed of the target camera obstacle is less than a preset speed threshold;
[0080] When the longitudinal speed is less than the preset speed threshold, identifying a mismatched obstacle in the matching list and releasing a current matching relationship of the mismatched obstacle;
[0081] When the longitudinal speed is not less than the preset speed threshold, the current matching relationship is retained, and new mismatched obstacles in the matching list are continuously identified until no mismatches exist in the matching list.
[0082] Through the above scheme, this embodiment obtains the lane line detection result of the lane to be detected, divides the obstacles detected by different sensors into zones according to the lane line detection result, and obtains obstacles in different zones; performs obstacle matching on the obstacles in different zones to obtain a matching list; identifies the mismatched obstacles in the matching list, and releases the matching relationship of the mismatched obstacles until there is no mismatch in the matching list; thus, it is possible to avoid the generation of fusion result errors caused by mismatches, avoid possible braking or automatic emergency braking caused by mismatches, improve the safety of autonomous driving, improve the accuracy of radar and camera obstacle recognition fusion results, and improve the speed and efficiency of releasing the mismatched obstacle relationship.
[0083] Based on the above hardware structure, an embodiment of a method for resolving an obstacle mismatching relationship of the present invention is proposed.
[0084] Reference Figure 2 , Figure 2 It is a flowchart diagram of the first embodiment of the method for resolving the mismatched obstacle relationship of the present invention.
[0085] In a first embodiment, the method for resolving an obstacle mismatching relationship comprises the following steps:
[0086] Step S10: Obtain a lane line detection result of a lane to be detected, and partition obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions.
[0087] It should be noted that the lane to be detected is the lane where the current autonomous driving vehicle is traveling and requires obstacle detection. After detecting the lane to be detected, a lane line detection result can be obtained. According to the lane line detection result, the obstacles detected by different sensors are partitioned to obtain obstacles in different partitions.
[0088] Step S20: Obstacle matching is performed on obstacles in different partitions to obtain a matching list.
[0089] It should be understood that after obstacle matching is performed on different partitions, a corresponding matching list can be obtained.
[0090] Step S30: Identify mismatched obstacles in the matching list, and release the matching relationship of the mismatched obstacles until there is no mismatch in the matching list.
[0091] It can be understood that by identifying the matching list, the mismatched obstacles in the matching list can be obtained, and then the matching relationship of the mismatched obstacles can be released until there is no mismatch in the matching list.
[0092] Through the above scheme, this embodiment obtains the lane line detection result of the lane to be detected, divides the obstacles detected by different sensors into zones according to the lane line detection result, and obtains obstacles in different zones; performs obstacle matching on the obstacles in different zones to obtain a matching list; identifies the mismatched obstacles in the matching list, and releases the matching relationship of the mismatched obstacles until there is no mismatch in the matching list; thus, it is possible to avoid the generation of fusion result errors caused by mismatches, avoid possible braking or automatic emergency braking caused by mismatches, improve the safety of autonomous driving, improve the accuracy of radar and camera obstacle recognition fusion results, and improve the speed and efficiency of releasing the mismatched obstacle relationship.
[0093] Further, Figure 3 FIG. 2 is a flow chart of a second embodiment of a method for removing mismatched obstacles according to the present invention. Figure 3 As shown, a second embodiment of the method for resolving the mismatched obstacle relationship of the present invention is proposed based on the first embodiment. In this embodiment, the step S10 specifically includes the following steps:
[0094] Step S11: Obtain a lane line detection result of a lane to be detected, and partition obstacles detected by different sensors according to the lane line detection result to obtain obstacles on the lane and obstacles outside the lane after partitioning.
[0095] It should be noted that after obtaining the lane line detection result of the lane to be detected, the lane line detection result is introduced. Based on the lane line, the obstacles detected by different sensors can be segmented or partitioned. The partition is mainly divided into on-lane and outside-lane. The obstacles are divided into on-lane (in-lane) obstacles and outside-lane obstacles, so that the possibility of matching obstacles on the lane with obstacles outside the lane is reduced.
[0096] Through the above scheme, this embodiment obtains the lane line detection result of the lane to be detected, partitions the obstacles detected by different sensors according to the lane line detection result, obtains the obstacles on the lane and obstacles outside the lane after partitioning, and can quickly distinguish obstacles, thereby improving the safety of autonomous driving.
[0097] Further, Figure 4 FIG. 1 is a flow chart of a third embodiment of a method for resolving mismatched obstacle relationships according to the present invention. Figure 4 As shown, a third embodiment of the method for resolving the mismatched obstacle relationship of the present invention is proposed based on the first embodiment. In this embodiment, step S20 specifically includes the following steps:
[0098] Step S21: Obtain the millimeter-wave radar detection result, and set the static obstacles near the leftmost lane line and the rightmost lane line as curb-type obstacles according to the millimeter-wave radar detection result.
[0099] It should be noted that before performing obstacle matching in post-fusion, lane lines can be introduced to distinguish the types of millimeter-wave radar results, and the static obstacles detected by the millimeter-wave radar near the leftmost and rightmost lane lines can be set as curb types.
[0100] Step S22: searching for the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result closest to the edge type obstacle according to the nearest neighbor search principle.
[0101] It can be understood that, according to the nearest neighbor search principle, the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result closest to the type of obstacle can be searched out.
[0102] Step S23: Match the obstacle recognition result of the millimeter-wave radar with the obstacle recognition result of the camera to obtain a matching list.
[0103] It should be understood that by matching the obstacle recognition result of the millimeter wave radar with the obstacle recognition result of the camera, a matching list can be obtained.
[0104] Further, the step S23 specifically includes the following steps:
[0105] The millimeter-wave radar obstacle recognition result and the camera obstacle recognition result are matched according to a preset matching condition to obtain a matching list of the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result.
[0106] In a specific implementation, the millimeter-wave radar obstacle results and the camera obstacle results are matched according to certain conditions according to a nearest neighbor search scheme to form a list of matching millimeter-wave radar obstacles and camera obstacles.
[0107] Through the above scheme, this embodiment obtains the millimeter-wave radar detection result, and sets the static obstacles near the leftmost lane line and the rightmost lane line as curb-type obstacles according to the millimeter-wave radar detection result; searches for the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result closest to the curb-type obstacle according to the nearest search principle; matches the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result to obtain a matching list, which can quickly obtain the matching list, avoid the generation of fusion result errors caused by mismatching, avoid possible braking or automatic emergency braking caused by mismatching, and improve the safety of autonomous driving.
[0108] Further, Figure 5 FIG. 4 is a flow chart of a fourth embodiment of a method for removing an obstacle mismatching relationship according to the present invention. Figure 5 As shown, a fourth embodiment of the method for resolving the mismatched obstacle relationship of the present invention is proposed based on the first embodiment. In this embodiment, step S30 specifically includes the following steps:
[0109] Step S31: traverse the matching list to obtain the lateral distance difference between the lateral distance of the target millimeter-wave radar obstacle and the lateral distance of the target camera obstacle that are matched in pairs in the matching list.
[0110] It should be noted that after traversing the matching list, the lateral distance of the target millimeter-wave radar obstacle and the lateral distance of the target camera obstacle that are matched in pairs in the matching list can be obtained, and then the difference between the two lateral distances can be calculated.
[0111] Step S32: determine whether the lateral distance difference is greater than a preset width, and identify mismatched obstacles in the matching list according to the difference determination result.
[0112] It can be understood that the preset width is a pre-set distance value. Taking the preset width of half a lane width as an example, after traversing the list of millimeter-wave radar obstacles and camera obstacles matching, it is possible to check whether the difference between the lateral distance of the paired matching millimeter-wave radar obstacles and the lateral distance of the camera obstacles in the list is greater than half a lane width, and then obtain the corresponding difference judgment result to identify the mismatched obstacles in the matching list.
[0113] Furthermore, the step S32 specifically includes the following steps:
[0114] Determine whether the lateral distance difference is greater than a preset width, and obtain a difference determination result;
[0115] When the difference judgment result is that the lateral distance difference is not greater than the preset width, retaining the current matching relationship;
[0116] When the difference judgment result is that the lateral distance difference is greater than the preset width, judging whether the type of the target millimeter-wave radar obstacle is a curb type, and obtaining a type judgment result;
[0117] The mismatched obstacles in the matching list are identified according to the type determination result.
[0118] It should be understood that when the difference judgment result is that the lateral distance difference is not greater than the preset width, the current matching relationship is retained; otherwise, it is determined whether the type of the target millimeter-wave radar obstacle is a curb type, and a type judgment result is obtained, and then the mismatched obstacles in the matching list are identified according to the type judgment result.
[0119] Furthermore, the step of identifying mismatched obstacles in the matching list according to the type judgment result specifically includes the following steps:
[0120] When the type judgment result is that the type of the target millimeter-wave radar obstacle is not a curb type, retaining the current matching relationship;
[0121] When the type determination result is that the type of the target millimeter-wave radar obstacle is a curb type, determining whether the type of the target camera obstacle is a curb type;
[0122] When the type of the target camera obstacle is not a curb type, retaining the current matching relationship;
[0123] When the type of the target camera obstacle is a curb type, determining whether the longitudinal speed of the target camera obstacle is less than a preset speed threshold;
[0124] When the longitudinal speed is less than the preset speed threshold, identifying a mismatched obstacle in the matching list and releasing a current matching relationship of the mismatched obstacle;
[0125] When the longitudinal speed is not less than the preset speed threshold, the current matching relationship is retained, and new mismatched obstacles in the matching list are continuously identified until no mismatches exist in the matching list.
[0126] It can be understood that when the type judgment result is that the type of the target millimeter-wave radar obstacle is not a curb type, the current matching relationship is retained; otherwise, it is determined whether the type of the target camera obstacle is a curb type, and further, when the type of the target millimeter-wave radar obstacle is a curb type, it is determined that when the type of the target camera obstacle is not a curb type, the current matching relationship is retained, and it is determined whether the longitudinal speed of the target camera obstacle is less than a preset speed threshold; further, when the longitudinal speed is less than the preset speed threshold, the mismatched obstacle in the matching list is identified, and the current matching relationship of the mismatched obstacle is released; otherwise, the current matching relationship is retained, and new mismatched obstacles in the matching list continue to be identified until there is no mismatch in the matching list.
[0127] In the specific implementation, further, Figure 6 FIG. 1 is a schematic diagram of an improved matching process of the method for removing the mismatched obstacle relationship of the present invention. Figure 6 As shown in the figure, 1. Before the post-fusion obstacle matching, we introduce lane lines to distinguish the types of millimeter-wave radar results, and set the static obstacles detected by the millimeter-wave radar near the leftmost and rightmost lane lines as the curb type; 2. According to the nearest neighbor search scheme, the millimeter-wave radar obstacle results and the camera obstacle results are matched according to certain conditions to form a list of millimeter-wave radar obstacle and camera obstacle matches; 3. The list of millimeter-wave radar obstacle and camera obstacle matches is traversed to check whether the difference between the lateral distance of the paired matched millimeter-wave radar obstacles and the lateral distance of the camera obstacles in the list is greater than half a lane width; 4. If the lateral distance difference is not greater than half a lane width, the matching relationship is still retained, and continue to step 3. If it is greater than half the lane width, execute step 5; 5. Check whether the radar obstacle type in the paired matched obstacles is the curb type. If not, keep the matching relationship and continue to step 3. If so, execute step 6; 6. Check whether the camera obstacle type in the paired matched obstacles is the curb type. If so, keep the matching relationship and continue to step 3. If not, execute step 7; 7. Determine whether the longitudinal speed of the camera obstacle is less than a certain value (such as 6m / s, of course, it can also be other values, which is not limited in this embodiment). If so, cancel the matching relationship. If not, keep the matching relationship and continue to step 3 until the traversal is completed. The above scheme is equivalent to patching the original matching scheme to eliminate possible mismatching conditions.
[0128] Step S33: cancel the matching relationship of the mismatched obstacle until there is no mismatch in the matching list.
[0129] It should be understood that the matching relationship of the mismatch obstacle may be released until there is no mismatch in the matching list, thereby eliminating the mismatch condition.
[0130] Through the above scheme, this embodiment traverses the matching list to obtain the lateral distance difference between the lateral distance of the target millimeter-wave radar obstacle and the lateral distance of the target camera obstacle that are matched in pairs in the matching list; determines whether the lateral distance difference is greater than a preset width, and identifies the mismatched obstacles in the matching list according to the difference judgment result; and releases the matching relationship of the mismatched obstacles until there is no mismatch in the matching list, thereby avoiding the generation of fusion result errors caused by mismatches, avoiding possible braking or automatic emergency braking caused by mismatches, improving the safety of autonomous driving, improving the accuracy of radar and camera obstacle recognition fusion results, and improving the speed and efficiency of releasing the mismatched obstacle relationship.
[0131] Accordingly, the present invention further provides a device for resolving an obstacle mismatching relationship.
[0132] Reference Figure 7 , Figure 7 It is a functional module diagram of the first embodiment of the obstacle mismatching relationship elimination device of the present invention.
[0133] In a first embodiment of the obstacle mismatching relation elimination device of the present invention, the obstacle mismatching relation elimination device comprises:
[0134] The partitioning module 10 is used to obtain a lane line detection result of a lane to be detected, and partition obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions.
[0135] The obstacle matching module 20 is used to perform obstacle matching on obstacles in different partitions to obtain a matching list.
[0136] The release module 30 is used to identify the mismatched obstacles in the matching list and release the matching relationship of the mismatched obstacles until there is no mismatch in the matching list.
[0137] The partitioning module 10 is further used to obtain a lane line detection result of a lane to be detected, and to partition obstacles detected by different sensors according to the lane line detection result to obtain obstacles on the lane and obstacles outside the lane after partitioning.
[0138] The obstacle matching module 20 is also used to obtain the millimeter-wave radar detection results, set the static obstacles near the leftmost lane line and the rightmost lane line as curb-type obstacles according to the millimeter-wave radar detection results; search for the millimeter-wave radar obstacle recognition results and the camera obstacle recognition results closest to the curb-type obstacles according to the nearest search principle; match the millimeter-wave radar obstacle recognition results and the camera obstacle recognition results to obtain a matching list.
[0139] The obstacle matching module 20 is further used to match the obstacle recognition result of the millimeter wave radar with the obstacle recognition result of the camera according to a preset matching condition, and obtain a matching list of the obstacle recognition result of the millimeter wave radar with the obstacle recognition result of the camera.
[0140] The release module 30 is further used to traverse the matching list to obtain the lateral distance difference between the lateral distance of the target millimeter-wave radar obstacle and the lateral distance of the target camera obstacle that are matched in pairs in the matching list; determine whether the lateral distance difference is greater than a preset width, and identify the mismatched obstacles in the matching list according to the difference judgment result; and release the matching relationship of the mismatched obstacles until there is no mismatch in the matching list.
[0141] The release module 30 is further used to determine whether the lateral distance difference is greater than a preset width, and obtain a difference judgment result; when the difference judgment result is that the lateral distance difference is not greater than the preset width, retain the current matching relationship; when the difference judgment result is that the lateral distance difference is greater than the preset width, determine whether the type of the target millimeter-wave radar obstacle is a curb type, and obtain a type judgment result; and identify the mismatched obstacles in the matching list according to the type judgment result.
[0142] The release module 30 is further configured to retain the current matching relationship when the type judgment result is that the type of the target millimeter-wave radar obstacle is not a curb type; determine whether the type of the target camera obstacle is a curb type when the type judgment result is that the type of the target millimeter-wave radar obstacle is a curb type; retain the current matching relationship when the type of the target camera obstacle is not a curb type; determine whether the longitudinal speed of the target camera obstacle is less than a preset speed threshold when the type of the target camera obstacle is a curb type; identify a mismatched obstacle in the matching list and release the current matching relationship of the mismatched obstacle when the longitudinal speed is less than the preset speed threshold; and retain the current matching relationship and continue to identify new mismatched obstacles in the matching list when the longitudinal speed is not less than the preset speed threshold until there is no mismatch in the matching list.
[0143] The steps implemented by the functional modules of the obstacle mismatching relationship removal device can refer to the various embodiments of the obstacle mismatching relationship removal method of the present invention, and will not be described in detail here.
[0144] In addition, an embodiment of the present invention further provides a storage medium, on which is stored a program for removing an obstacle mismatching relationship. When the program for removing an obstacle mismatching relationship is executed by a processor, the following operations are implemented:
[0145] Obtaining a lane line detection result of a lane to be detected, and partitioning obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions;
[0146] Perform obstacle matching on obstacles in different partitions to obtain a matching list;
[0147] Identify mismatched obstacles in the matching list, and release the matching relationship of the mismatched obstacles until no mismatches exist in the matching list.
[0148] Furthermore, when the obstacle mismatching relationship elimination program is executed by the processor, the following operations are also implemented:
[0149] Obtain a lane line detection result of a lane to be detected, partition obstacles detected by different sensors according to the lane line detection result, and obtain obstacles on the lane and obstacles outside the lane after partitioning.
[0150] Furthermore, when the obstacle mismatching relationship elimination program is executed by the processor, the following operations are also implemented:
[0151] Obtaining a millimeter-wave radar detection result, and setting a static obstacle near the leftmost lane line and the rightmost lane line as a curb-type obstacle according to the millimeter-wave radar detection result;
[0152] Searching for the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result closest to the obstacle of the edge type according to the nearest neighbor search principle;
[0153] The obstacle recognition result of the millimeter-wave radar and the obstacle recognition result of the camera are matched to obtain a matching list.
[0154] Furthermore, when the obstacle mismatching relationship elimination program is executed by the processor, the following operations are also implemented:
[0155] The millimeter-wave radar obstacle recognition result and the camera obstacle recognition result are matched according to a preset matching condition to obtain a matching list of the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result.
[0156] Furthermore, when the obstacle mismatching relationship elimination program is executed by the processor, the following operations are also implemented:
[0157] Traversing the matching list, obtaining a lateral distance difference between a lateral distance of a target millimeter-wave radar obstacle and a lateral distance of a target camera obstacle that are matched in pairs in the matching list;
[0158] Determine whether the lateral distance difference is greater than a preset width, and identify mismatched obstacles in the matching list according to the difference determination result;
[0159] The matching relationship of the mismatched obstacle is released until there is no mismatch in the matching list.
[0160] Furthermore, when the obstacle mismatching relationship elimination program is executed by the processor, the following operations are also implemented:
[0161] Determine whether the lateral distance difference is greater than a preset width, and obtain a difference determination result;
[0162] When the difference judgment result is that the lateral distance difference is not greater than the preset width, retaining the current matching relationship;
[0163] When the difference judgment result is that the lateral distance difference is greater than the preset width, judging whether the type of the target millimeter-wave radar obstacle is a curb type, and obtaining a type judgment result;
[0164] The mismatched obstacles in the matching list are identified according to the type determination result.
[0165] Furthermore, when the obstacle mismatching relationship elimination program is executed by the processor, the following operations are also implemented:
[0166] When the type judgment result is that the type of the target millimeter-wave radar obstacle is not a curb type, retaining the current matching relationship;
[0167] When the type determination result is that the type of the target millimeter-wave radar obstacle is a curb type, determining whether the type of the target camera obstacle is a curb type;
[0168] When the type of the target camera obstacle is not a curb type, retaining the current matching relationship;
[0169] When the type of the target camera obstacle is a curb type, determining whether the longitudinal speed of the target camera obstacle is less than a preset speed threshold;
[0170] When the longitudinal speed is less than the preset speed threshold, identifying a mismatched obstacle in the matching list and releasing a current matching relationship of the mismatched obstacle;
[0171] When the longitudinal speed is not less than the preset speed threshold, the current matching relationship is retained, and new mismatched obstacles in the matching list are continuously identified until no mismatches exist in the matching list.
[0172] Those skilled in the art will appreciate that all or part of the steps in the above-mentioned implementation method can be accomplished by instructing related hardware through a program, and the program is stored in a storage medium, including a number of instructions for enabling a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each implementation method of the present application; and the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
[0173] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0174] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0175] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for resolving mismatched obstacle relationships. It is characterized in that The obstacle mismatching relationship removal method comprises: Obtaining a lane line detection result of a lane to be detected, and partitioning obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions; Perform obstacle matching on obstacles in different partitions to obtain a matching list; Identify mismatched obstacles in the matching list, and release the matching relationship of the mismatched obstacles until no mismatches exist in the matching list.
2. The method for resolving an obstacle mismatching relationship according to claim 1, It is characterized in that The obtaining of a lane line detection result of a lane to be detected, and partitioning obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions, includes: Obtain a lane line detection result of a lane to be detected, partition obstacles detected by different sensors according to the lane line detection result, and obtain obstacles on the lane and obstacles outside the lane after partitioning.
3. The method for resolving an obstacle mismatching relationship according to claim 1, It is characterized in that The obstacle matching of obstacles in different partitions to obtain a matching list includes: Obtaining a millimeter-wave radar detection result, and setting a static obstacle near the leftmost lane line and the rightmost lane line as a curb-type obstacle according to the millimeter-wave radar detection result; Searching for the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result closest to the obstacle of the edge type according to the nearest neighbor search principle; The obstacle recognition result of the millimeter-wave radar and the obstacle recognition result of the camera are matched to obtain a matching list.
4. The method for resolving an obstacle mismatching relationship according to claim 3, It is characterized in that The matching of the obstacle recognition result of the millimeter wave radar and the obstacle recognition result of the camera to obtain a matching list includes: The millimeter-wave radar obstacle recognition result and the camera obstacle recognition result are matched according to a preset matching condition to obtain a matching list of the millimeter-wave radar obstacle recognition result and the camera obstacle recognition result.
5. The method for resolving an obstacle mismatching relationship according to claim 1, It is characterized in that The step of identifying the mismatched obstacles in the matching list and releasing the matching relationship of the mismatched obstacles until no mismatches exist in the matching list includes: Traversing the matching list, obtaining a lateral distance difference between a lateral distance of a target millimeter-wave radar obstacle and a lateral distance of a target camera obstacle that are matched in pairs in the matching list; Determine whether the lateral distance difference is greater than a preset width, and identify mismatched obstacles in the matching list according to the difference determination result; The matching relationship of the mismatched obstacle is released until there is no mismatch in the matching list.
6. The method for resolving an obstacle mismatching relationship according to claim 5, It is characterized in that The determining whether the lateral distance difference is greater than a preset width, and identifying the mismatched obstacle in the matching list according to the difference determination result, includes: Determine whether the lateral distance difference is greater than a preset width, and obtain a difference determination result; When the difference judgment result is that the lateral distance difference is not greater than the preset width, retaining the current matching relationship; When the difference judgment result is that the lateral distance difference is greater than the preset width, judging whether the type of the target millimeter-wave radar obstacle is a curb type, and obtaining a type judgment result; The mismatched obstacles in the matching list are identified according to the type determination result.
7. The method for resolving an obstacle mismatching relationship according to claim 6, It is characterized in that The step of identifying mismatched obstacles in the matching list according to the type determination result includes: When the type judgment result is that the type of the target millimeter-wave radar obstacle is not a curb type, retaining the current matching relationship; When the type determination result is that the type of the target millimeter-wave radar obstacle is a curb type, determining whether the type of the target camera obstacle is a curb type; When the type of the target camera obstacle is not a curb type, retaining the current matching relationship; When the type of the target camera obstacle is a curb type, determining whether the longitudinal speed of the target camera obstacle is less than a preset speed threshold; When the longitudinal speed is less than the preset speed threshold, identifying a mismatched obstacle in the matching list and releasing a current matching relationship of the mismatched obstacle; When the longitudinal speed is not less than the preset speed threshold, the current matching relationship is retained, and new mismatched obstacles in the matching list are continuously identified until no mismatches exist in the matching list.
8. A device for removing mismatched obstacles. It is characterized in that The obstacle mismatching relationship removal device comprises: A partitioning module, used to obtain a lane line detection result of a lane to be detected, and partition obstacles detected by different sensors according to the lane line detection result to obtain obstacles in different partitions; The obstacle matching module is used to match obstacles in different partitions and obtain a matching list; The release module is used to identify the mismatched obstacles in the matching list and release the matching relationship of the mismatched obstacles until there is no mismatch in the matching list.
9. A device for removing mismatched obstacles. It is characterized in that The obstacle mismatching relationship resolution device comprises: a memory, a processor, and an obstacle mismatching relationship resolution program stored in the memory and executable on the processor, wherein the obstacle mismatching relationship resolution program is configured to implement the steps of the obstacle mismatching relationship resolution method as described in any one of claims 1 to 7.
10. A storage medium, It is characterized in that The storage medium stores an obstacle mismatching relationship elimination program, and when the obstacle mismatching relationship elimination program is executed by the processor, the steps of the obstacle mismatching relationship elimination method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Object recognition apparatus
CN105522998A
Obstacle information fusion method and system for binocular camera and millimeter-wave radar
CN111898582A
Matching fusion obstacle detection method and system, electronic equipment and storage medium
CN115909281A
Obstacle recognition device
JP2008040819A
