Elevation target detection method and device based on elevation curve

Abstract

The application relates to an elevation target detection method and device based on an elevation curve, which comprises the following steps: acquiring a wheel track region of interest, extracting parallax points in a parallax map, and generating an elevation curve as original elevation data after processing according to a preset pretreatment mode; screening target signals based on the original elevation data, and determining candidate target elevations by using the target signals; traversing the candidate target elevations, screening candidate targets in the candidate target elevations, and extracting detection targets. The three-dimensional coordinates of objects in a scene are calculated by acquiring parallax data of the scene by using binocular cameras, so that the accuracy of overall detection is improved. Furthermore, detection data is processed in the order of data pretreatment, determination of candidate target elevations and determination of detection targets, so that accurate detection results are obtained, and the reliability of application in a car magic carpet is improved.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent driving technology, and in particular to a method and apparatus for detecting elevation targets based on elevation curves. Background Technology

[0002] With the rapid development of automotive intelligence, users are placing increasingly higher demands on vehicle comfort and driving stability. The "magic carpet" suspension system, a vehicle suspension technology that has emerged in recent years, uses precise electronic control and intelligent sensor technology to better perceive the unevenness of the road surface during vehicle operation. This allows for adjustments to the damping and stiffness of the vehicle's chassis suspension system, providing a more stable and safer driving experience. Effective sensing methods are a crucial prerequisite for ensuring the "magic carpet" system enhances vehicle driving comfort.

[0003] Currently, the main methods used in the automotive field for road condition perception include LiDAR, millimeter-wave radar, and computer vision technology. However, regarding the "magic carpet" perception method, LiDAR is costly and energy-intensive, and millimeter-wave radar cannot detect small or static targets. In the computer vision technology solution, a monocular camera is usually used to acquire images, and AI detection is used for target recognition, which has high accuracy. However, this method is difficult to identify non-standard obstacles and cannot obtain the location information of the identified obstacles. It also needs to be combined with LiDAR or millimeter-wave radar.

[0004] Therefore, to reduce the production cost and improve the sensing accuracy of the magic carpet sensing system, a binocular stereo vision system can be used for ground target detection. However, in existing technologies, because binocular stereo vision systems acquire target 3D information based on the principle of binocular disparity calculation, the acquired disparity data is subject to noise interference. Therefore, how to perform ground elevation target detection based on disparity data is a problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, this application proposes a method and apparatus for detecting elevation targets based on elevation curves to solve the above problems.

[0006] This application proposes a method for detecting elevation targets based on elevation curves, comprising the following steps:

[0007] The region of interest of the wheel trajectory is obtained, the disparity points in the disparity map are extracted, and after processing according to the preset preprocessing method, the generated elevation curve is used as the original elevation data.

[0008] Target signals are filtered based on the original elevation data, and candidate target elevations are determined using the target signals;

[0009] The candidate target elevations are traversed, and the candidate targets in the candidate target elevations are filtered to obtain the detection targets.

[0010] As an optional implementation of this application, optionally, the step of obtaining the region of interest of the wheel trajectory, extracting disparity points, and processing them according to a preset processing method to generate the corresponding elevation curve as the original elevation data includes:

[0011] Obtain the region of interest for the wheel trajectory and extract the disparity points in the binocular disparity V-map;

[0012] The average ordinate corresponding to each unit of disparity is calculated based on the binocular disparity V-map to obtain the point set, and a linear equation of the ground is fitted.

[0013] Using the ground as a reference, a corresponding elevation curve is generated based on the fitted linear equation of the ground as the original elevation data; and the elevation curve represents the correspondence between the detection distance and the elevation height.

[0014] As an optional implementation of this application, the step of filtering target signals based on the original elevation data and determining candidate target elevations using the target signals may include:

[0015] The original elevation data is filtered according to a height threshold to obtain candidate target signals;

[0016] The candidate target signals are filtered according to a preset length range to obtain the target signals;

[0017] Traverse the target signals and search for the corresponding target boundaries;

[0018] Based on the boundary range of the target boundary, the corresponding original elevation data is taken as the elevation value to obtain the candidate target elevation.

[0019] As an optional implementation of this application, optionally, before performing height threshold filtering on the original elevation data according to a height threshold to obtain candidate target signals, the method further includes:

[0020] The original elevation data is smoothed using a window to obtain smoothed elevations;

[0021] Subtract the smoothed elevation from the original elevation data to obtain the target reference elevation;

[0022] Based on the stated height threshold, the target reference elevation is filtered by height threshold to obtain the candidate target signal.

[0023] As an optional implementation of this application, the candidate target signals may be filtered according to a preset length range to obtain target signals, including:

[0024] Two adjacent candidate target signals whose interval length conforms to a preset interval size are obtained and denoted as adjacent candidate signals;

[0025] Determine whether two adjacent candidate signals are adjacent signals in the same direction, wherein the adjacent signals in the same direction are two adjacent candidate signals with the same elevation direction;

[0026] If so, the elevation between the adjacent signals in the same direction is supplemented, and the supplemented adjacent signals in the same direction are recorded as a target signal.

[0027] As an optional implementation of this application, optionally, the step of traversing the target signal and searching for the corresponding target boundary includes:

[0028] Determine the end point of the elevation;

[0029] Starting from the elevation end position, according to the preset search logic, perform boundary searches on the positive and negative elevations in the target signal respectively to determine the corresponding target boundaries;

[0030] The target boundary includes a positive elevation near-end boundary, a positive elevation far-end boundary, a sub-elevation near-end boundary, and a negative elevation far-end boundary.

[0031] As an optional implementation of this application, optionally, the step of traversing the candidate target elevations, filtering the candidate targets in the candidate target elevations, and extracting the detection targets includes:

[0032] Iterate through the candidate target elevations and filter each target in the candidate target elevations by applying height and width thresholds;

[0033] Determine the interval length between two adjacent targets in the filtered candidate target elevations;

[0034] When it is determined that the interval length is greater than or equal to the preset distance, the two adjacent targets are marked as one detection target.

[0035] As an optional implementation of this application, the step of traversing the candidate target elevations, filtering the candidate targets in the candidate target elevations, and extracting the detection targets further includes:

[0036] When the interval length is less than the preset distance, the two adjacent targets are determined to be adjacent targets and are marked with the same group number;

[0037] The adjacent targets are denoted as a single detection target.

[0038] As an optional implementation of this application, optionally, after determining that the two adjacent targets are adjacent targets and marking them with the same group number when the interval length is less than the preset distance, the method further includes:

[0039] Traverse each group of adjacent targets and calculate the screening index of the binocular disparity V-map for each target in the group;

[0040] The adjacent targets are filtered according to the filtering criteria;

[0041] If the screening index of a target within a group is the largest within the group or greater than the screening threshold, then the corresponding target is retained as the detection target.

[0042] As an optional implementation of this application, the screening criteria may optionally include at least one of the area and average height of the target V-graph.

[0043] As an optional implementation of this application, after traversing the candidate target elevations, filtering the candidate targets in the candidate target elevations, and extracting the detection targets, the method further includes:

[0044] Iterate through the tracking list and predict the position of the tracked targets in the tracking list at the current time;

[0045] Based on the current position of the tracked target and the current detection target, the lifecycle of the tracked target is updated, and the corresponding tracking type is determined;

[0046] The tracking target, after the tracking type is determined, is verified for multiple consecutive frames, and output after meeting the preset output conditions.

[0047] In another aspect, this application provides an apparatus for implementing the elevation target detection method based on elevation curves as described in any of the above claims, comprising:

[0048] The data preprocessing module is configured to acquire the region of interest of the wheel trajectory, extract the disparity points in the disparity map, process them according to the preset preprocessing method, and use the generated elevation curve as the original elevation data.

[0049] The candidate target elevation determination module is configured to filter target signals based on the original elevation data and use the target signals to determine the candidate target elevations;

[0050] The target determination module is configured to traverse the candidate target elevations, filter the candidate targets in the candidate target elevations, and extract the target to be detected.

[0051] As an optional implementation of this application, it may also include:

[0052] The prediction module is configured to traverse the tracking list and predict the position of the tracked targets in the tracking list at the current time.

[0053] The tracking and matching module is configured to match the tracking target's current position with the detected target's current position, update the tracking target's lifecycle, and determine the corresponding tracking type.

[0054] The output module is configured to perform continuous multi-frame verification on the tracking target after the tracking type is determined, and output the result after meeting the preset output conditions.

[0055] In another aspect, this application provides an electronic device, comprising:

[0056] processor;

[0057] Memory used to store processor-executable instructions;

[0058] The processor is configured to implement the elevation target detection method based on elevation curves as described above when executing the executable instructions.

[0059] In another aspect, this application provides a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the elevation target detection method based on elevation curves as described above.

[0060] Technical effects of the present invention:

[0061] This application utilizes elevation data acquired by a binocular camera to extract small or static targets on the road surface in front of a vehicle, making target detection unrestricted by target type. Specifically, it includes: acquiring the region of interest (ROI) of the wheel trajectory, extracting disparity points from the disparity map, processing them according to a preset preprocessing method, and using the generated elevation curve as the raw elevation data; filtering target signals based on the raw elevation data and determining candidate target elevations using these signals; traversing the candidate target elevations, filtering candidate targets within the candidate elevations, and extracting the detected targets. In other words, this application uses disparity data acquired by a binocular camera to calculate the 3D coordinates of objects in the scene, enabling not only the perception of small or static targets but also effective detection of non-standard obstacles, improving overall detection accuracy. Furthermore, the detection data is processed sequentially according to the order of data preprocessing, determining candidate target elevations, and determining the detected targets, thereby obtaining accurate detection results and improving the reliability of its application in automotive magic carpets.

[0062] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0063] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0064] Figure 1 The diagram shows a flowchart of the elevation target detection method based on elevation curves in an embodiment of this application.

[0065] Figure 2 The diagram shown is a schematic representation of the elevation curve processing procedure in an embodiment of this application.

[0066] Figure 3 The diagram shown is a schematic representation of the smoothing process in an embodiment of this application.

[0067] Figure 4 The diagram shown is a schematic diagram of the logic for determining the near-end boundary of positive elevation in an embodiment of this application;

[0068] Figure 5 The diagram shown is a schematic diagram of the logic for determining the far-end boundary of the positive elevation in an embodiment of this application;

[0069] Figure 6 The diagram shown is a schematic diagram of the logic for determining the near-end boundary of negative elevation in an embodiment of this application;

[0070] Figure 7 The diagram shown is a schematic diagram of the logic for determining the negative elevation far-end boundary in an embodiment of this application;

[0071] Figure 8 The diagram shown is a schematic representation of a real-world application scenario of the elevation target detection method based on elevation curves in this application.

[0072] Figure 9 The diagram shown is a schematic representation of the target lifecycle in an embodiment of this application.

[0073] Figure 10 The diagram shown is a schematic representation of target matching and intersection in an embodiment of this application.

[0074] Figure 11 The diagram shown is a schematic diagram of tracking type conversion conditions in an embodiment of this application;

[0075] Figure 12 The diagram shown is a schematic representation of the target detection results in an embodiment of this application. Detailed Implementation

[0076] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0077] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0078] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0079] Compared to LiDAR, millimeter-wave radar, and computer vision technologies, binocular vision methods offer advantages such as low cost, simple structure, and high reliability, making them suitable for application on automotive magic carpets. Therefore, this application utilizes elevation data acquired by binocular cameras to extract targets on the road surface in front of the vehicle, such as potholes, bumps, and speed bumps. By using dual cameras to obtain the parallax of the scene, the three-dimensional coordinates of objects in the scene are calculated. This method is not limited by target type and can detect small and static targets, while also offering high real-time performance.

[0080] Example 1

[0081] like Figure 1 As shown, this application proposes a method for detecting elevation targets based on elevation curves, comprising the following steps:

[0082] S100. Obtain the region of interest of the wheel trajectory, extract the disparity points in the disparity map, process them according to the preset preprocessing method, and use the generated elevation curve as the original elevation data.

[0083] In this step, the sampled images (including the left and right images) acquired by the binocular camera are used to calculate the corresponding disparity map based on the left (or right) image using the binocular disparity calculation principle. The calculated disparity map is then transformed using UV to obtain the corresponding binocular disparity V map.

[0084] Then, by combining the obtained wheel trajectory ROI (which can be a specified area in the image), the average ordinate corresponding to each unit disparity in the binocular disparity V map is calculated to obtain the corresponding point set. After fitting the ground linear equation through the algorithm, the corresponding elevation curve is generated with the ground as the reference, so as to obtain the correspondence between the detection distance Z and the elevation height H, which is used as the original elevation data.

[0085] S200. Target signals are filtered based on the original elevation data, and candidate target elevations are determined using the target signals;

[0086] Those skilled in the art will understand that the area with a value in the disparity map corresponds to an object in the left image. Therefore, after conversion to an elevation curve, a positive value in the elevation curve indicates that the corresponding object is above the ground, and vice versa, it indicates that the object is below the ground.

[0087] In this step, the raw elevation data can be filtered according to the set height threshold to obtain the corresponding target signal. Then, signals that meet the predetermined conditions can be selected from the target signals according to actual needs as the signals to be identified, so as to be input into the vehicle suspension system for real-time adjustment.

[0088] Specifically, after smoothing the original elevation data by windowing, the ground can be decomposed by subtracting the original elevation data from the smoothed elevation to obtain the target reference elevation. The purpose is to use the target reference elevation to screen out candidate target signals and obtain the target signal after filtering out abnormal signals. Then, after searching each target boundary, the candidate target elevation can be obtained using the original elevation data as the elevation value.

[0089] S300. Traverse the candidate target elevations, filter the candidate targets in the candidate target elevations, and extract the detection targets.

[0090] In this step, by traversing the candidate target elevations, each target is filtered using a threshold, targets exceeding the threshold are removed, and it is determined whether they can be marked as detection targets. It should be noted that after determining the detection targets, this application can also calculate the corresponding target attributes for subsequent tracking and discrimination, further achieving the goal of tracking elevation targets.

[0091] As an optional implementation of this application, optionally, the region of interest for the wheel trajectory is obtained, disparity points are extracted, and after processing according to a preset processing method, a corresponding elevation curve is generated as the original elevation data. This includes: obtaining the region of interest for the wheel trajectory and extracting disparity points from the binocular disparity V-map; calculating the average ordinate corresponding to each unit of disparity based on the binocular disparity V-map to obtain a point set, and fitting a linear equation of the ground; using the ground as a reference, generating a corresponding elevation curve as the original elevation data based on the fitted linear equation of the ground; and the elevation curve represents the correspondence between the detection distance Z and the elevation height H.

[0092] It should be noted that after obtaining the Region of Interest (ROI) of the wheel trajectory and extracting the disparity points within it, the average ordinate y corresponding to each unit disparity d is calculated based on the binocular disparity V-map, resulting in a point set {(d,y)}. For example, if the x-coordinate in the disparity V-map ranges from 200 to 900, the average ordinate y can be calculated at points 201 / 202 / 203… or at points 201 / 203 / 205…, depending on the specific compensation required. After obtaining the point set {(d,y)}, the ground linear equation is fitted using the RANSAC algorithm. Using the ground as a reference, a corresponding elevation curve is generated, revealing the correspondence between the detection distance Z and the elevation H.

[0093] Figure 2 The diagram shows the elevation curve processing procedure, and Figure 2 The elevation data are labeled ①-⑥ from bottom to top. For example... Figure 3 As shown in Figure ①, the horizontal axis represents the detection distance Z, from the starting point of 5.5m to the ending point of 18m, with a unit accuracy of cm. The vertical axis represents the elevation height H, which indicates the height of the elevation curve relative to the ground, with a unit accuracy of mm.

[0094] As an optional implementation of this application, in order to screen effective targets in the original elevation data and improve the accuracy of target identification, optionally, target signals are screened based on the original elevation data, and candidate target elevations are determined using the target signals, including: screening the original elevation data according to a height threshold to obtain candidate target signals; screening the candidate target signals according to a preset length range to obtain target signals; traversing the target signals to search for corresponding target boundaries; and taking the corresponding original elevation data as the elevation value according to the boundary range of the target boundary to obtain the candidate target elevations.

[0095] It should be noted that when determining the candidate target elevation, the original elevation data is first filtered according to a height threshold. By traversing the original elevation data, those that do not match the height threshold are filtered out, thus obtaining candidate target signals. Subsequently, the candidate target signals are filtered according to a preset length range, that is, signals that do not meet the range requirements are excluded based on the preset length range, thus obtaining the target signal. Preferably, the preset length range in this application is a signal length greater than or equal to 8 cm or less than or equal to 150 cm in the z-direction. That is, during the target signal filtering process, abnormal signals with a length less than 8 cm or greater than 150 cm need to be filtered out to obtain the target signal. It is particularly important to note that before obtaining the final candidate target elevation, the target boundary needs to be searched to determine the start and end positions of the signal. Based on this, corresponding elevation data segments are extracted from the original elevation data as candidate target elevations for subsequent analysis and processing.

[0096] As an optional implementation of this application, before performing height threshold filtering on the original elevation data according to the height threshold to obtain candidate target signals, the method further includes: performing window smoothing on the original elevation data to obtain smoothed elevation; subtracting the smoothed elevation from the original elevation data to obtain the target reference elevation; and performing height threshold filtering on the target reference elevation according to the height threshold to obtain candidate target signals.

[0097] It should be noted that the method of this application eliminates possible false targets caused by the offset of the ground fitting equation by subtracting the original elevation data from the smoothed elevation, and obtains the target reference elevation. Therefore, when using the target reference elevation for height threshold screening, more accurate candidate target signals can be obtained.

[0098] As an optional implementation of this application, the candidate target signals are optionally filtered according to a preset length range to obtain a target signal, including: obtaining two adjacent candidate target signals whose interval length conforms to a preset interval size, and recording them as adjacent candidate signals; determining whether the two adjacent candidate signals are adjacent signals in the same direction, wherein adjacent signals in the same direction are two adjacent candidate signals with the same elevation direction; if so, the elevation height between the adjacent signals in the same direction is supplemented, and the supplemented adjacent signals in the same direction are recorded as a target signal.

[0099] It should be noted that in the process of screening target signals, two adjacent candidate target signals are first selected according to a preset interval size. Preferably, the preset interval size in this application is less than 15 cm, meaning two candidate target signals with an interval length less than 15 cm are selected as adjacent candidate signals. When two adjacent candidate signals are determined to be adjacent signals in the same direction (i.e., two adjacent candidate signals with the same elevation direction), these adjacent signals in the same direction need to be supplemented to ensure signal continuity. After supplementing the elevation, the two adjacent signals in the same direction are considered as a single target signal, thus enabling the processing of continuous signals in the same direction during subsequent target boundary search and elevation value extraction. Furthermore, it should be noted that after supplementing the adjacent signals in the same direction, abnormal signals with a signal length less than 8 cm or greater than 150 cm are filtered again in the z-direction to obtain the target signal.

[0100] As an optional implementation of this application, optionally, traversing the target signal and searching for the corresponding target boundary includes: determining the elevation end position; taking the elevation end position as the starting point, and according to a preset search logic, performing boundary searches on the positive and negative elevations in the target signal respectively to determine the corresponding target boundary; wherein, the target boundary includes the positive elevation near-end boundary, the positive elevation far-end boundary, the sub-elevation near-end boundary, and the negative elevation far-end boundary.

[0101] It should be noted that this application employs different search logics for positive and negative elevations in the target signal. The search logic includes forward search and backward search, which differ for positive and negative elevations, thereby achieving the goal of comprehensively searching the boundaries of each target.

[0102] Specifically, for the original elevation data①, firstly, window smoothing is performed in the Z-direction, with the window range set to 100cm and the window sliding step size to 1cm. The smoothing process is as follows: Figure 3 As shown. When Z is less than 50cm from the starting point (point A), the average elevation between the starting point and Z is taken as the smoothed height at Z; when Z is less than 50cm from the ending point (point B), the average elevation between 50cm to the left and Z to the ending point is taken as the smoothed height at Z; when Z is at other distances (point C), the average elevation between 50cm to the left and right is taken as the smoothed height at Z. The smoothed height at each distance is calculated, and the ground change trend is fitted to obtain the smoothed elevation ②.

[0103] Furthermore, by subtracting the original elevation data from the smoothed elevation data, the target reference elevation ③ is obtained, which can eliminate possible false detections of targets caused by the offset of the ground fitting equation.

[0104] Subsequently, for each distance (per centimeter) of the target reference elevation, a height threshold is applied using formula 2-1 to filter candidate target signals ④. The specific height threshold can be manually set according to actual needs. The formula for calculating the height threshold is shown in formula 2-1:

[0105] HThresh = 2 * Z (2-1)

[0106] In the formula, Z is the distance (unit: m), and HThresh is the height threshold (unit: mm) at distance Z.

[0107] Furthermore, for two candidate target signals with an interval length of less than 15cm, if the elevation directions of the two signals are the same, the elevation between the two signals is supplemented. This can be done by proportionally increasing / decreasing the elevation difference between the two signals according to the distance between them, or by directly using the average elevation of the two signals. After supplementation, abnormal signals with a length less than 8cm or greater than 150cm are filtered again in the Z-direction to obtain the target signal ⑤.

[0108] Finally, the target signals are traversed, starting from the elevation end position, searching for each target boundary. The search logic includes forward search and backward search, which differs for positive and negative elevations. Specifically:

[0109] 1. Logic for determining the near-end boundary of positive elevation: Traverse the target benchmark elevation data towards the near end. If a continuous 10cm elevation mark appears, including negative elevations or positive elevations with a width not exceeding 5cm, then the farthest point excluding this 10cm elevation mark is taken as the near-end boundary. Figure 4 As shown;

[0110] 2. Logic for determining the far-end boundary of positive elevation: Traverse the target benchmark elevation data towards the far end. If a continuous 10cm elevation mark appears, including negative elevations or positive elevations with a width not exceeding 5cm, then the nearest end excluding this 10cm elevation mark is taken as the far-end boundary. Figure 5 As shown;

[0111] 3. Logic for determining the near-end boundary of negative elevation: Traverse the original elevation data towards the near end. If a cumulative elevation marker of 10cm appears (i.e., an elevation greater than -10mm), then take the farthest point excluding this 10cm elevation marker as the near-end boundary. Figure 6 As shown;

[0112] 4. Logic for determining the far-end boundary of negative elevation: Traverse the original elevation data to the far end. If an elevation marker (i.e., a negative elevation of -10mm) appears, then take that elevation location as the far-end boundary. Figure 7 As shown.

[0113] After determining the boundaries of each target, the original elevation data of the corresponding boundary range is taken as the final elevation value to obtain the candidate target elevations⑥.

[0114] As an optional implementation of this application, optionally, the candidate target elevations are traversed, and after filtering the candidate targets in the candidate target elevations, the detection targets are extracted, including: S310, traversing the candidate target elevations and filtering each target in the candidate target elevations by height threshold and width threshold; S320, determining the interval length between two adjacent targets in the filtered candidate target elevations; S330, when the interval length is determined to be greater than or equal to a preset distance, marking the two adjacent targets as a detection target respectively.

[0115] It should be noted that, through step S310, the candidate target elevations are traversed, and each target (candidate target, i.e. the area with elevation data in the candidate target elevation) is filtered by the height threshold of Equation 2-2 and the width threshold of Equation 2-3.

[0116] HThresh_out = 3 * Z (2-2)

[0117] LMax = 200(cm), LMin = 10(cm) (2-3)

[0118] In the formula, Z is the distance (in meters), HThresh_out is the height threshold (in mm) at distance Z, and LMax and LMin are the maximum and minimum values ​​of the target width, respectively. Targets exceeding these thresholds are thus removed.

[0119] Furthermore, step S320 determines the distance between two adjacent targets, and step S330 determines that when the distance between them is greater than or equal to a preset distance, the two adjacent targets are each marked as a detection target. Preferably, the preset distance in this application is an interval length of 20cm. That is, when the interval between two adjacent targets in the selected candidate target elevations is greater than or equal to 20cm, the two adjacent targets are each marked as a detection target, thereby achieving the determination of the detection target.

[0120] As an optional implementation of this application, optionally, after traversing the candidate target elevations and filtering the candidate targets in the candidate target elevations, the detection targets are extracted, and the method further includes: S340, when the interval length is less than a preset distance, determining two adjacent targets as adjacent targets and marking them with the same group number; and recording adjacent targets as one detection target.

[0121] It should be noted that, in order to obtain more accurate detection targets, the method of this application identifies two adjacent targets with an interval length less than a preset distance as adjacent targets, and records them as a single detection target after marking them with the same group number. That is, they are treated as a whole, and the elevation data, location information, etc., of adjacent targets are integrated to form a unified detection target representation. In other words, if the interval length between two adjacent targets in the selected candidate target elevation is less than 20cm, then the two adjacent targets need to be identified as adjacent targets first, marked with the same group number, and then recorded as a single detection target to obtain a more accurate detection target.

[0122] As an optional implementation of this application, when the interval length is less than a preset distance, after determining two adjacent targets as adjacent targets and marking them with the same group number, the method further includes: traversing each group of adjacent targets and calculating the screening index of the binocular disparity map of each target in the group; screening adjacent targets according to the screening index; if the screening index of a target in the group is the largest in the group or greater than the screening threshold, then the corresponding target is retained as the detection target.

[0123] It should be noted that the screening threshold in this application is set according to the type of screening index. That is, the size of the screening threshold is different when the screening index is different, so as to be able to screen out suitable detection targets from adjacent targets.

[0124] As an optional implementation of this application, the screening criteria may optionally include at least one of the area and average height of the target V-map. That is, the method of this application specifically uses at least one of the area and average height of the target V-map as a screening criterion to screen adjacent targets. The specific criteria can be selected flexibly according to actual needs to screen adjacent targets.

[0125] That is, if the distance between two adjacent targets in the z-direction is less than 20cm, they are considered adjacent targets and labeled with the same group number. Then, each group of adjacent targets is traversed, and the V-map area S and average height H of each target within the group are calculated, using formulas as shown in Figures 2-4 and 2-5.

[0126] S = (2-4)

[0127] H = S / L (2-5)

[0128] In the formula, a and b are the disparities corresponding to the distances to the far and near boundaries of the target, respectively. and , respectively, are the vertical coordinates of the target and the ground in the V diagram, and L is the elevation length of the target from the near boundary to the far boundary in the z-direction.

[0129] Furthermore, adjacent targets are screened based on screening criteria. If the screening criterion of a target within a group is the largest within the group or greater than the screening threshold, the corresponding target is retained as the detection target. Preferably, the screening criterion is at least one of the area and average height of the target's V-map. When the screening criterion is the area of ​​the target's V-map, the corresponding screening threshold is preferably 300; when the screening criterion is the average height, the corresponding screening threshold is preferably 6.

[0130] Specifically, in one scenario, if the screening criterion is the area of ​​the target V-map, then if the area S of the target V-map is the largest within the group or greater than the screening threshold (e.g., 300), the target is retained; otherwise, it is discarded. In another scenario, if the screening criterion is the average height H, then if the average height H of the target is the largest within the group or greater than the screening threshold (e.g., 6), the target is retained; otherwise, it is discarded. All the retained targets are the detection targets. Figure 12 The result is shown as the target detection result, for example, Z is 8.88m, H is 80mm, and L is 200mm. After determining the target, the target attributes are calculated as the basis for subsequent output, and the target is stored in the vector _m_vDetect. The target attributes include the target disparity density and the road surface disparity density, which are used to filter noise in subsequent steps S630 and S640.

[0131] The target disparity density is calculated as follows: The initial disparity is calculated based on the target's distance Z and length L. This initial disparity is then substituted into the ground equation to obtain the bottom and top pixels of the target on the image. This is then extended upwards and downwards by 3 pixels each. Combined with the wheel ROI, the target ROI is obtained. The number of disparity points within the target ROI is calculated, and divided by the pixel area of ​​the target ROI, yields the target disparity density.

[0132] The method for calculating road surface disparity density is as follows: Take a vertical distance of 5 pixels above and below the target ROI, and combine it with the wheel ROI to obtain the road surface ROI. Calculate the number of disparity points within the road surface ROI, and divide it by the pixel area of ​​the road surface ROI to obtain the road surface disparity density.

[0133] As an optional implementation of this application, optionally, after traversing the candidate target elevations, filtering the candidate targets in the candidate target elevations, and extracting the detection targets, the method further includes: S400, traversing the tracking list and predicting the position of the tracking targets in the tracking list at the current time; S500, matching the position of the tracking target at the current time with the detection targets at the current time, updating the lifecycle of the tracking target, and determining the corresponding tracking type; S600, performing continuous multi-frame verification on the tracking targets after determining the tracking type, and outputting the target after meeting the preset output conditions.

[0134] It should be noted that this application processes the detection data sequentially, following the order of data preprocessing, determining candidate target elevations, and determining the detection target, thereby obtaining accurate detection results that are more reliable for application in automotive magic carpet systems. Furthermore, this application also includes the prediction, tracking, and output of elevation targets. In step S400, during elevation target prediction, the current position of targets within the tracking list _m_vTrack is predicted and matched with the current detection target. The target's distance and position at the next moment are calculated based on vehicle speed and turning radius. Figure 8 As shown, it specifically includes:

[0135] S410. Check the tracking list _m_vTrack. If _m_vTrack is empty, proceed to step S500; otherwise, proceed to step S420.

[0136] S420. Traverse the tracking list _m_vTrack. For each tracking target, based on its distance z and vehicle turning radius r, the angle α between the corresponding vehicle speed and the horizontal direction can be calculated.

[0137] It is particularly important to emphasize that in the first frame of this application, the step of checking the tracking list will directly jump to step S500 to update the state of the detected target, thereby obtaining the tracking list _m_vTrack.

[0138] S430. Obtain the longitudinal speed relative to the target using trigonometric function relationships;

[0139] S440. Multiply the longitudinal vehicle speed and the frame interval time to obtain the distance z' of the tracked target in the next frame.

[0140] It should also be noted that in step S500, when matching the detected target and the tracked target, updating the lifecycle of the tracked target, and determining its tracking type, the tracked target has four lifecycles: trackage, matchage, missage, and bsage, which represent the tracking period, matching period, loss period, and blind zone period, respectively. Specific definitions and explanations are as follows: Figure 9 As shown:

[0141] trackage: Starts counting when the target is matched, and is set to 0 when the target is not matched;

[0142] matchage: The count starts when the target matches and remains unchanged when the target does not match.

[0143] missage: Starts counting when the target is not matched, otherwise sets it to 0;

[0144] bsage: Start counting when the target enters the blind zone, otherwise set to 0.

[0145] There are 5 tracking types for the target: candida, stable, missing, blindspot, and deleted, which represent pre-selected tracking, stable tracking, missing, blindspot, and deleted, respectively.

[0146] Specifically, step S500 includes:

[0147] S510. Traverse all tracked targets and delete those with the tracking type "deleted";

[0148] S520: Traverse all tracked targets, reset the matching attributes of each target, and enter the unmatched state;

[0149] S530. Traverse all tracked targets, and match each tracked target with all detected targets. The matching method and target intersection diagram are shown below. Figure 10 As shown.

[0150] 1. Determine whether the detected target and the tracked target have the same elevation in the same direction. If so, continue to determine the intersection of the two targets at distance z.

[0151] 2. If there is no intersection, skip the detection target; if there is an intersection, calculate the intersection-union ratio (IoU) between the detection target and the tracked target, and the calculation formula is shown in Equation 2-6.

[0152] IoU = 0.8 * I / L Trk + 0.2 * I / L Det (2-6)

[0153] 3. After the traversal is complete, if an intersecting target is found, the system enters the matched state and the target with the largest intersection-union ratio is taken as the matching object. The maximum intersection-union ratio is used as the confidence attribute of the target.

[0154] Subsequently, in step S540, for the matched tracking target, the tracking target is updated with the attributes of the matching object;

[0155] S550. For unmatched detected targets, add them as new tracking targets to the tracking list;

[0156] S560: Traverse all tracked targets and determine whether their distance z is less than 7 meters. If so, enter the blind zone.

[0157] S570. Traverse all tracked targets and update the lifecycle corresponding to the current tracking state;

[0158] S580. Traverse all tracked targets and convert the tracking type based on the target tracking lifecycle and distance z. Specific conversion conditions and diagrams are as follows: Figure 11 As shown:

[0159] (1) A: Trackage reaches 6, or Matchage reaches 30;

[0160] (2) B: bsage is not 0;

[0161] (3) C: z < 0;

[0162] (4) D: Missage reaches 2;

[0163] (5) E: missage is not 0;

[0164] (6) F: missage is 0;

[0165] (7) G: bsage is not 0;

[0166] (8) H: bsage is 0;

[0167] (9) I: Missage reaches 2;

[0168] In step S600, after determining the target's tracking type, its relevant attributes need to be verified across multiple consecutive frames. If the target meets the output conditions for multiple frames, the target is output; otherwise, it is not output. Specifically, this includes:

[0169] S610. Determine the output status of the target. If the target does not output, proceed to step S620; otherwise, proceed to step S660.

[0170] S620. Determine the tracking type of the target. If the tracking type is deleted or candidate, the target does not meet the output condition and proceed to step 660; otherwise, proceed to step 630.

[0171] S630. Determine the disparity density of the target. If the disparity density is less than 0.3, the target does not meet the output condition and proceed to step 660; otherwise, proceed to step 640.

[0172] S640. Determine the parallax density of the road surface. If the parallax density is less than 0.05, the target does not meet the output condition and proceed to step 660; otherwise, proceed to step 650.

[0173] S650, Set the target to meet the output conditions;

[0174] S660. For each target, set a pre-output cumulative value Acc and a non-output cumulative value NAcc, representing the frame counts for when the target meets and does not meet the output conditions, respectively. If the target meets the output conditions, Acc is incremented by 1 and NAcc is set to 0; otherwise, NAcc is incremented by 1, and when NAcc exceeds 10, Acc is set to 0. The target is output if and only if Acc is greater than or equal to 5.

[0175] In summary, the original elevation curve generated by the method of this application is as follows: Figure 2 As shown in ①, the original elevation data ① includes a positive elevation of 1100 (shown in red), a negative elevation of 1200 (shown in purple), and noise points of 1300 (shown in yellow and green). It should be noted that the purpose of this application is to remove noise interference from the elevation curve, thereby determining the candidate target elevation and providing a reliable basis for subsequent target detection. Specifically, the method of this application first smooths the original elevation data ① in the Z-direction to obtain a smoothed elevation ②, reducing noise and irregularities. The smoothed elevations corresponding to the positive elevation 1100 and the negative elevation 1200 are 2100 and 2200, respectively. Then, the original elevation data ① is subtracted from the smoothed elevation ② to eliminate possible false detections due to the shift in the ground fitting equation, thus obtaining the target reference elevation ③. At this point, interference such as noise 3300 still exists around the elevation 3100 or 3200 in the target reference elevation. It is particularly important to emphasize that this application obtains candidate target signals ④ by applying a height threshold filter based on the target reference elevation ③ using Equation 2-1. Furthermore, the candidate target signals ④ obtained at this point have undergone noise and other interference removal; for example, yellow-green noise no longer appears around candidate target signal 4100. This allows for the acquisition of the corresponding target signal ⑤ after further height compensation and abnormal signal filtering. Finally, the target signal ⑤ is traversed. For example, starting from the end position of the elevation for target signal 5100, the target boundary is searched, and the original elevation data ① of the target boundary range is taken as the final elevation value to obtain candidate target elevations ⑥, such as candidate target elevation 6100. After height and width threshold filtering, this allows for the identification of more accurate detection targets, such as the accurate extraction of targets like speed bumps.

[0176] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the control methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0177] Example 2

[0178] Furthermore, in another aspect, this application provides an apparatus for implementing the elevation target detection method based on elevation curves as described in any of the preceding claims, comprising:

[0179] The data preprocessing module is configured to acquire the region of interest of the wheel trajectory, extract the disparity points in the binocular view, process them according to the preset preprocessing method, and use the generated elevation curve as the original elevation data.

[0180] The candidate target elevation determination module is configured to filter target signals based on the original elevation data and use the target signals to determine the candidate target elevation.

[0181] The target determination module is configured to traverse the candidate target elevations, filter the candidate targets in the candidate target elevations, and then extract the target to be detected.

[0182] As an optional implementation of this application, it may also include:

[0183] The prediction module is configured to iterate through the tracking list and predict the position of the tracked targets in the tracking list at the current time.

[0184] The tracking and matching module is configured to match the tracking target's current position with the detected target at the current time, update the tracking target's lifecycle, and determine the corresponding tracking type.

[0185] The output module is configured to perform continuous multi-frame verification on the tracked target after the tracking type is determined, and output the result after meeting the preset output conditions.

[0186] Example 3

[0187] In another aspect, this application provides an electronic device, comprising:

[0188] processor;

[0189] Memory used to store processor-executable instructions;

[0190] The processor is configured to implement the elevation target detection method based on elevation curves as described above when executing executable instructions.

[0191] This disclosure discloses an electronic device including a processor and a memory for storing processor-executable instructions. The processor is configured to implement any of the aforementioned elevation curve-based elevation target detection methods when executing the executable instructions.

[0192] It should be noted here that the number of processors can be one or more. Furthermore, the electronic device in this embodiment may also include input devices and output devices. The processor, memory, input devices, and output devices can be connected via a bus or other means, without specific limitations herein.

[0193] As a computer-readable storage medium, the memory can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the elevation target detection method based on elevation curves in this disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.

[0194] Input devices can be used to receive input digital numbers or signals. These signals can be key signals related to user settings and function control of the device / terminal / server. Output devices can include display devices such as screens.

[0195] Example 4

[0196] In another aspect, this application provides a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the elevation target detection method based on elevation curves as described above.

[0197] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting an elevation target based on an elevation profile, characterized by, The method comprises the following steps: acquiring a wheel track region of interest, extracting parallax points in a parallax map, and generating an elevation curve as original elevation data after processing according to a preset preprocessing mode; screening target signals based on the original elevation data, and determining candidate target elevations using the target signals; traversing the candidate target elevations, screening candidate targets in the candidate target elevations, and extracting a detection target; the step of screening target signals based on the original elevation data, and determining candidate target elevations using the target signals comprises: performing height threshold screening on the original elevation data according to a height threshold to obtain candidate target signals; screening the candidate target signals according to a preset length range to obtain target signals; traversing the target signals to search for corresponding target boundaries; taking corresponding original elevation data as elevation values according to a boundary range of the target boundary to obtain candidate target elevations; the step of screening the candidate target signals according to a preset length range to obtain target signals comprises: acquiring two adjacent candidate target signals with an interval length meeting a preset interval size, denoted as adjacent candidate signals; determining whether the two adjacent candidate signals are adjacent homodirectional signals, wherein the adjacent homodirectional signals are two adjacent candidate signals with the same elevation direction; if yes, supplementing the elevation height between the adjacent homodirectional signals, and taking the supplemented adjacent homodirectional signals as one target signal; after determining the detection target, the method further comprises: calculating target attributes including target parallax density and road surface parallax density according to the detection target, wherein the calculation method of the target parallax density is as follows: calculating corresponding initial parallax according to the distance Z and length L of the detection target, substituting the initial parallax into a ground equation to obtain the pixel bottom and top of the detection target on the image, expanding 3 pixels upward and downward respectively, combining a wheel ROI to obtain a target ROI, calculating the number of parallax points in the target ROI, and dividing the number by the pixel area of the target ROI to obtain the target parallax density; the calculation method of the road surface parallax density is as follows: taking 5 longitudinal pixel distances above and below the target ROI, combining a wheel ROI to obtain a road surface ROI, calculating the number of parallax points in the road surface ROI, and dividing the number by the pixel area of the road surface ROI to obtain the road surface parallax density.

2. The elevation profile-based elevation object detection method of claim 1, wherein, before performing height threshold screening on the original elevation data according to a height threshold to obtain candidate target signals, the method further comprises: performing window smoothing on the original elevation data to obtain smoothed elevation data; subtracting the original elevation data from the smoothed elevation data to obtain target reference elevation data; performing height threshold screening on the target reference elevation data according to the height threshold to obtain the candidate target signals.

3. The elevation profile-based elevation object detection method of claim 1, wherein, the step of traversing the target signals to search for corresponding target boundaries comprises: determining an elevation end position; starting from the elevation end position, performing boundary search on positive elevation and negative elevation in the target signals respectively according to a preset search logic to determine corresponding target boundaries. The target boundary includes a positive elevation proximal boundary, a positive elevation distal boundary, a secondary elevation proximal boundary, and a negative elevation distal boundary.

4. The elevation profile-based elevation object detection method of claim 1, wherein, The traversing the candidate target elevations and screening the candidate targets in the candidate target elevations to extract the detection targets comprises: The traversing the candidate target elevations and screening each target in the candidate target elevations according to a height threshold and a width threshold; Determining a spacing length between two adjacent targets in the screened candidate target elevations; When the spacing length is greater than or equal to a preset distance, the two adjacent targets are marked as one detection target.

5. The elevation profile-based elevation object detection method of claim 4, wherein, The traversing the candidate target elevations and screening the candidate targets in the candidate target elevations to extract the detection targets further comprises: When the spacing length is less than the preset distance, the two adjacent targets are determined as adjacent targets and are marked with the same group number; The adjacent targets are marked as one detection target.

6. The elevation profile-based elevation object detection method of claim 5, wherein, After the spacing length is less than the preset distance and the two adjacent targets are determined as adjacent targets and are marked with the same group number, the method further comprises: Traversing each group of adjacent targets, calculating a screening index of a binocular disparity V map of each target in the group; Screening the adjacent targets according to the screening index; If the screening index of a target in the group is the maximum in the group or is greater than a screening threshold, the corresponding target is retained as a detection target.

7. The elevation profile-based elevation object detection method of claim 6, wherein, The screening index comprises at least one of an area of a target V map and an average height.

8. An apparatus for implementing the elevation profile-based elevation object detection method of any one of claims 1-7, wherein, The method comprises: A data preprocessing module configured to acquire a wheel track region of interest, extract disparity points in a disparity map, and process the disparity points according to a preset preprocessing manner, and generate an elevation curve as original elevation data; A candidate target elevation determination module configured to screen target signals based on the original elevation data, and determine candidate target elevations using the target signals; A detection target determination module configured to traverse the candidate target elevations, screen candidate targets in the candidate target elevations, and extract detection targets.

Images