A reference point determination method, device and equipment for tracking a target and a storage medium

Abstract

The application provides a reference point determination method and device for tracking a target and a storage medium. Information of a target frame of the tracked target on a laser radar and a boundary line of a field of view angle of the laser radar are acquired, wherein the laser radar is arranged on a movable carrier such as a vehicle or the like. Then, a warning area is determined based on the information of the target frame and the boundary line, wherein the warning area is at least partially located in a coverage range of the field of view angle within the boundary line. When the target frame enters the warning area, an observation point of the laser radar is re-determined in the range of the field of view angle. A point on the target frame closest to the observation point can be taken as a new tracking reference point of the target frame. The application can make the detection frame more stable, and thus the tracking result is more stable and reliable than that of the original method.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and more specifically to a method, apparatus, device, and storage medium for determining reference points for tracking targets. Background Technology

[0002] LiDAR used in target tracking can be broadly categorized into three types: mechanically rotating, semi-solid-state, and solid-state. Mechanically rotating LiDAR boasts a 360° FOV (field of view), offering a wider field of view. However, considering practical costs and implementation complexity, solid-state or semi-solid-state LiDARs are typically installed in mass-produced vehicles, with these types having a limited horizontal FOV, such as 60° or 110°. Target tracking usually uses a point within the target bounding box as a reference point. This reference point is chosen from nearby points with good point cloud observation within the target area. A well-observed point cloud is fundamental for stable detection. Occluded targets inevitably change size with position, while targets located in areas with good point cloud quality will not experience significant changes in the vertices of their bounding boxes.

[0003] Due to the rectilinear propagation characteristics of lasers, point cloud data is denser in the area of ​​the target closer to the lidar. Therefore, the point closest to the lidar is usually chosen as the reference point for tracking. However, when the target is located at the lidar's boundary, some areas are in the blind zone with no point cloud data, while others are within the visible range with stable point cloud data. In this case, the reference point closest to the lidar is now in the blind zone and lacks a well-observed point cloud, making it unsuitable as a reference point. A new reference point needs to be selected within the visible range, near a location with a good point cloud for tracking. However, because the target is now in the blind zone, the original reference point selection method is no longer effective, leading to highly unstable detection results. Summary of the Invention

[0004] To address the problem in existing technologies that cannot select reliable reference points for tracking targets with blind spots, this invention provides a method, apparatus, and storage medium for determining reference points for tracking targets, which features more accurate and reliable selected reference points.

[0005] A method for determining reference points for tracking targets, according to a specific embodiment of the present invention, includes:

[0006] The information of the target bounding box of the tracked target on the lidar and the boundary line of the lidar's field of view are obtained. The lidar is mounted on a movable vehicle.

[0007] The warning area is determined based on the information of the target box and the boundary line, and the warning area is at least partially located within the coverage area of ​​the field of view within the boundary line;

[0008] When the target box enters the warning area, the observation point of the lidar is redefined within the field of view.

[0009] The point on the target bounding box that is closest to the observation point is used as the new tracking reference point of the target bounding box.

[0010] Furthermore, the boundary line includes a first boundary line and a second boundary line, the first boundary line and the second boundary line are respectively located on both sides of the center line of the field of view, the warning area is polygonal in shape, the first side of the polygon is parallel to or coincides with the center line, and the range defined by the second side adjacent to the first side and the first side includes at least the first boundary line or the second boundary line.

[0011] Further, the polygon includes parallelograms and trapezoids, and the target bounding box information includes: the size and orientation of the target bounding box, and the speed of the tracked target relative to the vehicle. Determining the warning area based on the target bounding box information and the boundary line includes:

[0012] The length of the first side of the quadrilateral is determined based on the size and orientation of the target box, the speed of the tracked target relative to the vehicle, and the type of the tracked target;

[0013] The length of the second side is determined based on the length range of the first boundary line or the second boundary line.

[0014] Further, determining the length of the first side of the quadrilateral based on the size and orientation of the target box, the speed of the tracked target relative to the vehicle, and the type of the tracked target includes:

[0015] based on

[0016] area_length=(target.lenght×cos(target.ori)+target.width×sin(target.ori))×γ×target.vel

[0017] The length of the first side is determined, area_length is the length of the first side, target.length is the length of the target box, target.ori is the direction of the target box, target.width is the width of the target box, target.velo is the velocity of the tracked target relative to the vehicle, and γ is a constant characterizing the type of the tracked target.

[0018] Furthermore, the step of re-determining the observation point of the lidar within the field of view when the target box enters the warning area includes:

[0019] based on

[0020] x pl =x rl +area_length×α+|target.y|×β

[0021] y pl =y rl

[0022]

[0023] Obtain the coordinates of the observation point, x pl Let x be the x-coordinate of the observation point. rl Let y be the original abscissa of the lidar. pl Let y be the ordinate of the observation point. rl Let y be the original ordinate of the lidar, α and β are both coefficients proportional to the velocity of the tracked target relative to the vehicle, and |target.y| is the vertical distance of the center point of the target box relative to the center line.

[0024] Furthermore, the step of re-determining the observation point of the lidar within the field of view when the target box enters the warning area includes:

[0025] Obtain the intersection area between the target box and the warning area. If the intersection area is greater than 0, determine that the target box has entered the warning area.

[0026] Furthermore, the number of warning zones is at least two, and the warning zones are evenly distributed on both sides of the center line.

[0027] A reference point determination device for tracking a target, according to a specific embodiment of the present invention, includes:

[0028] The parameter acquisition module is used to acquire information about the target bounding box of the tracked target on the lidar and the boundary line of the lidar's field of view. The lidar is mounted on a movable vehicle.

[0029] The range determination module is used to determine a warning area based on the information of the target box and the boundary line, wherein the warning area is at least partially located within the coverage area of ​​the field of view within the boundary line;

[0030] An observation point determination module is used to redetermine the observation point of the lidar within the field of view when the target frame enters the warning area; and

[0031] The reference point determination module is used to select the point on the target bounding box that is closest to the observation point as the new tracking reference point of the target bounding box.

[0032] An apparatus according to a specific embodiment of the present invention includes: a memory and a processor;

[0033] The memory is used to store programs;

[0034] The processor is used to execute the program to implement the various steps of the reference point determination method for tracking targets as described above.

[0035] According to a specific embodiment of the present invention, a storage medium is provided thereon storing a computer program, which, when executed by a processor, implements the various steps of the reference point determination method for tracking targets as described above.

[0036] The target tracking reference point determination method provided by this invention can acquire information about the target bounding box of the tracked target on a lidar radar and the boundary line of the lidar's field of view, wherein the lidar is mounted on a movable vehicle, such as a car. Then, a warning area is determined based on the target bounding box information and the boundary line, wherein the warning area is at least partially located within the coverage area of ​​the field of view within the boundary line. When the target bounding box enters the warning area, the observation point of the lidar is re-determined within the field of view. The point on the target bounding box closest to the observation point can be used as a new tracking reference point for the target bounding box. This target tracking reference point determination method, when determining that the target is already within the warning area, selects a reference point using the determined observation point, ensuring that the selected reference point still has good and stable point cloud data. Therefore, the detection box at the corresponding position is also more stable, resulting in a more stable and reliable tracking result compared to previous methods. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This is a flowchart of a method for determining reference points of a tracking target according to an exemplary embodiment;

[0039] Figure 2 This is a positional map of reference points within a target frame provided according to an exemplary embodiment;

[0040] Figure 3It is a point cloud data and reference point location map provided according to an exemplary embodiment;

[0041] Figure 4 This is a location map of the warning area provided according to an exemplary embodiment;

[0042] Figure 5 It is a reference point map obtained according to an existing reference point determination method provided in an exemplary embodiment;

[0043] Figure 6 It is a location map of the determined reference points provided according to an exemplary embodiment;

[0044] Figure 7 This is a structural diagram of a reference point determination device for tracking a target, provided according to an exemplary embodiment.

[0045] Figure 8 This is a structural diagram of a device provided according to an exemplary embodiment. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Reference Figure 1 As shown, an embodiment of the present invention provides a method for determining reference points for tracking targets, which may include the following steps:

[0048] 101. Acquire information about the target bounding box of the tracked target on the lidar and the boundary lines of the lidar's field of view, wherein the lidar is mounted on a moving vehicle, such as a car or electric vehicle. The boundary lines include a first boundary line and a second boundary line, which are located on either side of the center line of the field of view.

[0049] Referring to Figure 2, existing target tracking technologies mostly use a single point in the target bounding box as a reference point to track the target. Generally, a target bounding box contains nine reference points. During tracking, the reference points used are often selected from those near the target where the point cloud data is best observed. Well-observed point cloud data is the foundation for stable detection. Occluded targets inevitably change size with position, while reference points located in areas with good point cloud quality do not experience significant changes in the vertices of the bounding box. Figure 3As shown, although the two target boxes are different in size, the well-observed point cloud 301 is located at the upper left corner of the target box, that is, the 0th reference point is a better reference point for tracking.

[0050] Due to the rectilinear propagation characteristics of lasers, the point cloud data is denser closer to the lidar on the target. Therefore, most existing methods select the point closest to the lidar as the reference point for tracking. However, when the target is at the boundary, part of it is in the blind zone with no point cloud data, while another part is within the visible range with stable point cloud data. In this case, the reference point closest to the lidar is already in the blind zone and lacks a well-observed point cloud, making it unsuitable as a reference point. It is necessary to select a reference point within the visible range, near a location with a good point cloud for tracking. Because the target is already in the blind zone, the original reference point selection method cannot find the optimal one.

[0051] 102. Determine the warning area based on the target box information and the boundary line, wherein the warning area is at least partially located within the field of view coverage area of ​​the boundary line.

[0052] Due to relative velocity during motion, the time it takes for a target to completely enter the blind zone from within the field of view is very short. For tracking stability, an optimal reference point needs to be determined before the target approaches its boundary. Based on this consideration, the boundary range of the warning area 402 can be set using information from each target bounding box 401. The warning area can be polygonal, where the first side of the polygon is parallel to or coincides with the center line, and the range defined by the second side adjacent to the first side includes at least the first or second boundary line. The second side is parallel to or coincides with the first or second boundary line, or intersects the first or second boundary line. The length of the second side is determined based on the length range of the first or second boundary line, where the length of the second side is less than or equal to the length of the first or second boundary line. (Refer to...) Figure 4 The boundary lines of the lidar shown can be two boundary lines 404 of the field of view. These two boundary lines 404 form the first boundary line on the left and the second boundary line on the right, respectively. The first and second boundary lines are located on either side of the center line 405 of the field of view. The warning area can be a regular quadrilateral, where the first side is parallel to or coincides with the center line, the third side 403 opposite to the first side is parallel to the first side, and the second side can coincide with either the first or second boundary line. (Refer to...) Figure 4 As shown, two or four equally even-numbered warning zones can be symmetrically set along the center line of the field of view, with the first side coinciding with the center line 405 and the third side 403 parallel to the center line 405. Parallelograms and trapezoids are commonly used as the shapes of the warning zones. However, in practical applications, the shape of the warning zones can also be other shapes such as fan-shaped or triangular; this invention does not impose any limitations on these shapes.

[0053] Taking the parallelogram-shaped warning area as an example, the target box information includes the length, width, height, direction, and speed relative to the vehicle of the reference point. This information allows us to determine the warning area of ​​the LiDAR within its field of view. Given the LiDAR's field of view angle as θ, the lateral detection distance as W, the length of the short side (first side) of the parallelogram (area_length), and the LiDAR's installation position relative to the vehicle coordinate system as the origin, the long side (second side) of the parallelogram can be set to coincide with either the first or second boundary line. This allows us to determine the coordinates of the four vertices of the parallelograms on both sides. Using two parallelograms to represent two warning areas, it can be seen that once the length of the short side of the parallelogram is obtained, the specific shape of the parallelogram is determined. The formula for calculating the length of the first side, based on the vehicle coordinate system, is as follows:

[0054] area_length=(target.lenght×cos(target.ori)+target.width×sin(target.ori))×γ×target.velo

[0055] Where area_length is the length of the first side, target.length is the length of the target bounding box, target.ori is the direction of the target bounding box, target.width is the width of the target bounding box, target.velo is the speed of the target bounding box relative to the LiDAR vehicle, and γ is a constant characterizing the type of the target bounding box. γ can be a coefficient related to the category. For example, when the category is car, γ = 2, and when the category is turkey, γ = 3.

[0056] 103. When the target frame enters the warning area, the observation point of the lidar is redefined within the field of view.

[0057] During the target bounding box movement, if the intersection area between the target bounding box and the warning area is greater than 0, it can be determined that the target bounding box has entered the warning area. The calculation of the intersection area can be performed using existing functions, which will not be elaborated here. Based on the target bounding box information, the length of the first side (area_length), and the original coordinates of the LiDAR origin (x... rl y rl Re-determine the coordinates (x, y) of the observation point of the lidar. pl y pl ), where x pl The original x-coordinate of the lidar rl Add the length of the first side multiplied by a coefficient α related to velocity, plus the target's lateral distance |target.y| multiplied by a coefficient β related to velocity. The y-value of the redefined observation point is then calculated. pl Coordinates and the original ordinate y of the lidarrl Consistent. The specific calculation formula is as follows:

[0058] x pl =x rl +area_length×α+|target.y|×β

[0059] y pl =y rl

[0060]

[0061] x pl x is the x-coordinate of the newly determined observation point. rl Let y be the original x-coordinate of the lidar. pl Let y be the ordinate of the newly determined observation point. rl Here, y represents the original vertical coordinate of the lidar, α and β are coefficients proportional to the relative speed of the vehicle, and |target.y| is the lateral distance from the center point of the target box to the center line, which is also the vertical distance.

[0062] 104. Use the point on the target bounding box that is closest to the observation point as the new tracking reference point for the target bounding box.

[0063] After obtaining the new origin coordinates of the LiDAR, the distances from all reference points of the target to the observation point are calculated, and the closest point is selected as the final tracking reference point. Reference 5 shows the reference points of the target bounding box determined based on the existing reference point determination method. It can be seen that during the movement of the vehicle and the target bounding box, the reference point closest to the original LiDAR coordinate origin 501 is already outside the LiDAR boundary line, i.e., within the blind zone of recognition, and the corresponding detection box is unstable. Therefore, when tracking is performed using this point, the tracking result will change with the change in the detection result, resulting in a highly unstable effect. Reference 6 shows... Figure 6 As shown, the coordinates of the observation point 601 determined by this method are still within the field of view of the lidar, and the point cloud data is good. Therefore, the detection box at the corresponding position is more stable, and the tracking result is more stable than the original method.

[0064] Based on the same design concept, and referring to Figure 7 As shown, embodiments of the present invention also provide a reference point determination device for tracking targets. This device, when in operation, can implement the various steps of the above-described reference point determination method for tracking targets. The device may include:

[0065] The parameter acquisition module 701 is used to acquire information about the target bounding box of the tracked target on the lidar and the boundary line of the lidar's field of view. The lidar is mounted on a movable vehicle.

[0066] The range determination module 702 is used to determine the warning area based on the target box information and the boundary line, wherein the warning area is at least partially located within the coverage area of ​​the field of view within the boundary line.

[0067] The observation point determination module 703 is used to re-determine the observation point of the lidar within the field of view when the target frame enters the warning area; and

[0068] The reference point determination module 704 is used to take the point on the target box that is closest to the observation point as the new tracking reference point of the target box.

[0069] The reference point determination device for the tracking target has the same beneficial effects as the reference point determination method for the tracking target described above. Its specific implementation can be referred to the embodiments of the reference point determination method for the tracking target described above, and will not be repeated here.

[0070] Reference Figure 8 As shown, embodiments of the present invention also provide a device, which may include a memory 801 and a processor 802.

[0071] The memory 801 is used to store the program.

[0072] Processor 802 is used to execute the program to implement the various steps of the reference point determination method for tracking targets as described in the above embodiment.

[0073] According to a specific embodiment of the present invention, a storage medium is provided thereon storing a computer program. When the computer program is executed by a processor, it implements the various steps of the reference point determination method for tracking targets as described in the above embodiment.

[0074] An embodiment of the present invention also provides a vehicle equipped with a reference point determination device for tracking a target as described in the above embodiment, the device being operable in the vehicle's electronic control unit.

[0075] The target tracking reference point determination method, apparatus, related equipment and vehicle provided in the above embodiments of the present invention, by setting the warning area of ​​the lidar, determine that the target is within the boundary range, and select the reference point closest to the observation point as the tracking reference point. At this time, the selected reference point still has good and stable point cloud data, so the detection box at the corresponding position is also relatively stable. Therefore, the tracking result is more stable than the original method.

[0076] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0077] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0078] The steps in the methods of the various embodiments of the present invention can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in the various embodiments can be replaced or combined.

[0079] The modules and sub-modules in the various embodiments of the present invention can be merged, divided, and deleted according to actual needs.

[0080] In the embodiments provided by this invention, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0081] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0082] Furthermore, the functional modules or sub-modules in the various embodiments of the present invention can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0083] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0084] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0085] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining reference points for tracking a target, characterized in that, include: The system acquires information about the tracked target on the lidar, information about the target bounding box of the tracked target, and the boundary line of the lidar's field of view, wherein the lidar is mounted on a movable vehicle. A warning area is determined based on the information of the tracked target, the information of the target bounding box, and the boundary line, wherein the warning area is at least partially located within the coverage area of ​​the field of view within the boundary line; The information of the tracked target includes the speed of the tracked target relative to the vehicle, and the information of the target frame includes: the size and orientation of the target frame; When the target box enters the warning area, the observation point of the lidar is redefined within the field of view. The point on the target bounding box that is closest to the observation point is used as the new tracking reference point of the target bounding box.

2. The method according to claim 1, characterized in that, The boundary line includes a first boundary line and a second boundary line, which are located on both sides of the center line of the field of view. The warning area is polygonal in shape, with the first side of the polygon parallel to or coincident with the center line. The area defined by the second side adjacent to the first side and the first side includes at least the first boundary line or the second boundary line.

3. The method according to claim 2, characterized in that, The polygon shape includes parallelograms and trapezoids. Determining the warning area based on the information of the tracked target, the target bounding box information, and the boundary line includes: The length of the first side of the quadrilateral is determined based on the size and orientation of the target box, the speed of the tracked target relative to the vehicle, and the type of the tracked target; The length of the second side is determined based on the length range of the first boundary line or the second boundary line.

4. The method according to claim 3, characterized in that, Determining the length of the first side of the quadrilateral based on the size and orientation of the target box, the speed of the tracked target relative to the vehicle, and the type of the tracked target includes: based on Determine the length of the first side. Let be the length of the first side. The length of the target bounding box. The direction of the target box. The width of the target bounding box. The speed of the tracked target relative to the vehicle. A constant characterizing the type of the tracked target.

5. The method according to claim 4, characterized in that, When the target frame enters the warning area, the observation point of the lidar is redefined within the field of view, including: based on ； ； ； ； Obtain the coordinates of the observation point. Let x be the x-coordinate of the observation point. The original abscissa of the lidar is... Let be the ordinate of the observation point. The original ordinate of the lidar is [reference needed]. and All are coefficients proportional to the velocity of the tracked target relative to the vehicle. The vertical distance between the center point of the target box and the center line.

6. The method according to claim 1, characterized in that, When the target frame enters the warning area, the observation point of the lidar is redefined within the field of view, including: Obtain the intersection area between the target box and the warning area. If the intersection area is greater than 0, determine that the target box has entered the warning area.

7. The method according to any one of claims 2 to 5, characterized in that, The number of warning zones is at least two, and the warning zones are evenly distributed on both sides of the center line.

8. A reference point determination device for tracking a target, characterized in that, include: The parameter acquisition module is used to acquire information about the tracked target on the lidar, information about the target bounding box of the tracked target, and the boundary line of the field of view of the lidar. The lidar is mounted on a movable vehicle. The range determination module is used to determine a warning area based on the information of the tracked target, the information of the target box, and the boundary line, wherein the warning area is at least partially located within the coverage area of ​​the field of view within the boundary line; The information of the tracked target includes the speed of the tracked target relative to the vehicle, and the information of the target frame includes: the size and orientation of the target frame; An observation point determination module is used to redetermine the observation point of the lidar within the field of view when the target frame enters the warning area; and The reference point determination module is used to take the point on the target box that is closest to the observation point as the new tracking reference point of the target box.

9. A device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the steps of the reference point determination method for tracking targets as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the reference point determination method for the tracking target as described in any one of claims 1 to 7.

Images