Target tracking list updating method and device of vehicle, vehicle machine controller and vehicle

Abstract

The invention relates to the technical field of vehicle driving, and discloses a target tracking list updating method and device of a vehicle, a vehicle machine controller and the vehicle. The method comprises the following steps: acquiring multiple pieces of target information of a predicted vehicle surrounding environment in a driving process, and 3D point cloud data of a drivable space point of the surrounding environment on an image observed by an optical camera after being projected in a world coordinate system; performing feature association on the multiple pieces of target information and the 3D point cloud data, and determining observation point information from a drivable space corresponding to each target; inputting observation point information into the error function model, and outputting specific positions and postures of the 3D point cloud on the multiple targets; and performing position and attitude updating on the current tracking list by using the feature observation list to obtain a new target tracking list. According to the scheme, the optical camera is added to perform close-range wide-angle observation on the driving space around the vehicle, the close-range view blind area of the detection camera is made up, and the close-range target observation and positioning precision in the automatic driving scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle driving technology, and in particular to a method and device for updating a target tracking list of a vehicle, a vehicle controller and a vehicle. Background Art

[0002] In the application scenario of vehicle autonomous driving, the input of the target tracking algorithm generally directly depends on the sensor's detection data of the target. When the target is partially blocked or enters a specific sensor blind spot, the target data collected by sensors such as lidar and telephoto cameras is incomplete or inaccurate. In the process of inputting these collected data into the target detection model of the neural network for analysis and processing, there are problems such as failure to identify the target or inaccurate target identification, which will lead to unstable target tracking or target loss, causing risks in the scenario of autonomous driving, and even serious consequences. Summary of the invention

[0003] In view of this, the present invention provides a vehicle target tracking list updating method, device, vehicle controller and vehicle to improve the accuracy of target detection and tracking and solve the problem of inaccurate target tracking and identification or target loss.

[0004] In a first aspect, the present invention provides a method for updating a target tracking list of a vehicle, the method comprising:

[0005] Acquire multiple target information of the vehicle's surrounding environment predicted during driving, as well as 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after being projected in the world coordinate system;

[0006] Performing feature association on the plurality of target information and the 3D point cloud data to determine observation point information from the drivable space corresponding to each target;

[0007] Inputting the observation point information corresponding to each of the targets into the error function model, outputting the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets, and obtaining a feature observation list;

[0008] The feature observation list is used to update the position and posture of the current tracking list containing multiple targets to obtain a new target tracking list.

[0009] In combination with the first aspect, in a possible implementation manner, obtaining 3D point cloud data of drivable space points of the surrounding environment on the image observed by the optical camera after being projected in a world coordinate system includes:

[0010] receiving a set of driving environment photos taken by the optical camera within a period of time, wherein the optical camera is installed at at least one preset position of the vehicle and moves with the vehicle;

[0011] Inputting the set of driving environment photos into a neural network model, and outputting a set of pixel points on the drivable space captured, each pixel point in the set of pixel points being represented by a two-dimensional coordinate;

[0012] The two-dimensional coordinate pixel points in the pixel point set are projected onto a three-dimensional world coordinate system through the vehicle's posture information to obtain the 3D point cloud data.

[0013] In combination with the first aspect, in another possible implementation, projecting the two-dimensional coordinate pixel points in the pixel point set to a three-dimensional world coordinate system through the posture information of the vehicle to obtain the 3D point cloud data includes:

[0014] Acquire the position and posture information of the vehicle collected by at least one sensor, and the position and posture information of the optical camera;

[0015] Performing time synchronization on the posture information of the vehicle and the posture information of the optical camera at different times by linear interpolation;

[0016] After time synchronization, all pixel points on the driving environment photos taken at the same time are converted into coordinate systems using the vehicle's position information to generate 3D point cloud data in the three-dimensional world coordinate system.

[0017] In combination with the first aspect, in another possible implementation manner, obtaining a plurality of target information of the vehicle surrounding environment predicted during driving includes:

[0018] After the vehicle is started, an initial target tracking list is generated by calculating the targets around the vehicle, wherein the initial target tracking list includes the vehicle and multiple targets, and bounding box information corresponding to the vehicle and each of the targets;

[0019] Each target in the initial target tracking list is predicted according to the bounding box information, the state and the motion trend of each target, to obtain a second target tracking list, wherein the second target tracking list includes multiple targets and target information corresponding to the multiple targets.

[0020] In combination with the first aspect, in another possible implementation, the performing feature association on the multiple target information and the 3D point cloud data to determine the observation point information from the drivable space corresponding to each target includes:

[0021] The target information corresponding to the multiple targets, the bounding box information corresponding to each target and the 3D point cloud data are associated according to a proximity association algorithm to determine whether each 3D point cloud data is located on the bounding box corresponding to one or more targets, and the 3D point cloud data corresponding to the bounding box of each target on the second target tracking list is obtained.

[0022] In combination with the first aspect, in another possible implementation, the inputting the observation point information corresponding to each of the targets into the error function model, and outputting the specific position and posture of each 3D point cloud observed in the drivable space on the multiple targets, includes:

[0023] The ICP error function model of iterative closest point algorithm is constructed by using the hybrid matching method of point-point and point-line.

[0024] Processing the observation point information through the ICP error function model to obtain the specific position and posture of each 3D point cloud on the multiple targets;

[0025] Wherein, the ICP error function model is expressed as:

[0026]

[0027] Among them, J(R,t) represents the ICP error function model, R is the rotation matrix, t is the translation vector, p i is any observation point in the drivable space, q i For p i Corresponding to the matching points on the body frame, N is the total number of 3D points that are closest to the bounding box surface or edge, M is the total number of 3D points that are closest to the bounding box vertex, n j is the normal vector of the edge where the matching point j is located.

[0028] In combination with the first aspect, in another possible implementation, using the feature observation list to update the position and posture of a current tracking list containing multiple targets to obtain a new target tracking list includes:

[0029] The Kalman gain is calculated for the specific position and the posture of each 3D point cloud in the feature observation list and the observation data in the tracking list currently containing multiple targets, and the specific position and the posture in the feature observation list are corrected to obtain the new target tracking list.

[0030] In a second aspect, the present invention provides a target tracking list updating device for a vehicle, the device comprising:

[0031] An acquisition module is used to acquire multiple target information of the vehicle's surrounding environment predicted during driving, as well as 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after being projected in the world coordinate system;

[0032] A determination module, used for performing feature association between the plurality of target information and the 3D point cloud data, and determining observation point information from the drivable space corresponding to each target;

[0033] A processing module, used for inputting the observation point information corresponding to each of the targets into an error function model, outputting the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets, and obtaining a feature observation list;

[0034] The updating module is used to use the feature observation list to update the position and posture of the current tracking list containing multiple targets to obtain a new target tracking list.

[0035] In a third aspect, the present invention provides a vehicle controller, comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the vehicle target tracking list updating method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the vehicle target tracking list updating method of the first aspect or any corresponding embodiment thereof.

[0037] In addition, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the vehicle target tracking list updating method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0038] In a fifth aspect, the present invention provides a vehicle, comprising a vehicle controller and an optical camera, wherein the optical camera is installed on the vehicle, wherein the optical camera is used to take a group of driving environment photos during a period of time when the vehicle is driving, and send the group of driving environment photos to the vehicle controller;

[0039] The vehicle controller is used to execute the vehicle target tracking list updating method of the first aspect or any corresponding embodiment thereof after receiving the group of driving environment photos.

[0040] In conjunction with the fifth aspect, in a possible implementation manner, a sensor set is further installed on the vehicle, wherein the sensor set includes at least one of a laser radar sensor, a telephoto or wide-angle camera, an accelerometer, and a gyroscope sensor;

[0041] At least one sensor in the sensor set is used to collect driving environment information and / or vehicle speed information during the driving of the vehicle, and report it to the vehicle controller.

[0042] The present invention provides a method, device, vehicle controller and vehicle for updating a target tracking list of a vehicle. On the basis of the original sensor collecting the surrounding environment information of the vehicle, an observation image collected by an optical camera is added, and the 3D point cloud data of the drivable space collected by the optical camera is feature-associated with multiple target information, and the observation point information from the drivable space corresponding to each target is determined, so as to realize the mapping from image data to actual physical space, and use the observation point information to correct the specific position and posture of multiple targets on the drivable space, and update the target tracking list generated by general sensor tracking and collecting the surrounding environment. Since the observation range of the optical camera is limited to a few meters near the vehicle, there is no target detection task, only the drivable space and parking space detection tasks serving parking. In addition, increasing the optical camera's close-range wide-angle observation of the drivable space around the vehicle can make up for the close-range field of view blind spot of the target detection camera, form a certain degree of complementarity, and improve the accuracy of close-range target observation and positioning in the automatic driving scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a schematic diagram of a new target tracking list updating method provided according to an embodiment of the present invention;

[0045] Figure 2 is a flow chart of a method for updating a target tracking list of a vehicle provided according to an embodiment of the present invention;

[0046] Figure 3 is a visual schematic diagram of vehicles in a target tracking list provided according to an embodiment of the present invention;

[0047] Figure 4 is a flow chart of obtaining 3D point cloud data observed by an optical camera according to an embodiment of the present invention;

[0048] Figure 5 is a schematic diagram of a coordinate system conversion provided according to an embodiment of the present invention;

[0049] Figure 6 It is a schematic diagram of solving the specific position of a drivable space point in a target frame according to an embodiment of the present invention;

[0050] Figure 7 is a structural block diagram of a target tracking list updating device provided according to an embodiment of the present invention;

[0051] Figure 8 is a schematic diagram of the hardware structure of a vehicle controller provided according to an embodiment of the present invention;

[0052] Fig. 9 It is a structural schematic diagram of a vehicle provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0054] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be internal communication between two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0055] The technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0056] The technical solution of the embodiment of the present invention is applied to the field of vehicle autonomous driving technology, especially to tracking, locating and identifying targets in the surrounding environment during the vehicle autonomous driving process.

[0057] In order to solve the problem that in the process of collecting surrounding environment photos and information by the sensor cluster on the vehicle, when the target is partially blocked or enters the blind spot of a specific sensor, the target detection model based on the neural network may not be able to recognize the target, which may lead to unstable target tracking or target loss, an embodiment of the present invention provides a target tracking framework that uses drivable space detection as a supplement.

[0058] like Figure 1 As shown, the first part of the method flow involves steps A1 to A5 and steps B1 to B5. The second part of the method flow involves steps C1 to C6 and steps B5 and B6. Among them, the first part of the method flow A1 to A5 is summarized as a traditional algorithm based on direct target detection, which is mainly based on the data collected by the general sensor cluster to locate and track at least one target in the vehicle's surrounding environment.

[0059] Specifically, the method flow includes:

[0060] Step A1: The laser radar sensor, telephoto or wide-angle camera and other devices collect information about the vehicle's surrounding environment. Step A2: Target detection is performed on the vehicle's surrounding environment based on the images, photos and data collected by the sensor cluster to determine the tracking target. Synchronously, the vehicle controller also receives vehicle speed information transmitted by accelerometers, gyroscope sensors, wheel speed meters and other devices, i.e., step B1, and step B2: the posture information collected by each sensor is input into the vehicle controller.

[0061] Step A3: The vehicle controller synchronizes the time and coordinate axis of the relevant information of the images / photos received by the laser radar and telephoto camera with the vehicle acceleration, speed and other information collected by the sensor to obtain an observation list, i.e., step A4. The observation list contains at least one target in the observed world coordinate system. Step A5: Perform target-level association matching on at least one target in the observation list with the targets in the target tracking list predicted in steps B3 and B4, and use the associated information to update or create the target, i.e., step B5, and finally output a new target tracking list.

[0062] In the above method, due to the blind spots of the telephoto or wide-angle camera or other sensors, or when the target is blocked or partially blocked, the target tracking may be unstable or lost. To solve this problem, the embodiment of the present invention supplements the information collection process of the fisheye camera or fisheye camera, which corresponds to steps C1 to C6.

[0063] Specifically, the newly added method flow includes: Step C1: The fisheye camera around the vehicle body detects and takes pictures of the vehicle's surrounding environment to obtain image pixel information of the drivable space, that is, step C2, and these image pixel information can be represented by pixel values. Then step C3: The pixel information of the drivable space is synchronized with the posture information collected by the accelerometer and gyroscope in steps B1 and B2 and the coordinate system is converted to obtain 3D point cloud data in the world coordinate system. Step C4: The converted 3D point cloud data is feature-associated with the bounding box information on the target tracking list predicted in B4, and step C5: target posture inference is obtained. Finally, the target posture inference data of step C5 is supplemented and stored in the feature-level observation list, that is, step C6. Finally, step B5 uses the feature-level observation list supplemented and stored in C6 to update the target tracking list currently collected and generated by the general sensor, and outputs B6: a new target tracking list is obtained.

[0064] This method does not rely on the detection of specific targets in the drivable space, but analyzes and infers the image collected by the entire fisheye camera, and marks the distance to the nearest obstacle in the current column of pixels on each column of pixels on the image.

[0065] In the above step C4, the entire image information is integrated, and the pixel set is divided into the boundaries of the driving space. At the same time, for each column of pixels, the detection network of the drivable space will mark whether the obstacle corresponding to the boundary is a static obstacle or a dynamic obstacle. In the general sense, this kind of fisheye camera can only observe the range of a few meters around the vehicle body, so there is no target detection task, only the drivable space and parking space detection tasks for parking. The fisheye camera's close-range wide-angle observation just makes up for the close-range blind spot of the target detection camera, forming a certain degree of complementarity.

[0066] This method uses the drivable space information supplemented by a close-range fisheye camera installed on the vehicle to screen, cluster and match obstacles, and ultimately uses it to infer and update the target's pose, greatly improving the accuracy of the close-range pedestrian position in parking scenarios.

[0067] The technical solution provided by the embodiment of the present invention is described in detail below.

[0068] An embodiment of the present invention provides an embodiment of a method for updating a target tracking list of a vehicle. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0069] In this embodiment, a method for updating a target tracking list of a vehicle is provided, which can be used in the above-mentioned vehicle controller or vehicle control system. Figure 2 is a flow chart of a method for updating a target tracking list according to an embodiment of the present invention, the method comprising:

[0070] Step S101: Acquire multiple target information of the vehicle's surrounding environment predicted during driving, and 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after being projected into the world coordinate system.

[0071] The multiple target information may be information on a target tracking list generated and predicted by general sensors after measuring the driving environment. The multiple targets include people or objects in the vehicle's surrounding environment; the multiple target information includes the location coordinates, moving speed, size, etc. of at least one target object.

[0072] In this embodiment, the multiple target information is the aforementioned Figure 1 The target tracking list predicted in step B3 and step B4 is shown.

[0073] In addition, in step S101, the optical camera includes but is not limited to a fisheye camera or a surround fisheye camera (referred to as "fisheye camera"). This embodiment takes a fisheye camera as an example, and the fisheye camera is located on the vehicle. The number of fisheye cameras can be one or more, and the position of each fisheye camera on the vehicle can be customized. For example, it is set at the front end or rear end of the vehicle, and one on each side.

[0074] Among them, the input type provided by the fisheye camera is a series of pixels with semantic information, which constitute the drivable boundary of the vehicle (the grounding point of the obstacle). A series of pixels depict the driving boundary of the vehicle. Different colors reflect different categories, including static, dynamic, etc. Combined with the pose information and the measured timestamp, the image points can be transformed into the global coordinate system through inverse perspective, which is used in the subsequent steps to match the targets in the distance scale and the predicted target tracking list to obtain the detection results of the drivable space. Each column of pixels gives a grounding point at the boundary that the vehicle can reach, and gives different attributes according to different types of obstacles.

[0075] In this step, the process of obtaining the 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after projection in the world coordinate system corresponds to the aforementioned Figure 1 The steps C1 to C3 shown specifically include:

[0076] The optical camera collects a set of photos or images of the surrounding environment at the time of shooting and sends them to the vehicle controller.

[0077] The vehicle controller receives and pre-processes the images taken by the fisheye camera, including grayscale conversion and filtering. In addition, it also includes feature extraction and feature point detection, such as using algorithms such as Canny edge detection to identify edge information in the image; and using algorithms such as SIFT, SURF or ORB to detect feature points in the image.

[0078] Then, use the calibration board to calibrate the camera and obtain the camera's intrinsic parameter matrix (focal length, optical center, etc.) and distortion coefficient. Convert pixel coordinates to normalized camera coordinates to obtain intrinsic parameters; extract depth information and convert pixel coordinates to camera coordinates. Combined with depth information, convert normalized camera coordinates to 3D camera coordinates, and convert the camera coordinates to point cloud data in the world coordinate system. The converted 3D world coordinate points are used as point cloud data.

[0079] In this step, the vehicle controller can obtain multiple target information of the vehicle's surrounding environment predicted during driving, which may include other vehicles, pedestrians, obstacles, etc. At the same time, through the images observed by the optical camera, the vehicle controller can extract the drivable space points of the surrounding environment and project these points into 3D point cloud data in the world coordinate system.

[0080] Step S102: feature association is performed on multiple target information and 3D point cloud data to determine observation point information from the drivable space corresponding to each target.

[0081] Specifically, multiple target information includes the predicted vehicle 3D box and other attributes. This information is associated with the 3D point cloud data projected on the drivable space, such as Figure 3 As shown in the figure, the purpose of this step is to find the matching relationship between the observed 3D point cloud data and the 3D box of the vehicle in the predicted target tracking list, so as to obtain the observation point information of each point on the 3D bounding box of the vehicle in the drivable space. The observation point information is generated after the fisheye camera takes a 2D photo or image and converts it into the world coordinate system.

[0082] Optionally, the method for performing the association operation in this step may be to perform the association through a nearest neighbor association algorithm.

[0083] Step S103: Input the observation point information corresponding to each target into the error function model, output the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets, and obtain a feature observation list.

[0084] The error function model is used to obtain the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets, and the specific position and posture information of the corrected 3D point cloud is added to the feature observation list.

[0085] The error function, also known as the error metric function or loss function, is used to quantify the deviation between the system output and a certain standard or expected value. The standard or expected value is usually called the target value or true value. The error function is used to objectively evaluate the performance of the model and guide the optimization and improvement of the model. Optionally, the error function model can be a hybrid ICP error function model.

[0086] The ICP (Iterative Closest Point) algorithm is an algorithm for point cloud registration, whose goal is to find the best transformation between two point clouds so that the error between them is minimized. In the ICP algorithm, the error function is usually defined as the sum of the squares of the distances between the corresponding points of the two sets of point clouds. Through iterative optimization, parameters that are closer to the true transformation are continuously sought until the convergence condition is met.

[0087] In this step, the ICP error function model can be used to degenerate the point-line error into a point-point error, and the solution can be obtained directly through singular value decomposition. In addition, for accurate solution, the rotation matrix R and the translation vector t can be derived and then solved through nonlinear optimization, so as to infer the specific matching position and form of the observations of these drivable space points on the target. Finally, after each pair of matching point clouds and 3D boxes undergoes such a process, the 3D box inferred from the 3D point cloud data is cached in the currently generated target tracking list. At this time, each target corresponds to two sets of state quantities, one set from the observed data and the other set from the predicted data.

[0088] Step S104: Use the feature observation list to update the position and posture of the current tracking list containing multiple targets to obtain a new target tracking list.

[0089] Specifically, one implementation method is to use the Kalman gain algorithm to update and correct the feature observation list output in step S103, for example, to correct the error of the prediction data by the observation amount, so as to obtain an accurate prediction result, and the corrected prediction result includes the corrected speed, position and other information of the target object. The new target tracking list is used to perform self-driving control such as positioning and navigation of the vehicle.

[0090] In this step, the feature observation list is used to update the position and attitude of the current tracking list containing multiple targets to obtain a new target tracking list. This step realizes the continuous tracking and updating of dynamic targets in the surrounding environment, providing key information for the decision-making and control of the autonomous driving system.

[0091] The target tracking list updating method provided in this embodiment adds the observation image collected by the optical camera on the basis of the original sensor collecting the vehicle's surrounding environment information, and performs feature association between the 3D point cloud data of the drivable space collected by the optical camera and multiple target information, determines the observation point information from the drivable space corresponding to each target, realizes the mapping from image data to actual physical space, and uses the observation point information to correct the specific position and posture of multiple targets on the drivable space, and updates the target tracking list generated by general sensor tracking and collecting the surrounding environment. Since the observation range of the optical camera is limited to a few meters around the vehicle, there is no target detection task, only the drivable space and parking space detection task serving parking. In addition, increasing the optical camera's close-range wide-angle observation of the drivable space around the vehicle can make up for the close-range field of view blind spot of the target detection camera, form a certain degree of complementarity, and improve the accuracy of close-range target observation and positioning in the autonomous driving scenario.

[0092] In addition, in a possible implementation of this embodiment, Figure 4 As shown, in step S101, obtaining 3D point cloud data of points of the surrounding environment on the image observed by the optical camera after being projected in the world coordinate system specifically includes:

[0093] Step S101 - 1 : receiving a set of driving environment photos taken by an optical camera within a period of time, where the optical camera is installed at at least one preset position of the vehicle and moves with the vehicle.

[0094] Among them, the optical camera can collect photos or pictures at regular intervals and then report them to the vehicle controller.

[0095] Step S101-2: Input a group of driving environment photos into the neural network model, and output a set of pixel points on the drivable space captured, where each pixel point in the pixel point set is represented by a two-dimensional coordinate.

[0096] Specifically, the vehicle controller inputs the captured photos of the driving environment into the neural network model, and outputs a set of pixel points of the vehicle in the drivable space at a set of shooting times. Each pixel point corresponds to a two-dimensional coordinate, for example, the two-dimensional coordinates of each pixel are represented by (x, y) coordinate points.

[0097] Step S101 - 3 : Project the two-dimensional coordinate pixel points in the pixel point set to the three-dimensional world coordinate system through the vehicle's posture information to obtain 3D point cloud data.

[0098] The vehicle's posture information reported by other sensors is synchronized with a set of pixel points at a shooting time in step S101-2 and the coordinates are converted to obtain the posture information corresponding to the pixel points of each photo taken by the optical camera at a shooting time.

[0099] Furthermore, step S101-3 specifically includes: obtaining the vehicle's posture information collected by at least one sensor and the optical camera's posture information; performing time synchronization on the vehicle's posture information and the optical camera's posture information at different times by linear interpolation; after time synchronization, performing coordinate system conversion on all pixel points on the driving environment photo taken at the same time by using the vehicle's posture information to generate 3D point cloud data in a three-dimensional world coordinate system. This step corresponds to the aforementioned embodiment Figure 1 Step C3 time synchronization and coordinate conversion.

[0100] Specifically, assuming that the sampling frequency of the gyroscope or accelerometer is the first frequency f1, the position information of the vehicle collected at the first frequency f1. In step S101-1, the sampling frequency of the optical camera, such as the fisheye camera, is the second frequency f2, and f1≠f2, then the position information of the fisheye camera is first obtained by difference.

[0101] One implementation method is to find matching pose information for the observation data at time tn by linear interpolation. Assume that the pose information input time is {…,t k-1 ,t k , ...}, if t k-1 ≤t n ≤t k , then the pose P(t n ). The formula is:

[0102]

[0103] Among them, P(t n ) is the pose sampled by the fisheye camera at time tn, P(t k+1 ) is the general sensor at t k+1 The pose sampled at the moment, P(t k ) is the general sensor at time t k The sampled pose, t k and t k+1 are the two sampling moments of a general sensor with a frequency of f1. In addition, P(t k )、P(t k+1 ) and other postures are directly measured by general sensors.

[0104] After time synchronization, the measurement data of different coordinate systems must be synchronized, that is, coordinate system conversion, corresponding to the above Figure 1 The purpose of step C3 is to project all 2D points detected in the image layer into the world coordinate system through the pose information of the vehicle. Figure 5As shown, this embodiment predefines the following symbols and multiple Euclidean coordinate systems:

[0105] 2D pixel coordinate system P, used to represent the position of pixels in the image;

[0106] 3D camera coordinate system c, used to represent the camera coordinate system under the projection model;

[0107] The 3D vehicle coordinate system v is a coordinate system with the projection of the center of the vehicle's rear wheel axle on the ground as the center, and the front, left, and top as the positive directions of the three coordinate axes. It is also called the vehicle coordinate system.

[0108] The 3D world coordinate system W is a fixed reference point.

[0109] In addition, the rigid body transformation in the homogeneous coordinate system is defined Among them, the rotation matrix Translation Vector

[0110] From the pose information, we can know the position T of the vehicle in the world coordinate system. wv ; The position of the camera in the vehicle coordinate system can be obtained through the camera installation position (camera external parameter) vc ; Assume that a pixel is [u,v] in the image coordinate system Τ , which is [x v ,y v ,0] Τ , that is, it is assumed that it is at the same level as the vehicle, such as the same grounding point.

[0111] According to the camera model, the calculation is: in is the normalized plane coordinate, similarly u,v are the coordinates of the pixel.

[0112] According to the coordinate system transformation, we get

[0113] The left side of the equation (2) represents the 3D vehicle coordinate system v, and the right side represents the 3D camera coordinate system c. Solve the equation in the third row to find z c , and derive the coordinates P of the point in the world coordinate system accordingly w =[x w ,y w .z w ] Τ .

[0114] In this embodiment, the above-mentioned time synchronization and coordinate system conversion method, by fusing the posture information of the vehicle and the fisheye camera, and generating 3D point cloud data in a three-dimensional world coordinate system, provides support for subsequent feature-level association and target tracking list update, environmental perception, decision-making and planning of the autonomous driving system.

[0115] In a specific implementation of this embodiment, the above step S101, obtaining multiple target information of the vehicle surrounding environment predicted during driving, specifically includes:

[0116] First, after the vehicle starts, the initial target tracking list is generated by calculating the targets around the vehicle. The initial target tracking list includes the vehicle and multiple targets, as well as the bounding box information corresponding to the vehicle and each target. This step corresponds to the above embodiment. Figure 1 Step B3 shown.

[0117] The initial target tracking list includes: at least one target (object) around the vehicle, as well as the speed, position, recording time of each object, and the bounding box information corresponding to the vehicle and each object. Figure 3 , which is a typical visualization diagram of vehicles in a target tracking list, wherein the vehicles in the list have been simplified into 3D boxes (bounding boxes) in the world coordinate system for storage.

[0118] In addition, the initial target tracking list also includes the historical targets that have been created. And the initial target tracking list is cached in the software for output to the regulation and control module for path planning, obstacle avoidance, etc.

[0119] Then, based on the bounding box information, the state and motion trend of each target, each target in the initial target tracking list is predicted to obtain a second target tracking list, which includes multiple targets and target information corresponding to the multiple targets. This step corresponds to the aforementioned embodiment Figure 1 Step B4 shown.

[0120] Specifically, according to the above initial target tracking list, the state of objects at the next moment, such as moment t2, is predicted through Kalman filtering and prediction model update. The initial target tracking list corresponds to the state at moment t1. According to the current state of each object in the initial target tracking list and the movement trend of each object, each target object in the initial target tracking list is predicted, and the prediction algorithm is the Kalman filter prediction algorithm to obtain the second target tracking list. The second target tracking list includes not only the state of each target object, but also the predicted 3Dbox information of the vehicle.

[0121] The following is expressed by the relation:

[0122] Assume that the predicted state vector at time k (which can be time t2) is expressed as The state vector It can be used to predict the position, box size, speed, and orientation of the object at time k. The prediction recursive matrix at time k is expressed as F k The prediction recursive matrix F k It is determined by the differential equations of the physical meaning of each state with respect to time, such as displacement depends on velocity*time, etc. The state vector after the update at time k-1 is expressed as This state vector can be obtained from the initial target tracking list in step B3. k Represents the covariance matrix of the process noise. represents the state quantity covariance matrix after the update at time k-1. The state quantity covariance matrix is ​​used to describe the uncertainty of each state quantity. Represents the covariance matrix of the state quantity at time k after prediction.

[0123] The Kalman filter prediction formula is:

[0124]

[0125] After being processed by the Kalman filter prediction formula and prediction model, the state quantity prediction of the initial target tracking list at time k-1 is predicted to obtain the state quantity covariance matrix at time k, which is used to describe the state quantity of each object at the next moment.

[0126] In another specific implementation of this embodiment, in combination with the second target tracking list obtained by the above prediction, step S102 performs feature association on multiple target information and 3D point cloud data, specifically including: associating the bounding box information corresponding to each target and the 3D point cloud data according to the proximity association algorithm, determining whether each 3D point cloud data is located on the bounding box corresponding to one or more targets, and obtaining the 3D point cloud data corresponding to the bounding box of each target on the second target tracking list. This step corresponds to the aforementioned embodiment. Figure 1 Step C4 shown.

[0127] Furthermore, the predicted vehicle 3D bounding box information and the point cloud data projected in the drivable space are matched to determine the bounding box position corresponding to each point cloud; the observation point information from the drivable space corresponding to each object in the predicted observation list is obtained. The purpose is to find the matching relationship between the observed point cloud and the target 3D box in the existing tracking list.

[0128] The predicted vehicle 3D box and other attributes, as well as the 3D ground point cloud data after projection of the drivable space, are matched and associated to determine whether the drivable space point belongs to the vehicle, where the vehicle is represented by a rectangular box. The dynamic semantic information of obstacles in the drivable space is used to first filter out points belonging to dynamic targets. Then, under the bird's-eye view (BEV), the distance from each point to the rectangular box of all predicted targets within the camera's field of view is calculated. If the distance is less than a certain threshold, the point is considered to be an observation point for this target. After all observation points are screened, multiple observation points from the drivable space corresponding to the target object in each predicted target tracking list are obtained.

[0129] In another specific implementation of this embodiment, the above step S103 inputs the observation point information corresponding to each target into the error function model, and outputs the specific position and posture of each 3D point cloud on multiple targets, which corresponds to step C5 of the above embodiment: the process of target posture inference, such as Figure 6 As shown, specifically including:

[0130] The ICP error function model of iterative closest point algorithm is constructed by using the point-point and point-line hybrid matching method. The observation point information is processed by the ICP error function model to obtain the specific position and posture of each 3D point cloud on multiple targets.

[0131] Among them, the ICP error function model is expressed as:

[0132]

[0133] Among them, J(R,t) represents the ICP error function model, R is the rotation matrix, t is the translation vector, p i is any observation point in the drivable space, q i For p i Corresponding to the matching points on the body frame, N is the total number of 3D points that are closest to the bounding box surface or edge, M is the total number of 3D points that are closest to the bounding box vertex, n j is the normal vector of the edge where the matching point j is located.

[0134] In this embodiment, after the observed point cloud and the corresponding target 3D box are given, the size information of each target object can be obtained. According to the size information of each target object, the specific position and posture of the target observed 3D box can be inferred through the point cloud.

[0135] In a physical sense, what is observed are the grounding points of the obstacle targets. However, since we do not know the specific positions of these points corresponding to the tracking targets, we cannot correct the real positions of the targets through targeted observations. In this embodiment, a hybrid matching method of point-point and point-line is used to construct an ICP error function model to solve the specific positions of these drivable space points in the target frame.

[0136] like Figure 6 As shown in the figure, for the points projected on the frame line segment, the distance from the frame is considered; for the points outside the line segment, the distance from the vertex needs to be calculated. In this way, a mixed ICP error function model can be obtained. Assuming that the drivable space point is p, and its corresponding matching point on the body frame is q, then some points are modeled according to the point-to-point ICP, and we get The other part is modeled according to the point-line ICP, and we get That is, the second term on the right side of the equation, where n j is the normal vector of the edge where the matching point j is located. The error function model J(R, t) of the mixed ICP is obtained through the first and second terms.

[0137] It should be noted that if the point-line error can be further degenerated into a point-point error in order to improve the solution efficiency, the analytical solution can be directly obtained through singular value decomposition; if an accurate solution is required, the rotation matrix R and the translation vector t can be differentiated through nonlinear optimization and then the solution can be obtained.

[0138] After obtaining the rotation and translation, the specific matching position and shape of the observations of these drivable space points on the target can be inferred. If the length, width, and height of the target are assumed to be unchanged, the 3D box that best fits these drivable space points can be restored, and the rectangular box is used as the inferred new observation value Z k .

[0139] The above step S104 uses the feature observation list to update the position and attitude of the current tracking list containing multiple targets to obtain a new target tracking list, which specifically includes: calculating the Kalman gain of the specific position and attitude of each 3D point cloud in the feature observation list and the observation data in the current tracking list containing multiple targets, correcting the specific position and attitude in the feature observation list, and obtaining a new target tracking list. This step corresponds to the above embodiment Figure 1 Step B6 shown.

[0140] After each pair of matched point clouds and 3D boxes is subjected to the above-mentioned method flow of S103, the 3D box inferred from the point cloud is cached into the feature observation list. At this time, each object corresponds to two groups of state quantities, one group from observation and the other group from prediction.

[0141] Furthermore, according to the update formula of the Kalman filter, the Kalman gain is calculated for each target prediction and observation pair, and the prediction error is corrected by the observation value. Whether it is the feature observation list / second target tracking list obtained from step C6, or the target tracking list after target-level association matching output from step A5, in step S104, the observation data and its corresponding covariance matrix are used as input data. The only difference is that for the method flow of steps A1 to A5, each observation value z in the observation list is obtained directly by direct measurement by a general sensor; while the second target tracking list output by the fisheye optical camera after shooting, that is, the observation values ​​obtained in the above steps C1 to C6 are obtained indirectly, and are the results of the prediction inferred according to step S103.

[0142] This step combines the observation data of the above step S101, and after the update in step S104, a new target tracking list is obtained, in which each state quantity is expressed as:

[0143] The Kalman gain matrix K consists of weight values, which describes how to optimally adjust the weights between prediction and observation based on the uncertainty from the prediction and observation. The observation vector z at time k k ; Observation model matrix H k , the H k Dependent on the measurement model. Observation noise R k , R k It is also related to the measurement model; the state vector after the observation at time k is corrected The covariance matrix of the corrected state vector at time k is I represents the identity matrix.

[0144] The Kalman filter update formulas (6), (7), and (8) are expressed as:

[0145]

[0146] It should be noted that the above formulas (6) to (8) are only one way of expressing the Kalman filter update formula, and other expressions may also be included, which are not limited in this embodiment.

[0147] The method provided in this embodiment uses a hybrid ICP algorithm based on boundary discrete points and geometric rectangular boxes to infer and predict the position of the target model, and combines the function of the fisheye camera to use its drivable space detection to supplement the information of the dynamic target tracking module, thereby solving the problems of missed detection and inaccurate detection in the target detection blind area, compensating for the close-range field of view blind area of ​​the target detection camera, and improving the accuracy of close-range target observation and positioning in autonomous driving scenarios.

[0148] Specifically, in the target tracking process, all targets can be described by a rectangular bounding box. The input of traditional target detection is only a rectangular box on a 2D image or in a 3D space. From a top view, it is a 2D rectangular vehicle bounding box. Through a series of preprocessing, including time synchronization and coordinate conversion, the method provided in this embodiment obtains its position and posture in the global coordinate system at the observation time, and passes it into the target tracking list.

[0149] At the same time, at least one target in the target tracking list is predicted by the Bayesian filter for the position of the tracked target at the moment. The predicted target position is matched with the position of the observed target, that is, the targets in the list are screened for feature-level association, and the successfully associated tracked targets are updated with the newly input observation information; if the input observation target is not in the current target tracking list, the target will be created in the new target tracking list, thereby making up for the close-range blind spot of the target monitoring camera and improving the accuracy of close-range target observation and positioning in autonomous driving scenarios.

[0150] In this embodiment, a target tracking list updating device for a vehicle is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0151] This embodiment provides a target tracking list updating device for a vehicle, such as Figure 7 As shown, the device includes: an acquisition module 710, a determination module 720, a processing module 730 and an update module 740. In addition, the device may also include other more or fewer modules or units, such as a sending module, a storage module, etc., which is not limited in this embodiment.

[0152] Among them, the acquisition module 710 is used to obtain multiple target information of the vehicle's surrounding environment predicted during driving, as well as 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after projection in the world coordinate system.

[0153] The determination module 720 is used to perform feature association on multiple target information and 3D point cloud data to determine the observation point information from the drivable space corresponding to each target.

[0154] The processing module 730 is used to input the observation point information corresponding to each target into the error function model, output the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets, and obtain a feature observation list.

[0155] The updating module 740 is used to update the position and posture of the current tracking list containing multiple targets by using the feature observation list to obtain a new target tracking list.

[0156] In some optional embodiments, the device includes a transceiver module for receiving a group of driving environment photos taken by an optical camera within a period of time, and the optical camera is installed at at least one preset position of the vehicle and moves with the vehicle.

[0157] The acquisition module 710 is specifically used to input a group of driving environment photos into the neural network model, and output a set of pixel points on the drivable space captured, where each pixel point in the pixel point set is represented by a two-dimensional coordinate; the two-dimensional coordinate pixel points in the pixel point set are projected to the three-dimensional world coordinate system through the vehicle's posture information to obtain 3D point cloud data.

[0158] In some other optional embodiments, the acquisition module 710 is specifically used to obtain the vehicle's posture information collected by at least one sensor, and the posture information of the optical camera; the vehicle's posture information and the optical camera's posture information at different times are time synchronized by linear interpolation; after time synchronization, all pixel points on the driving environment photos taken at the same time are converted into coordinate systems by the vehicle's posture information to generate 3D point cloud data in a three-dimensional world coordinate system.

[0159] In some other optional embodiments, the acquisition module 710 is further specifically used to obtain an initial target tracking list generated by calculating targets around the vehicle after the vehicle is started, wherein the initial target tracking list includes the vehicle and multiple targets, as well as bounding box information corresponding to the vehicle and each target.

[0160] The determination module 720 is further used to predict each target in the initial target tracking list according to the bounding box information, the state and the motion trend of each target, and obtain a second target tracking list, wherein the second target tracking list includes multiple targets and target information corresponding to the multiple targets.

[0161] In some other optional embodiments, the determination module 720 is specifically used to associate the bounding box information corresponding to each target and the 3D point cloud data in the target information corresponding to multiple targets according to a proximity association algorithm, determine whether each 3D point cloud data is located on the bounding box corresponding to one or more targets, and obtain the 3D point cloud data corresponding to the bounding box of each target on the second target tracking list.

[0162] In some other optional embodiments, the processing module 730 is specifically used to construct an iterative closest point algorithm ICP error function model using a mixed matching method of point-point and point-line; the observation point information is processed by the ICP error function model to solve the specific position and posture of each 3D point cloud on multiple targets.

[0163] Among them, the ICP error function model is expressed as:

[0164]

[0165] Among them, J(R,t) represents the ICP error function model, R is the rotation matrix, t is the translation vector, p i is any observation point in the drivable space, q i For p i Corresponding to the matching points on the body frame, N is the total number of 3D points that are closest to the bounding box surface or edge, M is the total number of 3D points that are closest to the bounding box vertex, n j is the normal vector of the edge where the matching point j is located.

[0166] In some other optional embodiments, the update module 740 is specifically used to calculate the Kalman gain of the specific position and posture of each 3D point cloud in the feature observation list and the observation data in the tracking list currently containing multiple targets, correct the specific position and posture in the feature observation list, and obtain a new target tracking list.

[0167] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0168] The target tracking list updating device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0169] The embodiment of the present invention also provides a vehicle controller having the above Figure 7 The target tracking list updating device is shown.

[0170] See also Figure 8, is a schematic diagram of the structure of a vehicle controller provided by an optional embodiment of the present invention, the vehicle controller includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or in other ways as needed. The processor can process instructions executed in the vehicle controller, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface).

[0171] In some optional embodiments, if desired, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0172] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0173] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the target tracking list updating method shown in the above embodiment.

[0174] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the vehicle controller, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the vehicle controller via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0176] The vehicle controller also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 can be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0177] The input device 30 can receive input digital or character information, and generate signal input related to the user settings and function control of the vehicle controller, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0178] The vehicle controller also includes at least one communication interface for communicating with other devices or a communication network.

[0179] In addition, an embodiment of the present invention further provides a vehicle, such as Fig. 9 As shown, the vehicle includes: a vehicle controller 200 and an optical camera 300, and the optical camera 300 is installed on the vehicle.

[0180] The optical camera 300 is used to take a group of driving environment photos during a period of time while the vehicle is traveling, and send the group of driving environment photos to the vehicle controller 200. Optionally, the optical camera 300 includes but is not limited to a fisheye camera.

[0181] The vehicle controller 200 is used to receive a set of driving environment photos reported by the optical camera 300 and execute the above Figure 2 , Figure 4 The target tracking list updating method of the vehicle shown.

[0182] In addition, the structure of the vehicle controller 200 can be the aforementioned Figure 8 The structure shown is not limited in this embodiment.

[0183] In addition, the vehicle is also equipped with a sensor set 100, which includes at least one of a laser radar sensor, a telephoto or wide-angle camera, an accelerometer, and a gyroscope sensor. At least one sensor in the sensor set is used to collect driving environment information and / or vehicle speed information during the driving of the vehicle, and report it to the vehicle controller 200.

[0184] For example, a laser radar sensor, a telephoto or wide-angle camera is used to collect information about the vehicle's surroundings, such as taking photos and pictures. An accelerometer, a gyroscope sensor, etc. are used to collect information such as acceleration, speed, and position during the operation of the vehicle, and report the collected data / information to the vehicle controller 200.

[0185] An embodiment of the present invention also provides a computer-readable storage medium, and the above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented by downloading through a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware.

[0186] The storage medium may be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid state drive, etc.; further, the storage medium may also include a combination of the above-mentioned types of memories. It is understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor or hardware, the method shown in the above embodiment is implemented.

[0187] Embodiments of the present application may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the above method. Wherein, the computer program product may be written in any combination of one or more programming languages ​​to perform program codes for performing the operations of the disclosed embodiments, wherein the programming languages ​​include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device, partially on a remote computing device, or entirely on a remote computing device or server.

[0188] The above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention are described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for updating a target tracking list of a vehicle, characterized in that: The method comprises: Acquire multiple target information of the vehicle's surrounding environment predicted during driving, as well as 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after being projected in the world coordinate system; Performing feature association on the plurality of target information and the 3D point cloud data to determine observation point information from the drivable space corresponding to each target; Inputting the observation point information corresponding to each of the targets into the error function model, outputting the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets, and obtaining a feature observation list; The feature observation list is used to update the position and posture of the current tracking list containing multiple targets to obtain a new target tracking list.

2. The method according to claim 1, characterized in that Acquiring 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after being projected in the world coordinate system, including: receiving a set of driving environment photos taken by the optical camera within a period of time, wherein the optical camera is installed at at least one preset position of the vehicle and moves with the vehicle; Inputting the set of driving environment photos into a neural network model, and outputting a set of pixel points on the drivable space captured, each pixel point in the set of pixel points being represented by a two-dimensional coordinate; The two-dimensional coordinate pixel points in the pixel point set are projected onto a three-dimensional world coordinate system through the vehicle's posture information to obtain the 3D point cloud data.

3. The method according to claim 2, characterized in that The step of projecting the two-dimensional coordinate pixel points in the pixel point set to a three-dimensional world coordinate system through the vehicle's posture information to obtain the 3D point cloud data includes: Acquire the position and posture information of the vehicle collected by at least one sensor, and the position and posture information of the optical camera; Performing time synchronization on the posture information of the vehicle and the posture information of the optical camera at different times by linear interpolation; After time synchronization, all pixel points on the driving environment photos taken at the same time are converted into coordinate systems using the vehicle's position information to generate 3D point cloud data in the three-dimensional world coordinate system.

4. The method according to claim 1, characterized in that: Obtain multiple target information of the vehicle's surrounding environment predicted during driving, including: After the vehicle is started, an initial target tracking list is generated by calculating the targets around the vehicle, wherein the initial target tracking list includes the vehicle and multiple targets, and bounding box information corresponding to the vehicle and each of the targets; Each target in the initial target tracking list is predicted according to the bounding box information, the state and the motion trend of each target, to obtain a second target tracking list, wherein the second target tracking list includes multiple targets and target information corresponding to the multiple targets.

5. The method according to claim 4, characterized in that The step of performing feature association on the plurality of target information and the 3D point cloud data to determine observation point information from the drivable space corresponding to each target includes: The target information corresponding to the multiple targets, the bounding box information corresponding to each target and the 3D point cloud data are associated according to a proximity association algorithm to determine whether each 3D point cloud data is located on the bounding box corresponding to one or more targets, and the 3D point cloud data corresponding to the bounding box of each target on the second target tracking list is obtained.

6. The method according to any one of claims 1 to 5, characterized in that: The step of inputting the observation point information corresponding to each target into the error function model and outputting the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets includes: The ICP error function model of iterative closest point algorithm is constructed by using the hybrid matching method of point-point and point-line. Processing the observation point information through the ICP error function model to obtain the specific position and posture of each 3D point cloud on the multiple targets; Wherein, the ICP error function model is expressed as: Among them, J(R,t) represents the ICP error function model, R is the rotation matrix, t is the translation vector, p i is any observation point in the drivable space, q i For p i Corresponding to the matching points on the body frame, N is the total number of 3D points that are closest to the bounding box surface or edge, M is the total number of 3D points that are closest to the bounding box vertex, n j is the normal vector of the edge where the matching point j is located.

7. The method according to any one of claims 1 to 5, characterized in that: The method of using the feature observation list to update the position and posture of the current tracking list containing multiple targets to obtain a new target tracking list includes: The Kalman gain is calculated for the specific position and the posture of each 3D point cloud in the feature observation list and the observation data in the tracking list currently containing multiple targets, and the specific position and the posture in the feature observation list are corrected to obtain the new target tracking list.

8. A target tracking list updating device for a vehicle, characterized in that: The device comprises: An acquisition module is used to acquire multiple target information of the vehicle's surrounding environment predicted during driving, as well as 3D point cloud data of the drivable space points of the surrounding environment on the image observed by the optical camera after being projected in the world coordinate system; A determination module, used for performing feature association between the plurality of target information and the 3D point cloud data, and determining observation point information from the drivable space corresponding to each target; A processing module, used for inputting the observation point information corresponding to each of the targets into an error function model, outputting the specific position and posture of each 3D point cloud observed in the drivable space on multiple targets, and obtaining a feature observation list; The updating module is used to use the feature observation list to update the position and posture of the current tracking list containing multiple targets to obtain a new target tracking list.

9. A vehicle controller, characterized in that: comprising a memory and a processor, wherein the memory and the processor are connected; The memory stores computer instructions, and the processor executes the target tracking list updating method for a vehicle according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the vehicle target tracking list updating method according to any one of claims 1 to 7.

11. A vehicle, characterized in that: It includes a vehicle controller and an optical camera, wherein the optical camera is installed on the vehicle, The optical camera is used to take a group of driving environment photos during a period of time when the vehicle is traveling, and send the group of driving environment photos to the vehicle controller; The vehicle controller is used to execute the vehicle target tracking list updating method according to any one of claims 1 to 7 after receiving the group of driving environment photos.

12. The vehicle according to claim 11, characterized in that The vehicle is also equipped with a sensor set, wherein the sensor set includes at least one of a laser radar sensor, a telephoto or wide-angle camera, an accelerometer, and a gyroscope sensor; At least one sensor in the sensor set is used to collect driving environment information and / or vehicle speed information during the driving of the vehicle, and report it to the vehicle controller.

Images