Vehicle driving control method, device, electronic equipment and storage medium

Abstract

The present disclosure provides a vehicle driving control method, device, electronic device and storage medium, the method comprising: determining a current detection result corresponding to a current frame of point cloud data, wherein the current detection result includes three-dimensional posture data of at least one detection object; when a tracking function is turned on, controlling vehicle driving based on the current detection result and historical tracking chain information of at least one detection object in a previous frame of point cloud data of the current frame of point cloud data.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle control technology, and in particular to a vehicle driving control method, device, electronic equipment and storage medium. Background Art

[0002] With the development of science and technology, automatic driving systems and assisted driving systems are widely used in vehicles to realize automatic driving or assisted driving of vehicles. Automatic driving systems or assisted driving systems can obtain each frame of point cloud data, identify each frame of point cloud data, detect motor vehicles, non-motor vehicles, pedestrians, animals, obstacles and other target objects on the road during vehicle driving, and control the driving of the vehicle based on the detected target objects to avoid collision accidents.

[0003] An autonomous driving system generally performs target detection and predicts collision accidents for each frame of image acquired. In this case, the accuracy of the target detection results is low and there are cases of misjudgment, which can easily cause traffic accidents when controlling vehicle driving or take unnecessary avoidance measures due to misjudgment. Summary of the invention

[0004] In view of this, the present disclosure at least provides a vehicle driving control method, device, electronic device and storage medium.

[0005] In a first aspect, the present disclosure provides a vehicle driving control method, comprising:

[0006] Determine a current detection result corresponding to the current frame point cloud data, wherein the current detection result includes three-dimensional pose data of at least one detection object;

[0007] When the tracking function is turned on, the vehicle driving is controlled based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame point cloud data of the current frame point cloud data.

[0008] By adopting the above method, after determining the current detection result of the current frame point cloud data, when the tracking function is turned on, the vehicle driving can be controlled based on the current detection result and the historical tracking chain information of at least one detection object corresponding to the previous frame point cloud data of the current frame point cloud data. The historical tracking chain information includes information such as the historical three-dimensional posture data of the corresponding detection object. The driving trajectory of the detection object can be determined through the historical tracking chain information and the current detection result, and then the driving of the vehicle can be controlled based on the driving trajectory of at least one detection object. In this way, after obtaining the detection result of the current frame point cloud data, the vehicle control is not performed directly based on the detection result of the current frame point cloud data, but first combined with the fusion detection result of the previous continuous frame point cloud data to determine the current fusion detection result (that is, the current tracking chain information), and then the vehicle driving is controlled. The detection result is more accurate, and the accuracy and safety of automatic control are also higher.

[0009] In a possible implementation, controlling the vehicle to travel based on the current detection result and historical tracking chain information of at least one detection object corresponding to a frame of point cloud data before the current frame of point cloud data includes:

[0010] For any detection object in the current detection result, detecting whether there is historical tracking chain information corresponding to the detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data;

[0011] In the case where there is historical tracking chain information corresponding to the detection object, based on the three-dimensional pose data of the detection object in the current detection result and the historical tracking chain information corresponding to the detection object, generate current tracking chain information of the detection object; wherein the object identifier included in the current tracking chain information of the detection object is consistent with the object identifier in the historical tracking chain information corresponding to the detection object;

[0012] The vehicle driving is controlled based on the current tracking link information of at least one detected object.

[0013] Here, an implementation process for generating current tracking chain information is provided, and the same object identifier is assigned to the same detection object in the identified multi-frame point cloud data to facilitate tracking of the same detection object.

[0014] In a possible implementation manner, controlling the vehicle to travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data further includes:

[0015] For a detection object in the current detection result, detecting whether there is historical tracking chain information corresponding to the detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data;

[0016] In the absence of historical tracking chain information corresponding to the detection object, generating current tracking chain information of the detection object based on the three-dimensional pose data of the detection object in the current detection result; the current tracking chain information of the detection object includes an object identifier assigned to the detection object;

[0017] The vehicle driving is controlled based on the current tracking link information of at least one detected object.

[0018] Under the above implementation, when the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data does not include the historical tracking chain information corresponding to the detection object, the current tracking chain information of the detection object can be generated based on the three-dimensional posture data of the detection object in the current detection result. The current tracking chain information of the detection object includes the object identifier assigned to the detection object, so that the detection object can be tracked subsequently based on the assigned object identifier.

[0019] In a possible implementation, controlling the vehicle to travel based on the current detection result and historical tracking chain information of at least one detection object corresponding to a frame of point cloud data before the current frame of point cloud data includes:

[0020] Determine a target detection object having corresponding historical tracking chain information but not appearing in the current detection result based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data;

[0021] For the target detection object, determine the first number of frames of point cloud data in which the target detection object has not been detected continuously in the current frame point cloud data and at least one frame of historical point cloud data before the current frame point cloud data; if the first number of frames reaches a set loss threshold, delete the historical tracking chain information;

[0022] Determine the current tracking chain information of at least one detection object in the current detection result according to the current detection result and the historical tracking chain information of the detection objects other than the target detection object whose historical tracking chain information is deleted among at least one detection object in the previous frame of point cloud data of the current frame;

[0023] The vehicle driving is controlled according to the current tracking chain information of at least one detection object in the current detection result.

[0024] Under the above implementation, considering that the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data may contain historical tracking information of a detection object that is not detected in the current detection result, in this case, the detection object may no longer exist in the current frame of point cloud data. Therefore, here, the first number of frames of point cloud data in which the target detection object has not been detected continuously in the current frame of point cloud data and at least one frame of historical point cloud data before the current frame of point cloud data can be determined. When the first number of frames is greater than the set loss threshold, it is determined that the target detection object no longer exists, and the historical tracking chain information corresponding to the target detection object can be deleted, and the object identifier corresponding to the target detection object can be deleted, and the target detection object will no longer be tracked subsequently.

[0025] In a possible implementation, controlling the vehicle to travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data includes:

[0026] Determine a target detection object having corresponding historical tracking chain information but not appearing in the current detection result based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data;

[0027] For the target detection object, when it is determined that the historical tracking chain corresponding to the target detection object meets the preset stability condition, the number of historical continuous supplementary frames corresponding to the target detection object is determined based on the historical tracking chain information corresponding to the target detection object; wherein the preset stability condition includes that the number of continuous frames corresponding to the historical tracking chain information containing the target detection object is greater than the preset number of frames;

[0028] Determine the current tracking chain information according to the number of historical continuous supplemented frames and a set supplemented frame threshold;

[0029] Based on the current tracking link information, the vehicle is controlled to travel.

[0030] In a possible implementation manner, determining the current tracking chain information according to the historical continuous frame supplementation number and a set frame supplementation threshold includes:

[0031] When the number of historical continuous supplementary frames is less than a set supplementary frame threshold, based on the historical tracking chain information corresponding to the target detection object, the predicted three-dimensional pose data of the target detection object in the current frame point cloud data is generated; and based on the predicted three-dimensional pose data and the historical tracking chain information corresponding to the target detection object, the current tracking chain information is generated;

[0032] When the number of historical continuous supplementary frames is equal to the set supplementary frame threshold, the historical tracking chain information corresponding to the target detection object is determined as the current tracking chain information.

[0033] In the above implementation, considering that the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data may contain historical tracking information of a detection object that is not detected in the current detection result, in this case, there may also be a missed detection in the current frame of point cloud data, that is, the detection object exists in the current frame of point cloud data, but is not detected. Therefore, here, through the above process, when a missed detection occurs in the current frame of point cloud data, predicted three-dimensional pose data can be generated for the missed target detection object, and based on the predicted three-dimensional pose data and the historical tracking chain information corresponding to the target detection object, the current tracking chain information is generated, which can avoid driving accidents caused by the occurrence of missed detection when controlling the vehicle, thereby improving the safety of vehicle driving.

[0034] In a possible implementation, controlling the vehicle to travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data includes:

[0035] For any detection object in the current detection result, when the tracking chain length indicated by the historical tracking chain information corresponding to the detection object is equal to the set maximum maintenance length, the historical tracking data of the detection object corresponding to the earliest frame of point cloud data acquired in the historical tracking chain information is deleted, and the generated current tracking data of the detection object is added to the historical tracking chain information to generate the current tracking chain information;

[0036] Based on the current tracking link information, the vehicle is controlled to travel.

[0037] Under the above implementation mode, since the earlier the historical tracking data in the historical tracking chain information is obtained, the smaller the effect on controlling the vehicle driving at the current moment, in order to save storage resources, a maximum maintenance length can be set. When the tracking chain length indicated by the historical tracking chain information corresponding to any detection object is equal to the set maximum maintenance length, the historical tracking data of the detection object corresponding to the earliest frame of point cloud data obtained in the historical tracking chain information can be deleted, and the current tracking data of the detection object can be added to the historical tracking chain information to generate the current tracking chain information, so that the tracking chain length of the current tracking chain information corresponding to each detection object is less than or equal to the maximum maintenance length.

[0038] In a possible implementation manner, the current tracking chain information of a detection object includes the current average speed of the detection object, and the current average speed of the detection object in the current frame point cloud data is determined according to the following steps:

[0039] Determine a current speed of the detection object based on the three-dimensional pose data of the detection object in the current detection result and the previous three-dimensional pose data in the historical tracking chain information corresponding to the detection object;

[0040] Based on the current speed of the detection object and the latest N-1 historical speeds in the historical tracking chain information corresponding to the detection object, the current average speed of the detection object is determined; N is a preset number of frames for calculating the average speed.

[0041] In a possible implementation manner, the vehicle is controlled to travel according to the following steps:

[0042] For a detection object, when the number of consecutive frames contained in the current tracking chain information of the detection object is greater than the preset number of frames, the detection object is regarded as a real target object;

[0043] The vehicle driving is controlled based on the current tracking link information of at least one real target object determined.

[0044] In the above implementation, for each detection object, when the number of consecutive frames containing the detection object corresponding to the current tracking chain information is greater than the preset number of frames, the detection object can be regarded as a real target object. Based on the current tracking chain information of at least one real target object determined, the vehicle driving can be controlled more accurately to ensure the driving safety of the vehicle.

[0045] In a possible implementation, the current detection result corresponding to the current frame point cloud data is determined according to the following steps:

[0046] Based on the current frame point cloud data, generating at least one target sparse matrix corresponding to the current frame point cloud data; the target sparse matrix is ​​used to characterize whether there is a detection object at different positions of the target scene;

[0047] Based on the at least one target sparse matrix and the current frame point cloud data, a current detection result corresponding to the current frame point cloud data is determined.

[0048] By adopting the above method, at least one corresponding target sparse matrix can be generated for the acquired current frame point cloud data, and the target sparse matrix is ​​used to characterize whether there is a detection object at different positions of the target scene; in this way, when determining the current detection result corresponding to the frame point cloud data based on the target sparse matrix and the frame point cloud data, the target position where the detection object exists can be determined based on the target sparse matrix, so that the features corresponding to the target position can be processed, and the features corresponding to other positions except the target position in different positions are not processed, thereby reducing the amount of calculation for obtaining the three-dimensional posture data of the detection object and improving the detection efficiency.

[0049] In a possible implementation, generating at least one target sparse matrix corresponding to the current frame point cloud data based on the current frame point cloud data includes:

[0050] Based on the current frame point cloud data, a target sparse matrix corresponding to each layer of convolutional modules in a neural network used to detect the detection object is determined.

[0051] Under the above implementation, the corresponding target sparse matrix can be determined for each convolution module of the neural network based on the current frame point cloud data, so that each convolution module can process the input feature map based on the target sparse matrix.

[0052] In a possible implementation, based on the current frame point cloud data, determining the target sparse matrix corresponding to each convolution module in the neural network includes:

[0053] Based on the current frame point cloud data, generating an initial sparse matrix;

[0054] Based on the initial sparse matrix, a target sparse matrix matching a target size of a feature map input to each convolution module layer of the neural network is determined.

[0055] Under the above implementation, an initial sparse matrix can be generated based on the current point cloud data, and then based on the initial sparse matrix, a corresponding target sparse matrix can be determined for each layer of convolution module of the neural network, and the target sparse matrix corresponding to each layer of convolution module matches the target size of the feature map input to the convolution module of that layer, so that each layer of convolution module can process the input feature map based on the target sparse matrix.

[0056] In a possible implementation, generating an initial sparse matrix based on the current frame point cloud data includes:

[0057] Determine a target area corresponding to the current frame point cloud data, and divide the target area into a plurality of grid areas according to a preset number of grids;

[0058] Determine the matrix element value corresponding to each grid area based on the grid area where the point cloud point corresponding to the current frame point cloud data is located;

[0059] Based on the matrix element values ​​corresponding to each grid area, an initial sparse matrix corresponding to the current frame point cloud data is generated.

[0060] Here, based on the point cloud data of the current frame, it is possible to determine whether there are point cloud points in each grid area, and based on the judgment result, determine the matrix element value of each grid area. For example, if there are point cloud points in the grid area, the matrix element value of the grid area is 1, indicating that there is a detection object in the grid area. Then, based on the matrix element values ​​corresponding to each grid area, an initial sparse matrix is ​​generated, which provides data support for the subsequent determination of the three-dimensional pose data of the detection object.

[0061] In a possible implementation, determining a target sparse matrix that matches a target size of a feature map input to each convolution module layer of the neural network based on the initial sparse matrix includes:

[0062] Based on the initial sparse matrix, determine the output sparse matrix corresponding to each convolution module in the neural network, and use the output sparse matrix as the target sparse matrix; or

[0063] Based on the initial sparse matrix, determining the input sparse matrix corresponding to each convolution module in the neural network, and using the input sparse matrix as the target sparse matrix; or,

[0064] Based on the initial sparse matrix, the input sparse matrix and output sparse matrix corresponding to each layer of the convolution module in the neural network are determined, the input target sparse matrix and the output target sparse matrix are fused to obtain a fused sparse matrix, and the fused sparse matrix is ​​used as the target sparse matrix corresponding to the convolution module of this layer.

[0065] In the above implementation, multiple methods are set to generate the target sparse matrix corresponding to each layer of convolution module, that is, the target sparse matrix can be an input sparse matrix, an output sparse matrix, or a fused sparse matrix generated based on the input sparse matrix and the output sparse matrix.

[0066] In a possible implementation, determining the input sparse matrix corresponding to each convolution module in the neural network based on the initial sparse matrix includes:

[0067] Using the initial sparse matrix as the input sparse matrix corresponding to the first convolution module of the neural network;

[0068] Based on the input sparse matrix corresponding to the i-1th layer convolution module, determine the input sparse matrix corresponding to the i-th layer convolution module and matching the target size of the feature map input by the i-th layer convolution module; wherein i is a positive integer greater than 1 and less than n+1, and n is the number of layers of the convolution module of the neural network.

[0069] In the above method, the initial sparse matrix can be used as the input sparse matrix corresponding to the first layer of convolution module, and the input sparse matrix of each layer of convolution module can be determined in turn, and then the target sparse matrix can be determined based on the input sparse matrix, which provides data support for the subsequent determination of the three-dimensional pose data of the detection object based on the target sparse matrix of each layer of convolution module.

[0070] In a possible implementation, determining, based on the initial sparse matrix, an output sparse matrix corresponding to each convolution module in the neural network includes:

[0071] Determining an output sparse matrix corresponding to the neural network based on a size threshold of the detection object and the initial sparse matrix;

[0072] Based on the output sparse matrix, generating an output sparse matrix corresponding to the n-th layer convolution module and matching the target size of the feature map input by the n-th layer convolution module;

[0073] Based on the output sparse matrix corresponding to the j+1th convolution module, an output sparse matrix corresponding to the jth convolution module is generated, which matches the target size of the feature map input to the jth convolution module, wherein j is a positive integer greater than or equal to 1 and less than n, and n is the number of layers of the convolution module of the neural network.

[0074] In the above method, the output sparse matrix can be determined based on the initial sparse matrix, and the output sparse matrix of the nth layer convolution module, ..., the output sparse matrix of the first layer convolution module can be determined in sequence using the output sparse matrix. Then, the target sparse matrix can be determined based on the output sparse matrix of each layer, which provides data support for subsequently determining the three-dimensional pose data of the detection object based on the target sparse matrix of each layer of convolution module.

[0075] The effects of the following devices, electronic devices, etc. are described in the description of the above method and will not be repeated here.

[0076] In a second aspect, the present disclosure provides a vehicle driving control device, comprising:

[0077] A determination module, used to determine a current detection result corresponding to the current frame point cloud data, wherein the current detection result includes three-dimensional pose data of at least one detection object;

[0078] The control module is used to control the vehicle driving based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame point cloud data of the current frame point cloud data when the tracking function is turned on.

[0079] In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the vehicle driving control method described in the first aspect or any one of the embodiments are performed.

[0080] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the vehicle driving control method as described in the first aspect or any one of the embodiments above are executed.

[0081] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0083] Figure 1 A schematic diagram showing a flow chart of a vehicle driving control method provided by an embodiment of the present disclosure;

[0084] Figure 2 A schematic diagram showing a target area and an initial sparse matrix corresponding to the target area provided by an embodiment of the present disclosure is shown;

[0085] Figure 3 A schematic diagram of the architecture of a vehicle driving control device provided by an embodiment of the present disclosure is shown;

[0086] Figure 4 A schematic structural diagram of an electronic device 400 provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0088] Since the automatic driving system generally performs target detection and collision prediction for each frame of the acquired image, the target detection result determined in this case is less accurate and may cause misjudgment, which may easily lead to traffic accidents or unnecessary avoidance measures due to misjudgment when controlling the vehicle. Therefore, in order to solve the above problems, the embodiments of the present disclosure provide a vehicle control method.

[0089] To facilitate understanding of the embodiments of the present disclosure, a vehicle driving control method disclosed in the embodiments of the present disclosure is first introduced in detail.

[0090] See also Figure 1 FIG. 1 is a flow chart of a vehicle driving control method provided by an embodiment of the present disclosure, the method comprising S101-S102, wherein:

[0091] S101, determining a current detection result corresponding to the point cloud data of a current frame, wherein the current detection result includes three-dimensional posture data of at least one detection object.

[0092] S102, when the tracking function is turned on, the vehicle is controlled to travel based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data.

[0093] In the above method, after determining the current detection result of the current frame point cloud data, when the tracking function is turned on, the vehicle driving can be controlled based on the current detection result and the historical tracking chain information of at least one detection object corresponding to the previous frame point cloud data of the current frame point cloud data. The historical tracking chain information includes information such as the historical three-dimensional posture data of the corresponding detection object. The driving trajectory of the detection object can be determined through the historical tracking chain information and the current detection result, and then the driving of the vehicle can be controlled based on the driving trajectory of at least one detection object. In this way, after obtaining the detection result of the current frame point cloud data, the vehicle is not directly controlled based on the detection result of the current frame point cloud data, but first combined with the fusion detection result of the previous continuous frame point cloud data to determine the current fusion detection result (that is, the current tracking chain information), and then the vehicle driving is controlled. The detection result is more accurate, and the accuracy and safety of automatic control are also higher.

[0094] For S101:

[0095] Here, each frame of point cloud data corresponding to the target scene collected by the radar can be obtained first, and each frame of point cloud data can be detected to obtain a detection result, which includes the three-dimensional pose data of at least one detection object. Among them, the detection objects include but are not limited to motor vehicles, non-motor vehicles, pedestrians, animals, obstacles (such as stones), etc. The three-dimensional pose data of each detection object may include the coordinates of the center point of the detection frame of the detection object, the three-dimensional size of the detection frame, the orientation angle of the detection frame, the category of the detection frame, the confidence of the detection frame, etc.

[0096] The radar device may be a laser radar, a millimeter wave radar, etc. The disclosed embodiment is described by taking the radar device as a laser radar device as an example. The laser radar device collects point cloud data of the target scene by transmitting scan lines in real time. The target scene may be any scene, for example, the target scene may be a real-time scene encountered by a vehicle equipped with a laser radar device during driving.

[0097] As an optional implementation, the current detection result corresponding to the current frame point cloud data may be determined according to the following steps:

[0098] a1, based on each frame of point cloud data, generate at least one target sparse matrix corresponding to the current frame of point cloud data; the target sparse matrix is ​​used to characterize whether there is a detection object at different positions of the target scene.

[0099] a2, based on at least one target sparse matrix and the current frame point cloud data, determine the current detection result corresponding to the current frame point cloud data.

[0100] Here, after acquiring the current frame point cloud data, at least one corresponding target sparse matrix may be generated based on the current frame point cloud data, wherein the target sparse matrix may represent whether there is a detection object at different positions of the target scene.

[0101] Exemplarily, the target sparse matrix may be a matrix including 0 and 1, that is, the element value of the target sparse matrix is ​​0 or 1. For example, the matrix element value corresponding to the position where the detection object exists in the target scene may be set to 1, and the matrix element value corresponding to the position where the detection object does not exist in the target scene may be set to 0.

[0102] In a specific implementation, the current frame point cloud data can be input into a neural network used to detect the detection object, and the frame point cloud data can be preprocessed to generate a point cloud feature map corresponding to the current frame point cloud data (the point cloud feature map is a feature map obtained after rasterizing the point cloud data); then at least one target sparse matrix and the point cloud feature map corresponding to the current frame point cloud data can be input into the neural network used to detect the detection object to determine the current detection result corresponding to the current frame point cloud data.

[0103] By adopting the above method, at least one corresponding target sparse matrix can be generated for the acquired current frame point cloud data, and the target sparse matrix is ​​used to characterize whether there is a detection object at different positions of the target scene; in this way, when determining the current detection result corresponding to the frame point cloud data based on the target sparse matrix and the frame point cloud data, the target position where the detection object exists can be determined based on the target sparse matrix, so that the features corresponding to the target position can be processed, and the features corresponding to other positions except the target position in different positions are not processed, thereby reducing the amount of calculation for determining the three-dimensional posture data of the detection object and improving the detection efficiency.

[0104] In a possible implementation, generating at least one target sparse matrix corresponding to the current frame point cloud data based on the current frame point cloud data may include: determining, based on the current frame point cloud data, the target sparse matrix corresponding to each layer of convolution module in the neural network used to detect the detection object.

[0105] Here, the neural network may be a trained network for detecting the detection object, and the neural network may include multiple layers of convolution modules, each of which may include a convolution layer. In a specific implementation, a corresponding target sparse matrix may be determined for each convolution module, that is, a corresponding target sparse matrix is ​​determined for each convolution layer. Alternatively, the neural network may also include multiple network module blocks, each of which includes multiple layers of convolution layers. In a specific implementation, a corresponding target sparse matrix may be determined for each network module, that is, a corresponding target sparse matrix is ​​determined for the multiple layers of convolution layers included in the network module. Among them, the structure of the neural network used to detect the detection object can be set as needed, and this is only an exemplary description.

[0106] After determining the trained neural network for detecting the detection object, the corresponding target sparse matrix can be determined for each convolution module in the neural network based on the current frame point cloud data.

[0107] When training a neural network for detecting the detection object, training sample data can be obtained, and based on the training sample data, at least one sample sparse matrix corresponding to the training sample data can be generated. Then, based on the training sample data and the corresponding at least one sample sparse matrix, the neural network can be trained to obtain a trained neural network.

[0108] Under the above implementation, the corresponding target sparse matrix can be determined for each convolution module of the neural network based on the current frame point cloud data, so that each convolution module can process the input feature map based on the target sparse matrix.

[0109] In a possible implementation, based on the current frame point cloud data, determining the target sparse matrix corresponding to each convolution module in the neural network includes:

[0110] Step 1: Generate an initial sparse matrix based on the current frame point cloud data;

[0111] Step 2: Based on the initial sparse matrix, determine the target sparse matrix that matches the target size of the feature map input to each convolution module of the neural network.

[0112] Under the above implementation, an initial sparse matrix can be generated based on the current frame point cloud data, and then based on the initial sparse matrix, a corresponding target sparse matrix can be determined for each layer of convolution module of the neural network, and the target sparse matrix corresponding to each layer of convolution module matches the target size of the feature map input to the convolution module of that layer, so that each layer of convolution module can process the input feature map based on the target sparse matrix.

[0113] For step one:

[0114] In a possible implementation, generating an initial sparse matrix based on the current frame point cloud data may include:

[0115] Determine the target area corresponding to the current frame point cloud data, and divide the target area into multiple grid areas according to the preset number of grids;

[0116] Based on the grid area where the point cloud point corresponding to the point cloud data of the current frame is located, determine the matrix element value corresponding to each grid area;

[0117] Based on the matrix element values ​​corresponding to each grid area, an initial sparse matrix corresponding to the current frame point cloud data is generated.

[0118] Here, based on the point cloud data of the current frame, it is possible to determine whether there are point cloud points in each grid area, and based on the judgment result, determine the matrix element value of each grid area. For example, if there are point cloud points in the grid area, the matrix element value of the grid area is 1, indicating that there is a detection object in the grid area. Then, based on the matrix element values ​​corresponding to each grid area, an initial sparse matrix is ​​generated, which provides data support for the subsequent determination of the three-dimensional pose data of the detection object.

[0119] For example, the target area corresponding to the current frame point cloud data can be: the position when the laser radar device obtains the current frame point cloud data is the starting position, and the maximum distance that the laser radar device can detect is the length, and the measurement area is determined. Among them, the target area can be determined according to the actual situation and the current frame point cloud data.

[0120] In specific implementation, the preset number of grids may be N×M, and the target area may be divided into N×M grid areas, where N and M are positive integers, and the values ​​of N and M may be set according to actual needs.

[0121] In a specific implementation, the current frame point cloud data includes position information of multiple point cloud points. Based on the position information of each point cloud point, the grid area where each point cloud point is located can be determined. Furthermore, for each grid area, when there is a corresponding point cloud point in the grid area, the value of the matrix element corresponding to the grid area can be 1; when there is no corresponding point cloud point in the grid area, the value of the matrix element corresponding to the grid area can be 0, so the matrix element value corresponding to each grid area is determined.

[0122] After determining the matrix element values ​​corresponding to each grid area, the initial sparse matrix corresponding to the current frame point cloud data can be generated based on the matrix element values ​​corresponding to each grid area, wherein the number of rows and columns of the initial sparse matrix corresponds to the number of grids. For example, if the number of grids is N×M, the number of rows and columns of the initial sparse matrix is ​​N, and the number of columns is M, that is, the initial sparse matrix is ​​an N×M matrix.

[0123] See also Figure 2 As shown in the figure, a laser radar device 21 is included, and a target area 22 is obtained with the laser radar device as the center, and the target area is divided into multiple grid areas according to a preset number of grids, and multiple grid areas 221 after division are obtained. Then the grid areas where multiple point cloud points corresponding to the current frame point cloud data are located are determined, and the matrix element values ​​of the grid areas where point cloud points exist (i.e., the grid areas with black shadows in the figure) are set to 1, and the matrix element values ​​of the grid areas where no point cloud points exist are set to 0, and the matrix element values ​​of each grid area are obtained. Finally, based on the matrix element values ​​corresponding to each grid area, the initial sparse matrix 23 corresponding to the current frame point cloud data is generated.

[0124] For step 2:

[0125] After the initial sparse matrix is ​​obtained, a target sparse matrix matching the target size of the feature map input to each convolution module of the neural network layer can be determined based on the initial sparse matrix.

[0126] In one possible implementation, determining a target sparse matrix that matches a target size of a feature map input to each convolution module of a neural network layer based on the initial sparse matrix may include:

[0127] Method 1: Based on the initial sparse matrix, determine the output sparse matrix corresponding to each layer of convolution module in the neural network, and use the output sparse matrix as the target sparse matrix;

[0128] Method 2: Based on the initial sparse matrix, determine the input sparse matrix corresponding to each convolution module in the neural network, and use the input sparse matrix as the target sparse matrix;

[0129] Method three: Based on the initial sparse matrix, determine the input sparse matrix and output sparse matrix corresponding to each layer of convolution module in the neural network, fuse the input target sparse matrix and the output target sparse matrix to obtain a fused sparse matrix, and use the fused sparse matrix as the target sparse matrix corresponding to the convolution module of this layer.

[0130] Here, the target sparse matrix may be obtained from the output sparse matrix, may be obtained from the input sparse matrix, or may be obtained by fusing the input sparse matrix and the output sparse matrix.

[0131] In the above implementation, multiple methods are set to generate the target sparse matrix corresponding to each layer of convolution module, that is, the target sparse matrix can be an input sparse matrix, an output sparse matrix, or a fused sparse matrix generated based on the input sparse matrix and the output sparse matrix.

[0132] For method one, this method is to obtain a target sparse matrix from an output sparse matrix. In a specific implementation, the output sparse matrix corresponding to each layer of convolutional module in the neural network can be determined based on the initial sparse data, and the output sparse matrix is ​​the target sparse matrix. Among them, the output sparse matrix can be used to characterize whether there is a detection object at different positions of the corresponding target scene in the output results of each layer of convolutional module of the neural network. For example, if there is a detection object at position A corresponding to the target scene in the output results of each layer of convolutional module of the neural network, then in the sparse matrix, the matrix element value at the position corresponding to the position A can be 1; if there is no detection object at position A, then in the sparse matrix, the matrix element value at the position corresponding to the position A can be 0.

[0133] For the second method, the target sparse matrix is ​​obtained from the input sparse matrix. In the specific implementation, the input sparse matrix corresponding to each layer of convolution module in the neural network can be determined based on the initial sparse data, and the input sparse matrix is ​​the target sparse matrix. Among them, the input sparse matrix can be the input data representing each layer of convolution module of the neural network, corresponding to whether there is a detection object at different positions of the target scene.

[0134] For method three, the output sparse matrix corresponding to each layer of convolution module can be determined by method one, and the input sparse matrix corresponding to each layer of convolution module can be determined by method two. The input sparse matrix and output sparse matrix corresponding to each layer of convolution module are fused to obtain a fused sparse matrix, and the fused sparse matrix is ​​used as the target sparse matrix corresponding to the convolution module of this layer.

[0135] In specific implementation, the input sparse matrix and the output sparse matrix can be intersected to obtain a fused sparse matrix; or the input sparse matrix and the output sparse matrix can be unioned to obtain a fused sparse matrix. For example, if the input sparse matrix is:

[0136]

[0137] If the output sparse matrix is:

[0138]

[0139] Then take the intersection of the input sparse matrix and the output sparse matrix, and the resulting fused sparse matrix is:

[0140]

[0141] Then take the union of the input sparse matrix and the output sparse matrix, and the resulting fused sparse matrix is:

[0142]

[0143] In a possible implementation, based on the initial sparse matrix, determining the input sparse matrix corresponding to each convolution module in the neural network includes:

[0144] The initial sparse matrix is ​​used as the input sparse matrix corresponding to the first convolution module of the neural network;

[0145] Based on the input sparse matrix corresponding to the i-1th convolution module, determine the input sparse matrix corresponding to the i-th convolution module and matching the target size of the feature map input to the i-th convolution module; wherein i is a positive integer greater than 1 and less than n+1, and n is the number of layers of the convolution module of the neural network.

[0146] Here, the initial sparse matrix can be used as the input sparse matrix corresponding to the first convolution module of the neural network. The input sparse matrix corresponding to the second convolution module can be obtained from the input sparse matrix corresponding to the first convolution module, and the number of rows and columns of the input sparse matrix corresponding to the second convolution module is consistent with the target size of the feature map input to the second convolution module.

[0147] Exemplarily, the input sparse matrix corresponding to the first layer convolution module can be processed by image expansion processing operation or image corrosion processing operation to obtain a processed sparse matrix, and the number of rows and columns of the processed sparse matrix is ​​adjusted to match the target size of the feature map input by the second layer convolution module, and then the input sparse matrix of the second layer convolution module is obtained. By analogy, the input sparse matrix corresponding to the first layer convolution module, the input sparse matrix corresponding to the second layer convolution module, ..., the input sparse matrix corresponding to the nth layer convolution module (i.e., the input sparse matrix corresponding to the last layer convolution module of the neural network) can be obtained.

[0148] Exemplarily, the expansion processing range can be determined in advance, and image expansion processing can be performed on the input sparse matrix based on the expansion processing range to obtain a processed sparse matrix, wherein the expansion processing range can be determined based on a size threshold of the detection object, and can also be determined according to actual needs.

[0149] For example, if the input sparse matrix is:

[0150]

[0151] Then the sparse matrix after expansion can be:

[0152]

[0153] The above expansion process is only for illustrative purposes.

[0154] Exemplarily, the erosion process of the input sparse matrix is ​​the inverse process of the expansion process. Specifically, the erosion process range can be determined in advance, and the input sparse matrix is ​​subjected to image erosion process based on the erosion process range to obtain the processed sparse matrix. The erosion process range can be determined based on the size threshold of the detection object, or can be determined according to actual needs.

[0155] For example, if the input sparse matrix is:

[0156]

[0157] Then the sparse matrix after corrosion processing can be:

[0158]

[0159] The above-mentioned corrosion treatment process is only for illustrative purposes.

[0160] In a specific implementation, the number of rows and columns of the processed sparse matrix can be adjusted to a matrix that matches the target size of the feature map input to the second-layer convolution module by upsampling or downsampling, thereby obtaining the input sparse matrix of the second-layer convolution module. There are multiple processes for adjusting the number of rows and columns of the processed sparse matrix, which are only exemplified here.

[0161] In a specific implementation, the sparsity of the sparse matrix can also be adjusted, for example, by adjusting the number of grids, the sparsity of the sparse matrix can be adjusted; or the sparsity of the sparse matrix can be adjusted through an erosion process. The sparsity of the sparse matrix is ​​the ratio of the number of matrix element values ​​1 in the sparse matrix to the total number of all matrix element values ​​included in the sparse matrix.

[0162] In the above method, the initial sparse matrix can be used as the input sparse matrix corresponding to the first layer of convolution module, and the input sparse matrix of each layer of convolution module can be determined in turn, and then the target sparse matrix can be determined based on the input sparse matrix, which provides data support for the subsequent determination of the three-dimensional pose data of the detection object based on the target sparse matrix of each layer of convolution module.

[0163] In a possible implementation, based on the initial sparse matrix, determining the output sparse matrix corresponding to each convolution module in the neural network includes:

[0164] Based on the size threshold of the detected object and the initial sparse matrix, an output sparse matrix corresponding to the neural network is determined;

[0165] Based on the output sparse matrix, generate an output sparse matrix corresponding to the n-th layer convolution module and matching the target size of the feature map input to the n-th layer convolution module;

[0166] Based on the output sparse matrix corresponding to the j+1th convolution module, generate an output sparse matrix corresponding to the jth convolution module and matching the target size of the feature map input to the jth convolution module, where j is a positive integer greater than or equal to 1 and less than n, and n is the number of layers of the convolution module of the neural network.

[0167] Here, the expansion processing range can be determined according to the size threshold of the detection object, and the initial sparse matrix can be expanded based on the expansion processing range to obtain a processed sparse matrix, which is the output sparse matrix corresponding to the neural network. The expansion processing process can refer to the above description and will not be repeated here.

[0168] The output sparse matrix is ​​used to determine the output sparse matrix of the nth convolution module of the neural network (i.e., the last convolution module of the neural network), and so on, to obtain the output sparse matrix of the n-1th convolution module, ..., the output sparse matrix of the second convolution module, and the output sparse matrix of the first convolution module.

[0169] Exemplarily, the output sparse matrix corresponding to the previous layer of convolution module can be processed by image dilation processing operation or image corrosion processing operation to obtain a processed sparse matrix, and the number of rows and columns of the processed sparse matrix is ​​adjusted to match the target size of the feature map input to the current layer of convolution module, and then the output sparse matrix of the current layer of convolution module is obtained. The process of determining the output sparse matrix of each layer of convolution module can refer to the above process of determining the input sparse matrix, and will not be described in detail here.

[0170] When the target sparse matrix of each convolution module of the neural network is obtained by fusing the input sparse matrix and the output sparse matrix, the above method can be used to obtain the output sparse matrix and the input sparse matrix of each convolution module respectively, and the obtained output sparse matrix and input sparse matrix are fused to obtain the target sparse matrix of each convolution module.

[0171] In the above method, the output sparse matrix can be determined based on the initial sparse matrix, and the output sparse matrix of the nth layer convolution module, ..., the output sparse matrix of the first layer convolution module can be determined in sequence using the output sparse matrix. Then, the target sparse matrix can be determined based on the output sparse matrix of each layer, which provides data support for subsequently determining the three-dimensional pose data of the detection object based on the target sparse matrix of each layer of convolution module.

[0172] For S102:

[0173] Here, you can set whether to turn on the tracking function. After determining that the tracking function is turned on, you can control the vehicle's driving based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data; after determining that the tracking function is not turned on, you may not perform operations to control the vehicle's driving based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data.

[0174] In an optional implementation, controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data of the current frame of point cloud data may include:

[0175] b1, for any detection object in the current detection result, check whether there is historical tracking chain information corresponding to any detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data.

[0176] b2. When there is historical tracking chain information corresponding to the detection object, generate current tracking chain information of the detection object based on the three-dimensional posture data of the detection object in the current detection result and the historical tracking chain information corresponding to the detection object; wherein the current tracking chain information of the detection object includes an object identifier, which is consistent with the object identifier in the historical tracking chain information corresponding to the detection object; and control vehicle driving based on the current tracking chain information of at least one detection object.

[0177] b3, in the absence of historical tracking chain information corresponding to the detection object, based on the three-dimensional posture data of the detection object in the current detection result, generate current tracking chain information of the detection object; the current tracking chain information of the detection object includes the object identifier assigned to the detection object; based on the current tracking chain information of at least one detection object, control the vehicle driving.

[0178] Here, an implementation process for generating current tracking chain information is provided, and the same object identifier is assigned to the same detection object in the identified multi-frame point cloud data to facilitate tracking of the same detection object.

[0179] Here, the previous frame of point cloud data corresponds to historical tracking chain information of at least one detected object, and the historical tracking chain information of each detected object may include the 3D position data, current speed, average speed, object identification, etc. of the detected object. Specifically, the historical tracking chain information can be set according to actual needs.

[0180] For each detection object in the current detection result, determine whether there is historical tracking chain information corresponding to the detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data. If there is, then based on the three-dimensional posture data of the detection object in the current detection result and the historical tracking chain information corresponding to the detection object, generate the current tracking chain information of the detection object; wherein, the current tracking chain information of the detection object contains the object identifier of the detection object, which is consistent with the object identifier in the historical tracking chain information corresponding to the detection object, that is, the object identifier in the historical tracking chain information is set as the object identifier of the detection object. If it does not exist, then based on the three-dimensional posture data of the detection object in the current detection result, generate the current tracking chain information of the detection object; the current tracking chain information contains the object identifier assigned to the detection object, that is, when the detection object did not exist before, an object identifier is generated for the detection object. Different detection objects correspond to different object identifiers, and the same detection object in different frames of point cloud data corresponds to the same object identifier.

[0181] Exemplarily, for each detection object corresponding to the current detection result, the predicted pose information of the detection object can be determined based on the historical tracking chain information of the detection object in the previous frame of point cloud data, and the pose deviation between the predicted pose information of the detection object and the three-dimensional pose data of the detection object in the current detection result can be determined. If there is target predicted pose information whose pose deviation is less than a preset deviation threshold, it is determined that the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data contains historical tracking chain information corresponding to the detection object, wherein the historical tracking chain information corresponding to the target predicted pose information is the historical tracking chain information of the detection object in the current detection result.

[0182] Exemplarily, if the current detection result includes detection object A, detection object B, and detection object C, the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data includes: historical tracking chain information corresponding to detection object A, and historical tracking chain information corresponding to detection object B.

[0183] It can be known from the calculation for the detection object A that there is historical tracking chain information corresponding to the detection object A in the previous frame of point cloud data. Therefore, the current tracking chain information corresponding to the detection object A can be generated based on the three-dimensional pose data of the detection object A in the current detection result and the historical tracking chain information corresponding to the detection object A in the previous frame of point cloud data, so that the generated current tracking chain information includes the tracking information of the detection object A in the current frame of point cloud data. At the same time, the object identifier of the detection object A in the current frame of point cloud data is set to the object identifier indicated by the historical tracking chain information corresponding to the detection object A. For example, if the object identifier indicated by the historical tracking chain information corresponding to the detection object A is vehicle 001, the object identifier of the detection object A in the current detection result is also vehicle 001. The process of generating the detection object B is similar to that of the detection object A, and will not be repeated here.

[0184] For the detection object C, it can be known that there is no historical tracking chain information corresponding to the detection object C in the previous frame of point cloud data. Therefore, the current tracking chain information can be generated based on the three-dimensional pose data of the detection object C in the current detection result. At the same time, a new object identifier is generated for the detection object C. Finally, based on the current tracking chain information corresponding to the detection object A, the current tracking chain information corresponding to the detection object B, and the current tracking chain information corresponding to the detection object C, the vehicle is controlled to travel.

[0185] Under the above implementation, when the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data does not include the historical tracking chain information corresponding to the detection object, the current tracking chain information of the detection object can be generated based on the three-dimensional posture data of the detection object in the current detection result. The current tracking chain information of the detection object includes the object identifier assigned to the detection object, so that the detection object can be tracked subsequently based on the assigned object identifier.

[0186] In an optional implementation, controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object corresponding to a frame of point cloud data before the current frame of point cloud data includes:

[0187] c1, based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame point cloud data of the current frame point cloud data, determine the target detection object that has corresponding historical tracking chain information but does not appear in the current detection result.

[0188] c2, for the target detection object, determine the first number of frames of point cloud data in which the target detection object has not been detected continuously in the current frame point cloud data and at least one frame of historical point cloud data before the current frame point cloud data; when the first number of frames reaches the set loss threshold, delete the historical tracking chain information.

[0189] c3, determine the current tracking chain information of at least one detection object in the current detection result based on the current detection result and the historical tracking chain information of the detection objects in at least one detection object in the previous frame point cloud data of the current frame point cloud data, except for the target detection object whose historical tracking chain information has been deleted.

[0190] c4, controlling the vehicle to travel according to the current tracking chain information of at least one detection object in the current detection result.

[0191] Here, the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data may include detection objects that are not detected in the current detection result. For example, if the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data includes: the historical tracking chain information corresponding to detection object A, the historical tracking chain information corresponding to detection object B, and the historical tracking chain information corresponding to detection object D, and the current detection result includes detection object A, detection object B, and detection object C, it can be known that the previous frame of point cloud data includes the historical tracking chain data corresponding to detection object D, while the detection result corresponding to the current frame of point cloud data does not include detection object D, that is, detection object D is the target detection object.

[0192] For the detection object D, if the loss threshold is set to 3, the first frame number of point cloud data in which the detection object D has not been detected continuously in the current frame point cloud data and at least one frame of historical point cloud data before the current frame point cloud data is 3 (including the current frame point cloud data, that is, the detection object D was not detected in the current frame point cloud data and the two adjacent frames of historical point cloud data before the current frame point cloud data), that is, the first frame number corresponding to the detection object is equal to the set loss threshold, then the detection object D is considered to no longer exist, and the historical tracking chain information corresponding to the detection object D can be deleted, and the object identifier corresponding to the detection object D can be deleted.

[0193] In the current frame point cloud data and at least one frame of historical point cloud data before the current frame point cloud data, the first frame number of point cloud data in which the detection object D is not detected continuously is 2 (i.e., the detection object D is not detected in the current frame point cloud data and one frame of adjacent historical point cloud data before the current frame point cloud data), that is, the first frame number corresponding to the detection object is less than the set loss threshold, then the historical tracking chain information corresponding to the detection object D is retained, and the object identification corresponding to the detection object D is retained; or, after the first frame number corresponding to the detection object is less than the set loss threshold, it is also possible to determine whether it is necessary to perform frame supplementation processing for the current frame point cloud data for the detection object D. The loss threshold can be set according to. For example, the loss threshold can be 3, 5, etc.

[0194] Here, considering that the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data may contain historical tracking information of a detection object that is not detected in the current detection result, in this case, the detection object may no longer exist in the current frame of point cloud data. Therefore, here, the first number of frames of point cloud data in which the target detection object has not been detected continuously in the current frame of point cloud data and at least one frame of historical point cloud data before the current frame of point cloud data can be determined. When the first number of frames is greater than the set loss threshold, it is determined that the target detection object no longer exists, and the historical tracking chain information corresponding to the target detection object can be deleted, and the object identifier corresponding to the target detection object can be deleted, and the target detection object will no longer be tracked subsequently.

[0195] In another optional implementation, controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame may include:

[0196] 1. Based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame, determine the target detection object that has corresponding historical tracking chain information but does not appear in the current detection result.

[0197] 2. For the target detection object, when it is determined that the historical tracking chain corresponding to the target detection object meets the preset stability condition, the number of historical continuous supplementary frames corresponding to the target detection object is determined based on the historical tracking chain information corresponding to the target detection object; wherein the preset stability condition includes that the number of continuous frames containing the target detection object corresponding to the historical tracking chain information is greater than the preset number of frames. Each supplementary frame means that when the target detection object is not currently detected, the predicted three-dimensional pose data of the target detection object is added to the current tracking chain information.

[0198] 3. Determine the current tracking chain information according to the number of historical continuous supplementary frames and the set supplementary frame threshold; and control the vehicle driving based on the current tracking chain information.

[0199] Exemplarily, for detection object D (i.e., target detection object), it can be determined whether the historical tracking chain corresponding to detection object D meets the stability condition. When the historical tracking chain corresponding to detection object D meets the stability condition, the number of historical continuous supplementary frames corresponding to detection object D is determined based on the historical tracking chain information corresponding to detection object D; and then the current tracking chain information can be determined based on the number of historical continuous supplementary frames and the set supplementary frame threshold. When the historical tracking chain corresponding to detection object D does not meet the stability condition, the number of historical continuous supplementary frames corresponding to detection object D is not determined, that is, no supplementary frame processing is performed on detection object D. The supplementary frame threshold can be set according to actual needs, for example, the supplementary frame threshold can be 3, 5, 6, etc. Preferably, the supplementary frame threshold can be less than the set loss threshold.

[0200] Among them, it can be determined that the historical tracking chain information corresponding to the detection object D contains the continuous frame number M (M is a positive integer) of the continuous point cloud data of the detection object D, that is, if the detection object D is detected by continuous M frames of point cloud data in the historical tracking chain information, when the value of M is greater than or equal to the preset number of frames, the historical tracking chain corresponding to the detection object D satisfies the preset stability condition, that is, the historical tracking chain corresponding to the detection object D is a stable tracking chain; when the value of M is less than the preset number of frames, the historical tracking chain corresponding to the detection object D does not satisfy the preset stability condition, that is, the historical tracking chain corresponding to the detection object D is not a stable tracking chain.

[0201] For example, if the target detection object is not detected in each frame of the three frames of historical point cloud data before the current frame of point cloud data, and the predicted three-dimensional pose data of the target detection object is added to the current tracking chain information, then the three frames of historical point cloud data before the current frame of point cloud data all have frame supplementation processing, and thus the number of historical continuous supplementation frames corresponding to the target detection object is 3.

[0202] In an optional implementation, the current tracking chain information is determined according to the number of historical continuous supplementary frames and a set supplementary frame threshold, including:

[0203] Case 1: When the number of historical continuous supplementary frames is less than the set supplementary frame threshold, the predicted three-dimensional pose data of the target detection object in the current frame point cloud data is generated based on the historical tracking chain information corresponding to the target detection object; and the current tracking chain information is generated based on the predicted three-dimensional pose data and the historical tracking chain information corresponding to the target detection object.

[0204] Case 2: When the number of historical continuous supplementary frames is equal to the set supplementary frame threshold, the historical tracking chain information corresponding to the target detection object is determined as the current tracking chain information.

[0205] Considering that when detecting point cloud data and obtaining the detection results, there may be missed detections, where the target detection object exists in the point cloud data but is not detected in the point cloud data. At this time, it can be determined whether the historical tracking chain corresponding to the missed target detection object is a stable tracking chain. If it is not a stable tracking chain, no frame supplementation processing is performed on the historical tracking chain corresponding to the missed target detection object; if it is a stable tracking chain, the historical continuous supplementary frame data of the historical tracking chain corresponding to the missed target detection object can be determined. When the determined number of historical continuous supplementary frames is less than the set supplementary frame threshold, the predicted three-dimensional pose data of the target detection object in the current frame point cloud data can be generated based on the historical tracking chain information corresponding to the target detection object; and the current tracking chain information is generated based on the predicted three-dimensional pose data and the historical tracking chain information corresponding to the target detection object, that is, the frame supplementation processing of the current frame point cloud data is completed.

[0206] When the determined number of historical continuous supplementary frames is equal to the set supplementary frame threshold, the historical tracking chain corresponding to the missed target detection object is not supplemented. Alternatively, for the target detection object, when the determined number of historical continuous supplementary frames is equal to the set supplementary frame threshold, it can also be determined whether the historical tracking chain information corresponding to the target detection object meets the deletion condition, that is, it is determined whether the first frame number corresponding to the historical tracking chain information of the target detection object reaches the set loss threshold, and if so, the historical tracking chain information is deleted.

[0207] Among them, the predicted three-dimensional pose data of the target detection object in the current frame point cloud data can be generated based on the current speed of the target detection object indicated in the historical tracking chain information corresponding to the target detection object and the historical three-dimensional pose data.

[0208] In the above implementation, considering that the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data may contain historical tracking information of a detection object that is not detected in the current detection result, in this case, there may also be a missed detection in the current frame of point cloud data, that is, the detection object exists in the current frame of point cloud data, but is not detected. Therefore, here, through the above process, when a missed detection occurs in the current frame of point cloud data, predicted three-dimensional pose data can be generated for the missed target detection object, and the current tracking chain information is generated based on the predicted three-dimensional pose data and the historical tracking chain information corresponding to the target detection object, which can avoid driving accidents caused by missed detection when controlling the vehicle, thereby improving the safety of vehicle driving.

[0209] In an optional implementation, vehicle travel is controlled based on a current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before a current frame of point cloud data, including: for any detection object in the current detection result, when the tracking chain length indicated by the historical tracking chain information corresponding to the detection object is equal to the set maximum maintenance length, the historical tracking data of the detection object corresponding to the earliest frame of point cloud data acquired in the historical tracking chain information is deleted, and the generated current tracking data of the detection object is added to the historical tracking chain information to generate current tracking chain information; and based on the current tracking chain information, the vehicle travel is controlled.

[0210] Here, the tracking data may include the three-dimensional posture data, current speed, average speed, object identification, etc. of the detection object. For example, the three-dimensional posture data of the detection object in the detection result can be determined as the three-dimensional posture data of the detection object included in the tracking data.

[0211] Among them, the maximum maintenance length can be set according to actual needs, for example, the maximum maintenance length can be 10, 15, etc. The maximum maintenance length is the maximum number of frames of the detection results of the point cloud data contained in the tracking chain. For example, if the maximum maintenance length is 10, the current tracking chain information or the historical tracking chain information includes at most 10 three-dimensional pose data and 10 current speeds of the detection object. The tracking chain length indicated by the historical tracking chain information corresponding to the detection object can be determined based on the number of tracking data in the historical tracking chain information. For example, if the historical tracking chain information includes 8 three-dimensional pose data of the detection object, the tracking chain length is 8.

[0212] Furthermore, after generating the current tracking data corresponding to each detection object (including at least one detection object indicated in the detection result and also the corresponding target detection object in the historical tracking chain), for each detection object, when the tracking chain length indicated by the historical tracking chain information corresponding to the detection object is equal to the set maximum maintenance length, the historical tracking data of the detection object corresponding to the earliest frame of point cloud data obtained in the historical tracking chain information can be deleted, and the generated current tracking data of the detection object can be added to the historical tracking chain information to generate the current tracking chain information, so that the generated current tracking chain length is not greater than the maximum maintenance length.

[0213] Under the above implementation mode, since the earlier the historical tracking data in the historical tracking chain information is obtained, the smaller the effect on controlling the vehicle driving at the current moment, in order to save storage resources, a maximum maintenance length can be set. When the tracking chain length indicated by the historical tracking chain information corresponding to any detection object is equal to the set maximum maintenance length, the historical tracking data of the detection object corresponding to the earliest frame of point cloud data obtained in the historical tracking chain information can be deleted, and the current tracking data of the detection object can be added to the historical tracking chain information to generate the current tracking chain information, so that the tracking chain length of the current tracking chain information corresponding to each detection object is less than or equal to the maximum maintenance length.

[0214] In an optional implementation, the current tracking chain information of a detection object includes the current average speed of the detection object, and the current average speed of the detection object in the current frame point cloud data is determined according to the following steps:

[0215] Determine a current speed of the detection object based on the three-dimensional pose data of the detection object in the current detection result and the previous three-dimensional pose data in the historical tracking chain information corresponding to the detection object;

[0216] Based on the current speed of the detection object and the latest N-1 historical speeds in the historical tracking chain information corresponding to the detection object, the current average speed of the detection object is determined; N is a preset number of frames for calculating the average speed.

[0217] Here, method 1 can first determine the current speed corresponding to the detection object in the current frame point cloud data, then add the current speed and the most recent N-1 historical speeds and average them to generate the current average speed corresponding to the detection object. In specific implementation, the moving distance of the detection object can be determined based on the three-dimensional posture data of the detection object in the current detection result and the previous three-dimensional posture data in the historical tracking chain information corresponding to the detection object; then determine the time interval between the acquisition of the current frame point cloud data and the previous frame historical point cloud data, and then determine the current speed corresponding to the detection object in the current frame point cloud data based on the determined moving distance and time interval of the detection object.

[0218] Here, when the historical speed included in the historical tracking chain information corresponding to the detection object is less than N-1, the historical speed and the current speed included in the historical tracking chain information can be used to calculate the current average speed corresponding to the detection object.

[0219] Alternatively, method two can also be based on the ath three-dimensional pose data (the current three-dimensional pose data is the 0th three-dimensional pose data) located before the current three-dimensional pose data in the historical tracking chain information corresponding to the detection object, and the three-dimensional pose data of the detection object in the current detection result, to determine the current average speed of the detection object between the current three-dimensional pose data and the ath three-dimensional pose data.

[0220] Alternatively, the first current average speed of the detection object may be generated by method 1, and the second current average speed of the detection object may be generated by method 2, and then the first current average speed and the second current average speed may be weighted and averaged to generate the current average speed of the detection object.

[0221] Here, after the current tracking chain information of at least one detection object is generated, the vehicle driving may be controlled based on the current tracking chain information of the at least one detection object.

[0222] As an optional implementation, the vehicle can be controlled to travel according to the following steps:

[0223] Step 1: for a detection object, when the number of continuous frames contained in the current tracking chain information of the detection object is greater than a preset number of frames, the detection object is regarded as a real target object.

[0224] Step 2: Control the vehicle driving based on the current tracking chain information of at least one real target object.

[0225] In step one, for each detection object (including at least one detection object indicated in the detection result and the corresponding target detection object in the historical tracking chain), if the number of consecutive frames contained in the current tracking chain information of the detection object is greater than the preset number of frames, that is, the current tracking chain corresponding to the detection object is a stable tracking chain, then the detection object is regarded as a real target object; if the number of consecutive frames contained in the current tracking chain information corresponding to the detection object is less than or equal to the preset number of frames, that is, the current tracking chain corresponding to the detection object is not a stable tracking chain, then the detection object is not regarded as a real target object.

[0226] In step 2, the vehicle can be controlled to travel based on the current tracking link information of at least one real target object. Here, the vehicle can be an autonomous driving vehicle, a vehicle equipped with an Advanced Driving Assistance System (ADAS), etc. When controlling the vehicle to travel, the vehicle can be controlled to accelerate, decelerate, turn, brake, etc., or a voice prompt message can be played to prompt the driver to control the vehicle to accelerate, decelerate, turn, brake, etc.

[0227] In the above implementation, for each detection object, when the number of consecutive frames containing the detection object corresponding to the current tracking chain information is greater than the preset number of frames, the detection object can be regarded as a real target object. Based on the current tracking chain information of at least one real target object determined, the vehicle driving can be controlled more accurately to ensure the driving safety of the vehicle.

[0228] Exemplarily, in a specific implementation, by setting a variety of interface fields in the interface code, it is possible to achieve a time-series fusion of the current detection result corresponding to the current frame point cloud data and the historical detection result corresponding to the historical point cloud data. For example, various interface fields may include but are not limited to: an enable_tracking field, which is used to indicate whether the tracking function is started. For example, if the field is set to 1, it indicates that the tracking function is started; if the field is set to 0, it indicates that the tracking function is not started; a stabilize_tracking_threshold field, which is used to indicate the preset number of frames. For example, if the field is set to 3, the preset number of frames is 3, that is, when the detection object is detected in 3 consecutive frames of point cloud data, the tracking chain corresponding to the detection object is a stable tracking chain; a terminate_tracking_threshold field, which is used to indicate the loss threshold. For example, if the field is set to 5, the loss threshold is 5, that is, when the detection object is not detected in 5 consecutive frames of point cloud data, the historical tracking chain information corresponding to the detection object can be deleted, and the object identifier corresponding to the detection object can be deleted; a maximum_trajectory_length field, which is used to indicate the maximum maintenance length of the tracking chain. For example, if the field is set to 10, the maximum maintenance length of the tracking chain is is 10; the maximum_missed_detection_recovery_length field is used to represent the frame supplement threshold. For example, if this field is set to 3, the frame supplement threshold is 3; the output_unstable_tracking_info field is used to represent whether to output unstable tracking chain information. If this field is set to 1, it represents the output of unstable tracking chain information. If this field is set to 0, it represents that unstable tracking chain information is not output; the use_velocity_in_trajectory_conf field is used to represent whether to increase the speed smoothness when calculating the detection result of the detection object. If this field is set to 1, it represents that the speed smoothness needs to be increased when calculating the detection result of the detection object. If this field is set to 0, it represents that the speed smoothness does not need to be increased when calculating the detection result of the detection object; the velocity_smoothing_cache_length field is used to represent the maximum number of frames for calculating the average speed. For example, if this field is 5, up to 5 current speeds can be used to calculate the average speed.

[0229] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0230] Based on the same concept, the present disclosure also provides a vehicle driving control device, see Figure 3 As shown, it is a schematic diagram of the architecture of the vehicle driving control device provided by the embodiment of the present disclosure, including a determination module 301 and a control module 302. Specifically:

[0231] A determination module 301 is used to determine a current detection result corresponding to the current frame point cloud data, wherein the current detection result includes three-dimensional pose data of at least one detection object;

[0232] The control module 302 is used to control the vehicle driving based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame point cloud data of the current frame point cloud data when the tracking function is turned on.

[0233] In a possible implementation manner, the control module 302, when controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object corresponding to a frame of point cloud data before the current frame of point cloud data, is used to:

[0234] For any detection object in the current detection result, detecting whether there is historical tracking chain information corresponding to the detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data;

[0235] In the case where there is historical tracking chain information corresponding to the detection object, based on the three-dimensional pose data of the detection object in the current detection result and the historical tracking chain information corresponding to the detection object, generate current tracking chain information of the detection object; wherein the current tracking chain information of the detection object includes an object identifier, which is consistent with the object identifier in the historical tracking chain information corresponding to the detection object;

[0236] The vehicle driving is controlled based on the current tracking link information of at least one detected object.

[0237] In a possible implementation manner, the control module 302, when controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, is further configured to:

[0238] For a detection object in the current detection result, detecting whether there is historical tracking chain information corresponding to the detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data;

[0239] In the absence of historical tracking chain information corresponding to the detection object, generating current tracking chain information of the detection object based on the three-dimensional pose data of the detection object in the current detection result; the current tracking chain information of the detection object includes an object identifier assigned to the detection object;

[0240] The vehicle driving is controlled based on the current tracking link information of at least one detected object.

[0241] In a possible implementation manner, the control module 302, when controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object corresponding to a frame of point cloud data before the current frame of point cloud data, is used to:

[0242] Determine a target detection object having corresponding historical tracking chain information but not appearing in the current detection result based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data;

[0243] For the target detection object, determine the first number of frames of point cloud data in which the target detection object has not been detected continuously in the current frame point cloud data and at least one frame of historical point cloud data before the current frame point cloud data; if the first number of frames reaches a set loss threshold, delete the historical tracking chain information;

[0244] Determine the current tracking chain information of at least one detection object in the current detection result according to the current detection result and the historical tracking chain information of the detection objects other than the target detection object whose historical tracking chain information is deleted among at least one detection object in the previous frame of point cloud data of the current frame;

[0245] The vehicle driving is controlled according to the current tracking chain information of at least one detection object in the current detection result.

[0246] In a possible implementation manner, the control module 302, when controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, is used to:

[0247] Determine a target detection object having a corresponding historical tracking chain but not appearing in the current detection result based on the current detection result and the historical tracking chain information of at least one detection object corresponding to the previous frame point cloud data of the current frame point cloud data;

[0248] For the target detection object, when it is determined that the historical tracking chain corresponding to the target detection object meets the preset stability condition, the number of historical continuous supplementary frames corresponding to the target detection object is determined based on the historical tracking chain information corresponding to the target detection object; wherein the preset stability condition includes that the number of continuous frames corresponding to the historical tracking chain information containing the target detection object is greater than the preset number of frames;

[0249] Determine the current tracking chain information according to the number of historical continuous supplemented frames and a set supplemented frame threshold;

[0250] Based on the current tracking link information, the vehicle is controlled to travel.

[0251] In a possible implementation manner, when determining the current tracking chain information according to the number of historical continuous supplemented frames and a set supplemented frame threshold, the control module 302 is configured to:

[0252] When the number of historical continuous supplementary frames is less than a set supplementary frame threshold, based on the historical tracking chain information corresponding to the target detection object, the predicted three-dimensional pose data of the target detection object in the current frame point cloud data is generated; and based on the predicted three-dimensional pose data and the historical tracking chain information corresponding to the target detection object, the current tracking chain information is generated;

[0253] When the number of historical continuous supplementary frames is equal to the set supplementary frame threshold, the historical tracking chain information corresponding to the target detection object is determined as the current tracking chain information.

[0254] In a possible implementation manner, the control module 302, when controlling the vehicle to travel based on the current detection result and the historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, is used to:

[0255] For any detection object in the current detection result, when the tracking chain length indicated by the historical tracking chain information corresponding to the detection object is equal to the set maximum maintenance length, the historical tracking data of the detection object corresponding to the earliest frame of point cloud data acquired in the historical tracking chain information is deleted, and the generated current tracking data of the detection object is added to the historical tracking chain information to generate the current tracking chain information;

[0256] Based on the current tracking link information, the vehicle is controlled to travel.

[0257] In a possible implementation manner, the control module 302 is configured to determine the current average speed of a detection object in the current frame point cloud data according to the following steps when the current tracking chain information of a detection object includes the current average speed of the detection object:

[0258] Determine a current speed of the detection object based on the three-dimensional pose data of the detection object in the current detection result and the previous three-dimensional pose data in the historical tracking chain information corresponding to the detection object;

[0259] Based on the current speed of the detection object and the latest N-1 historical speeds in the historical tracking chain information corresponding to the detection object, the current average speed of the detection object is determined; N is a preset number of frames for calculating the average speed.

[0260] In a possible implementation manner, the control module 302 is used to control the vehicle to travel according to the following steps:

[0261] For a detection object, when the number of consecutive frames contained in the current tracking chain information of the detection object is greater than the preset number of frames, the detection object is regarded as a real target object;

[0262] The vehicle driving is controlled based on the current tracking link information of at least one real target object determined.

[0263] In a possible implementation manner, the determination module 301 is used to determine the current detection result corresponding to the current frame point cloud data according to the following steps:

[0264] Based on the current frame point cloud data, generating at least one target sparse matrix corresponding to the current frame point cloud data; the target sparse matrix is ​​used to characterize whether there is a detection object at different positions of the target scene;

[0265] Based on the at least one target sparse matrix and the current frame point cloud data, a current detection result corresponding to the current frame point cloud data is determined.

[0266] In a possible implementation manner, the determination module 301, when generating at least one target sparse matrix corresponding to the current frame point cloud data based on the current frame point cloud data, is used to:

[0267] Based on the current frame point cloud data, a target sparse matrix corresponding to each layer of convolutional modules in a neural network used to detect the detection object is determined.

[0268] In a possible implementation manner, the determination module 301, when determining the target sparse matrix corresponding to each convolution module in the neural network based on the current frame point cloud data, is used to:

[0269] Based on the current frame point cloud data, generating an initial sparse matrix;

[0270] Based on the initial sparse matrix, a target sparse matrix matching a target size of a feature map input to each convolution module layer of the neural network is determined.

[0271] In a possible implementation manner, the determination module 301, when generating an initial sparse matrix based on the current frame point cloud data, is used to:

[0272] Determine a target area corresponding to the current frame point cloud data, and divide the target area into a plurality of grid areas according to a preset number of grids;

[0273] Determine the matrix element value corresponding to each grid area based on the grid area where the point cloud point corresponding to the current frame point cloud data is located;

[0274] Based on the matrix element values ​​corresponding to each grid area, an initial sparse matrix corresponding to the current frame point cloud data is generated.

[0275] In a possible implementation, the determination module 301, when determining a target sparse matrix matching a target size of a feature map input by each convolution module of the neural network based on the initial sparse matrix, is used to:

[0276] Based on the initial sparse matrix, determine the output sparse matrix corresponding to each convolution module in the neural network, and use the output sparse matrix as the target sparse matrix; or

[0277] Based on the initial sparse matrix, determining the input sparse matrix corresponding to each convolution module in the neural network, and using the input sparse matrix as the target sparse matrix; or,

[0278] Based on the initial sparse matrix, the input sparse matrix and output sparse matrix corresponding to each layer of the convolution module in the neural network are determined, the input target sparse matrix and the output target sparse matrix are fused to obtain a fused sparse matrix, and the fused sparse matrix is ​​used as the target sparse matrix corresponding to the convolution module of this layer.

[0279] In a possible implementation, the determination module 301, when determining the input sparse matrix corresponding to each convolution module in the neural network based on the initial sparse matrix, is used to:

[0280] Using the initial sparse matrix as the input sparse matrix corresponding to the first convolution module of the neural network;

[0281] Based on the input sparse matrix corresponding to the i-1th layer convolution module, determine the input sparse matrix corresponding to the i-th layer convolution module and matching the target size of the feature map input by the i-th layer convolution module; wherein i is a positive integer greater than 1 and less than n+1, and n is the number of layers of the convolution module of the neural network.

[0282] In a possible implementation manner, the determination module 301, when determining the output sparse matrix corresponding to each convolution module in the neural network based on the initial sparse matrix, is used to:

[0283] Determining an output sparse matrix corresponding to the neural network based on a size threshold of the detection object and the initial sparse matrix;

[0284] Based on the output sparse matrix, generating an output sparse matrix corresponding to the n-th layer convolution module and matching the target size of the feature map input by the n-th layer convolution module;

[0285] Based on the output sparse matrix corresponding to the j+1th convolution module, an output sparse matrix corresponding to the jth convolution module is generated, which matches the target size of the feature map input to the jth convolution module, wherein j is a positive integer greater than or equal to 1 and less than n, and n is the number of layers of the convolution module of the neural network.

[0286] In some embodiments, the functions or templates contained in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0287] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Figure 4 As shown, it is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, including a processor 401, a memory 402, and a bus 403. Among them, the memory 402 is used to store execution instructions, including a memory 4021 and an external memory 4022; the memory 4021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 401, and the data exchanged with the external memory 4022 such as a hard disk. The processor 401 exchanges data with the external memory 4022 through the memory 4021. When the electronic device 400 is running, the processor 401 communicates with the memory 402 through the bus 403, so that the processor 401 executes the following instructions:

[0288] Determine a current detection result corresponding to the current frame point cloud data, wherein the current detection result includes three-dimensional pose data of at least one detection object;

[0289] When the tracking function is turned on, the vehicle driving is controlled based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame point cloud data of the current frame point cloud data.

[0290] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle driving control method described in the above method embodiment are executed.

[0291] The computer program product of the vehicle driving control method provided in the embodiment of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the vehicle driving control method described in the above method embodiment. Please refer to the above method embodiment for details, which will not be repeated here.

[0292] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0293] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0294] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0295] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0296] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A vehicle driving control method, characterized in that: include: Determine a current detection result corresponding to the current frame point cloud data, wherein the current detection result includes three-dimensional pose data of at least one detection object; When the tracking function is turned on, the vehicle is controlled to travel based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data; Determine the current detection result corresponding to the current frame point cloud data according to the following steps: Based on the current frame point cloud data, generating at least one target sparse matrix corresponding to the current frame point cloud data; the target sparse matrix is ​​used to characterize whether there is a detection object at different positions of the target scene; based on the at least one target sparse matrix and the current frame point cloud data, determining a current detection result corresponding to the current frame point cloud data; The method of generating at least one target sparse matrix corresponding to the current frame point cloud data based on the current frame point cloud data includes: determining, based on the current frame point cloud data, the target sparse matrix corresponding to each layer of convolution module in the neural network used to detect the detection object.

2. The method according to claim 1, characterized in that Controlling vehicle travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, including: For a detection object in the current detection result, detecting whether there is historical tracking chain information corresponding to the detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data; In the case where there is historical tracking chain information corresponding to the detection object, based on the three-dimensional pose data of the detection object in the current detection result and the historical tracking chain information corresponding to the detection object, generate current tracking chain information of the detection object; wherein the object identifier included in the current tracking chain information of the detection object is consistent with the object identifier in the historical tracking chain information corresponding to the detection object; The vehicle driving is controlled based on the current tracking link information of at least one detected object.

3. The method according to claim 1, characterized in that Controlling vehicle travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, including: For a detection object in the current detection result, detecting whether there is historical tracking chain information corresponding to the detection object in the historical tracking chain information of at least one detection object corresponding to the previous frame of point cloud data; In the absence of historical tracking chain information corresponding to the detection object, generating current tracking chain information of the detection object based on the three-dimensional pose data of the detection object in the current detection result; the current tracking chain information of the detection object includes an object identifier assigned to the detection object; The vehicle driving is controlled based on the current tracking link information of at least one detected object.

4. The method according to claim 1, characterized in that: Controlling vehicle travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, including: Determine a target detection object having corresponding historical tracking chain information but not appearing in the current detection result based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data; For the target detection object, determine the first number of frames of point cloud data in which the target detection object has not been detected continuously in the current frame point cloud data and at least one frame of historical point cloud data before the current frame point cloud data; if the first number of frames reaches a set loss threshold, delete the historical tracking chain information; Determine the current tracking chain information of at least one detection object in the current detection result according to the current detection result and the historical tracking chain information of the detection objects other than the target detection object whose historical tracking chain information is deleted among at least one detection object in the previous frame of point cloud data of the current frame; The vehicle driving is controlled according to the current tracking chain information of at least one detection object in the current detection result.

5. The method according to claim 1, characterized in that Controlling vehicle travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, including: Determine a target detection object having corresponding historical tracking chain information but not appearing in the current detection result based on the current detection result and the historical tracking chain information of at least one detection object in the previous frame of point cloud data of the current frame of point cloud data; For the target detection object, when it is determined that the historical tracking chain corresponding to the target detection object meets the preset stability condition, the number of historical continuous supplementary frames corresponding to the target detection object is determined based on the historical tracking chain information corresponding to the target detection object; wherein the preset stability condition includes that the number of continuous frames corresponding to the historical tracking chain information containing the target detection object is greater than the preset number of frames; Determine the current tracking chain information according to the historical continuous supplementary frame number and the set supplementary frame threshold; Based on the current tracking link information, the vehicle is controlled to travel.

6. The method according to claim 5, characterized in that Determining the current tracking chain information according to the historical continuous supplementary frame number and the set supplementary frame threshold includes: When the number of historical continuous supplementary frames is less than a set supplementary frame threshold, based on the historical tracking chain information corresponding to the target detection object, the predicted three-dimensional pose data of the target detection object in the current frame point cloud data is generated; and based on the predicted three-dimensional pose data and the historical tracking chain information corresponding to the target detection object, the current tracking chain information is generated; When the number of historical continuous supplementary frames is equal to the set supplementary frame threshold, the historical tracking chain information corresponding to the target detection object is determined as the current tracking chain information.

7. The method according to claim 1, characterized in that Controlling vehicle travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data, including: For any detection object in the current detection result, when the tracking chain length indicated by the historical tracking chain information corresponding to the detection object is equal to the set maximum maintenance length, the historical tracking data of the detection object corresponding to the earliest frame of point cloud data acquired in the historical tracking chain information is deleted, and the generated current tracking data of the detection object is added to the historical tracking chain information to generate current tracking chain information; Based on the current tracking link information, the vehicle is controlled to travel.

8. The method according to any one of claims 2 to 7, characterized in that: The current tracking chain information of a detection object includes the current average speed of the detection object. The current average speed of the detection object in the current frame point cloud data is determined according to the following steps: Determine a current speed of the detection object based on the three-dimensional pose data of the detection object in the current detection result and the previous three-dimensional pose data in the historical tracking chain information corresponding to the detection object; Determine the current average speed of the detection object based on the current speed of the detection object and the most recent N-1 historical speeds in the historical tracking chain information corresponding to the detection object; N is the preset number of frames used to calculate the average speed.

9. The method according to any one of claims 2 to 7, characterized in that: Control the vehicle according to the following steps: For a detection object, when the number of consecutive frames contained in the current tracking chain information of the detection object is greater than the preset number of frames, the detection object is regarded as a real target object; The vehicle driving is controlled based on the current tracking link information of at least one real target object determined.

10. The method according to claim 1, characterized in that Based on the current frame point cloud data, determining the target sparse matrix corresponding to each convolution module in the neural network, including: Based on the current frame point cloud data, generating an initial sparse matrix; Based on the initial sparse matrix, a target sparse matrix matching a target size of a feature map input to each convolution module layer of the neural network is determined.

11. The method according to claim 10, characterized in that Based on the current frame point cloud data, an initial sparse matrix is ​​generated, including: Determine a target area corresponding to the current frame point cloud data, and divide the target area into a plurality of grid areas according to a preset number of grids; Determine the matrix element value corresponding to each grid area based on the grid area where the point cloud point corresponding to the current frame point cloud data is located; Based on the matrix element values ​​corresponding to each grid area, an initial sparse matrix corresponding to the current frame point cloud data is generated.

12. The method according to claim 10 or 11, characterized in that: Determining a target sparse matrix that matches a target size of a feature map input to each convolution module of the neural network based on the initial sparse matrix includes: Based on the initial sparse matrix, determine the output sparse matrix corresponding to each convolution module in the neural network, and use the output sparse matrix as the target sparse matrix; or Based on the initial sparse matrix, determining the input sparse matrix corresponding to each convolution module in the neural network, and using the input sparse matrix as the target sparse matrix; or, Based on the initial sparse matrix, the input sparse matrix and output sparse matrix corresponding to each layer of the convolution module in the neural network are determined, the input target sparse matrix and the output target sparse matrix are fused to obtain a fused sparse matrix, and the fused sparse matrix is ​​used as the target sparse matrix corresponding to the convolution module of this layer.

13. The method according to claim 12, characterized in that The step of determining, based on the initial sparse matrix, an input sparse matrix corresponding to each convolution module in the neural network comprises: Using the initial sparse matrix as the input sparse matrix corresponding to the first convolution module of the neural network; Based on the input sparse matrix corresponding to the i-1th layer convolution module, determine the input sparse matrix corresponding to the i-th layer convolution module and matching the target size of the feature map input by the i-th layer convolution module; wherein i is a positive integer greater than 1 and less than n+1, and n is the number of layers of the convolution module of the neural network.

14. The method according to claim 12, characterized in that The step of determining, based on the initial sparse matrix, an output sparse matrix corresponding to each convolutional module in the neural network comprises: Determining an output sparse matrix corresponding to the neural network based on a size threshold of the detection object and the initial sparse matrix; Based on the output sparse matrix, generating an output sparse matrix corresponding to the n-th layer convolution module and matching the target size of the feature map input by the n-th layer convolution module; Based on the output sparse matrix corresponding to the j+1th convolution module, an output sparse matrix corresponding to the jth convolution module is generated, which matches the target size of the feature map input to the jth convolution module, wherein j is a positive integer greater than or equal to 1 and less than n, and n is the number of layers of the convolution module of the neural network.

15. A vehicle driving control device, characterized in that: include: A determination module, used to determine a current detection result corresponding to the current frame point cloud data, wherein the current detection result includes three-dimensional pose data of at least one detection object; A control module, configured to control the vehicle to travel based on the current detection result and historical tracking chain information of at least one detection object in a frame of point cloud data before the current frame of point cloud data when the tracking function is turned on; The determination module is used to determine the current detection result corresponding to the current frame point cloud data according to the following steps: Based on the current frame point cloud data, generating at least one target sparse matrix corresponding to the current frame point cloud data; the target sparse matrix is ​​used to characterize whether there is a detection object at different positions of the target scene; based on the at least one target sparse matrix and the current frame point cloud data, determining a current detection result corresponding to the current frame point cloud data; The determination module, when generating at least one target sparse matrix corresponding to the current frame point cloud data based on the current frame point cloud data, is used to: determine, based on the current frame point cloud data, the target sparse matrix corresponding to each layer of convolution module in the neural network used to detect the detection object.

16. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the vehicle driving control method as described in any one of claims 1 to 14 are performed.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle driving control method as claimed in any one of claims 1 to 14 are executed.

Images