Target detection method, vehicle, computing device and computer readable storage medium

Abstract

The invention provides a target detection method, a vehicle, a computing device and a computer readable storage medium, and the method comprises the steps: obtaining historical sensor data corresponding to each of a plurality of historical frames, and enabling the plurality of historical frames to be before a current frame in a time sequence; generating a reference detection mark corresponding to at least one target object in the current frame according to historical sensor data of each historical frame; and generating a target detection result in the current frame according to each reference detection mark. Through the method provided by the invention, the problem of missing detection or wrong detection caused by a shielding problem is avoided, the detection effect of executing the target detection method is enhanced, and the detection accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a target detection method. The present disclosure also relates to a vehicle using the target detection method, a computing device, and a computer-readable storage medium. Background Art

[0002] In the task of 3D point cloud target detection, it is necessary to predict the 3D bounding box of the object based on the 3D point cloud information. The current 3D target detection method that uses a single frame point cloud as input only outputs the detection result of the current frame, and often has no prediction box when the target is occluded. Some other detection methods have also introduced a multi-frame result fusion method for target detection, such as encoding and fusing feature information between multiple frames through a complex multi-layer network structure. Although this can improve the detection results, it requires the use of a specific network structure, additional model parameters and multi-stage model training, and introduces a high delay in the model processing process, which is not suitable for use scenarios with high real-time requirements. Therefore, a new target detection method is urgently needed to solve the current problem of inaccurate target detection. Summary of the invention

[0003] In view of this, an embodiment of the present disclosure provides a target detection method. The present disclosure also relates to a vehicle with the target detection method applied thereto, a computing device, and a computer-readable storage medium to solve the above problems existing in the prior art.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a target detection method, comprising:

[0005] Acquire historical sensor data corresponding to each of a plurality of historical frames, wherein the plurality of historical frames are before the current frame in time series;

[0006] generating a reference detection marker corresponding to at least one target object in the current frame based on the historical sensor data of each historical frame;

[0007] An object detection result is generated in the current frame according to each reference detection mark.

[0008] According to a second aspect of an embodiment of the present disclosure, there is provided a target detection method, including:

[0009] Acquire vehicle historical sensor data corresponding to each of a plurality of historical frames, wherein the plurality of historical frames are before the current frame in time sequence;

[0010] generating a reference detection mark corresponding to at least one detected vehicle in the current frame based on the vehicle historical sensor data of each historical frame;

[0011] A target vehicle detection result is generated in the current frame according to each reference detection mark.

[0012] According to a third aspect of an embodiment of the present disclosure, a vehicle is provided, comprising: a processor, wherein the processor implements the steps of the above-mentioned target detection method when executing the computer instructions.

[0013] According to a fourth aspect of an embodiment of the present disclosure, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the target detection method when executing the computer instructions.

[0014] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed by a processor, the steps of the target detection method are implemented.

[0015] The target detection method provided by the present disclosure obtains historical sensor data corresponding to each of a plurality of historical frames, wherein, in a time series, the plurality of historical frames are before a current frame; generates a reference detection mark corresponding to at least one target object in the current frame based on the historical sensor data of each historical frame; and generates a target detection result in the current frame based on each reference detection mark.

[0016] The embodiments of the present disclosure implement predicting reference detection marks in the current frame based on historical sensor data of multiple historical frames, generating target detection results in the current frame through each reference detection mark, and making predictions through historical sensor data, thereby avoiding missed detections or false detections due to occlusion problems, enhancing the detection effect of executing the target detection method, and improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an architecture diagram of a target detection system provided by an embodiment of the present disclosure;

[0018] Figure 2 is a flow chart of a target detection method provided by an embodiment of the present disclosure;

[0019] Figure 3 is a schematic diagram of a single-wheel drive model and a dual-wheel drive model provided by an embodiment of the present disclosure;

[0020] Figure 4 is a processing flow chart of a landmark detection method applied to a roadside detection device provided by an embodiment of the present disclosure;

[0021] Figure 5 is a schematic diagram of a target detection method provided by an embodiment of the present disclosure;

[0022] Figure 6 is a flow chart of another target detection method provided by an embodiment of the present disclosure;

[0023] Figure 7 It is a structural block diagram of a computing device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Many specific details are described in the following description to facilitate a full understanding of the present disclosure. However, the present disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present disclosure, so the present disclosure is not limited by the specific implementation disclosed below.

[0025] The terms used in one or more embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present disclosure. The singular forms of "a", "said" and "the" used in one or more embodiments of the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present disclosure, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present disclosure, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0028] First, the terms involved in one or more embodiments of the present disclosure are explained.

[0029] Target detection: An algorithm is used to find the location of a target object in the raw sensor data. A rectangle or cuboid is usually used to represent the location of the target object in two-dimensional (2D) or three-dimensional (3D) space.

[0030] Point cloud: Data of the surrounding environment collected by LiDAR, represented by a set of sparse three-dimensional space points.

[0031] Intersection-to-union ratio: The ratio of the volume of the intersection of two 3D detection boxes to the volume of their union.

[0032] Detection box pose p = (x, y, φ): the x-axis and y-axis coordinates of the center point of the detection box, and the orientation angle φ.

[0033] In the task of 3D point cloud target detection, 3D point cloud information is needed to predict the 3D bounding box of the object. The current 3D target detection method that uses a single-frame point cloud as input only outputs the detection result of the current frame, and there is often no prediction box when the target is occluded. Some detection methods introduce multi-frame result fusion to perform target detection, and use timing information to further improve the detection effect. For example, MppNet uses a complex three-layer structure to encode and fuse features between multiple frames. However, this method requires a specific network structure and cannot be directly applied to existing 3D detectors. In addition, this method requires additional model parameters and multi-stage training. At the same time, this method also introduces higher latency, making it difficult to cope with some application scenarios with high real-time requirements.

[0034] Based on the above problems, the present disclosure provides a target detection method, and the present disclosure also relates to a vehicle with the target detection method applied thereto, a computing device, and a computer-readable storage medium. The following embodiments will describe them in detail one by one.

[0035] See also Figure 1 , Figure 1 An architecture diagram of an object detection system provided by an embodiment of the present disclosure is shown. The object detection system may include a client 100 and a server 200;

[0036] The client 100 is used to send a plurality of historical frames to the server 200, wherein, in time sequence, the plurality of historical frames are before the current frame, and each historical frame includes historical sensor data;

[0037] The server 200 is used to obtain historical sensor data corresponding to each of multiple historical frames, generate a reference detection mark corresponding to at least one target object in the current frame based on the historical sensor data of each historical frame, and generate a target detection result in the current frame based on each reference detection mark; and send the target detection result to the client 100.

[0038] The client 100 is also used to receive the target detection result sent by the server 200.

[0039] The target detection system may include multiple clients 100 and a server 200, wherein the client 100 may be referred to as a terminal-side device, and the server 200 may be referred to as a cloud-side device. A communication connection may be established between the multiple clients 100 through the server 200. In an autonomous driving scenario, the server 200 is used to provide target detection services between the multiple clients 100. The multiple clients 100 may serve as a sender or a receiver respectively, and realize communication through the server 200.

[0040] The user can interact with the server 200 through the client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the autonomous driving scenario, the user can publish a data stream to the server 200 through the client 100, and the server 200 generates a target detection result based on the data stream and pushes the target detection result to the current client or other clients that have established communication.

[0041] The client 100 and the server 200 are connected via a network. The network provides a medium for a communication link between the client 100 and the server 200. The network may include various connection types, such as wired or wireless communication links or optical fiber cables, etc. The data transmitted by the client 100 may need to be encoded, transcoded, compressed, etc. before being released to the server 200.

[0042] The client 100 can be deployed in an electronic device and needs to rely on the device to run or some APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0043] The server 200 may include servers that provide various services, such as servers that provide communication services to multiple clients, servers for background training that provide support for models used on clients, and servers that process data sent by clients. It should be noted that the server 200 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0044] It is worth noting that the target detection method provided in the embodiments of the present disclosure is generally executed by the server, but in other embodiments of the present disclosure, the client may also have similar functions to the server, thereby executing the target detection method provided in the embodiments of the present disclosure. In other embodiments, the target detection method provided in the embodiments of the present disclosure may also be jointly executed by the client and the server.

[0045] Figure 2 A flow chart of a target detection method provided according to an embodiment of the present disclosure is shown, which specifically includes the following steps:

[0046] Step 202: Acquire historical sensor data corresponding to each of a plurality of historical frames, wherein, in time series, the plurality of historical frames are before the current frame.

[0047] Among them, the historical frame and the current frame are multiple images in a time sequence. In practical applications, the target object is detected through the images collected by the image acquisition device. The image acquisition device collects images according to the time sequence. For example, when the image acquisition device is a camera, the historical frame and the current frame are the video frames in the video collected by the camera. The current frame is the image currently collected by the image acquisition device in the current time sequence. The historical frame is the image collected by the image acquisition device before the current frame.

[0048] In this embodiment, historical sensor data refers to the sensor acquisition information in each historical frame. In actual applications, when the image acquisition device is a lidar, the historical sensor data corresponding to the historical frame is the point cloud data collected by the lidar; when the image acquisition device is a camera, the historical sensor data corresponding to the historical frame is the image data collected by the camera.

[0049] Acquiring multiple historical frames and the corresponding historical sensor data in each historical frame can facilitate effective prediction based on the historical sensor data in the historical frames in the subsequent processing process. In order to improve the prediction efficiency, the multiple historical frames are historical frames adjacent to the current frame. For example, when the current frame is the tth frame in the time series, the multiple historical frames may be the t-1th frame, the t-2th frame, the t-3th frame...the tnth frame. In order to make subsequent predictions more accurate and reduce the amount of calculation, the number n can be set to be less than or equal to a preset threshold. In a specific embodiment provided in the present disclosure, the value of n can be an integer between 3 and 10. Preferably, n can be 3, 4 or 5. For example, when n is 3, it is to obtain 3 historical frames before the current frame.

[0050] Step 204: Generate a reference detection mark corresponding to at least one target object in the current frame according to the historical sensor data of each historical frame.

[0051] After determining each historical frame, at least one reference detection mark of the target object is generated in the current frame according to the historical sensor data of each historical frame, wherein the reference detection mark specifically refers to the detection mark of the target object predicted according to the historical frame.

[0052] The target object in the historical frame can be detected based on the historical sensor data, so that the position of the target object in the current frame can be predicted based on the historical sensor data.

[0053] In practical applications, different reference detection marks can be generated in the current frame based on historical sensor data from different historical frames. For example, when the same target object exists in three historical frames, reference detection mark 1 of the target object in the current frame may be predicted based on the first historical frame; reference detection mark 2 of the target object in the current frame may be predicted based on the second historical frame; and reference detection mark 3 of the target object in the current frame may be predicted based on the third historical frame.

[0054] In practical applications, the reference detection mark may be in the form of a detection frame, that is, the reference detection mark is a reference detection frame. However, in some embodiments, the reference detection mark may also be in the form of a marking point to mark the target object.

[0055] In a specific embodiment provided by the present disclosure, the historical sensor data includes reference point cloud data. Accordingly, based on the historical sensor data of each historical frame, a reference detection mark corresponding to at least one target object is generated in the current frame, including:

[0056] For each history frame:

[0057] Clustering the reference point cloud data in the historical frame to obtain at least one reference target object;

[0058] Predicting reference pose information of each reference target object in the current frame;

[0059] A reference detection mark corresponding to each reference target object in the current frame is determined according to each reference pose information.

[0060] For any historical frame, when the historical sensor data in the historical frame is reference point cloud data, the reference point cloud data can be clustered to obtain corresponding initial clustering results. Specifically, each initial clustering result corresponds to a reference target object. For example, a historical frame may include multiple reference point cloud data. After clustering the reference point cloud data in this historical frame, three initial clustering results are obtained, and three reference target objects can be determined.

[0061] After determining each reference target object in the historical frame, the reference pose information of each reference target object in the historical frame in the current frame can be predicted based on the relationship between the historical frame and the current frame. Specifically, the reference pose information refers to the predicted position information and pose information of the reference target object in the current frame. The position information is used to represent the predicted position information of the reference target object in the current frame, while the pose information is used to represent the predicted angle information of the reference target object in the current frame. Afterwards, the reference detection mark corresponding to each reference target object in the current frame can be determined based on the reference pose information of each reference target object.

[0062] For example, the reference pose information of the reference target object in the current frame t can be predicted based on the information of the reference target object in the historical frame t-1. Then, the reference detection mark of the reference target object in the current frame can be determined based on the reference pose information.

[0063] Further, in one or more embodiments provided in the present disclosure, the above-mentioned prediction of reference pose information of each reference target object in the current frame includes:

[0064] For each reference target object:

[0065] Acquire initial reference pose information of the reference target object in the historical frame.

[0066] In the embodiments of the present disclosure, the same processing method may be performed on any reference target object. For ease of description, one of the reference target objects is taken as an example for explanation.

[0067] Specifically, the initial reference pose information refers to the pose information of the reference target object in its corresponding historical frame. For example, the pose information of the reference target object in its corresponding historical frame t-1 is the initial reference pose information.

[0068] After the initial reference pose information of the reference target object is determined, it can be predicted to predict the reference pose information of the reference target object in the current frame according to the initial reference pose information. Specifically, the motion of the reference target object can be predicted by a preset motion prediction model.

[0069] In practical applications, a unified motion prediction model can be used to predict each reference target object, or multiple motion prediction models can be designed to adapt to different types of reference target objects. This is not limited in the present disclosure.

[0070] The above-mentioned prediction of reference pose information of each reference target object in the current frame also includes:

[0071] For each reference target object:

[0072] The initial reference pose information is input into a motion prediction model to obtain reference pose information of the reference target object in the current frame.

[0073] Specifically, after the initial reference pose information of each reference target object is determined, it can be input into the motion prediction model to predict the reference position information of the reference target object in the current frame based on the initial reference pose information of the reference target object in the historical frame.

[0074] The motion prediction model refers to a mathematical model used to predict the motion trajectory of the target object, which can predict the pose information of the target object in the current frame based on the pose information of the reference target object in the historical frame. Specifically, the initial reference pose information needs to be input into the motion prediction model so that the motion prediction model can make predictions.

[0075] In a specific implementation provided by the present disclosure, the initial reference pose information is input into a motion prediction model, specifically including:

[0076] Acquiring motion parameter information of the reference target object;

[0077] The initial reference pose information and the motion parameter information are input into a motion prediction model.

[0078] In practical applications, in addition to inputting the initial reference pose information into the motion prediction model, the motion information of the reference target object also needs to be input into the motion prediction model. Specifically, the motion information of the reference target object can be understood as the motion parameter information of the reference target object, such as motion speed, motion angular velocity, etc. After obtaining the motion parameter information, the motion parameter information and the initial reference pose information can be input into the motion prediction model.

[0079] Furthermore, the motion parameter information of the reference target object can be determined based on at least two frames for the reference target object. For example, the motion parameter information of the reference target object can be determined based on the position information of the reference target object in several adjacent historical frames.

[0080] Specifically, the motion prediction model refers to a mathematical model used to predict the reference target object, and is used to predict the position and posture of the reference target object at a certain moment in the future based on the current position and posture information and motion parameters of the reference target object. For details, see the following formula (1).

[0081] p t =p0+F M (p0,t;θ M ) Formula (1)

[0082] Among them, p0 is the current posture information, F M (p0,t;θ M ) is the forward model of the motion prediction model, t is the predicted time, θ M It is the motion parameter information.

[0083] In a specific implementation manner provided in the present disclosure, the motion prediction model may be one motion prediction model, or different motion prediction models may be set for different types of reference target objects, which is not limited in the embodiments provided in the present disclosure.

[0084] In order to better predict the reference target object, in the embodiment of the present disclosure, it is preferred to use multiple motion prediction models to predict different types of reference target objects. Based on this, the method further includes:

[0085] Determining object attribute information of the reference target object;

[0086] A motion prediction model corresponding to the reference target object is determined according to the object attribute information.

[0087] The object attribute information specifically refers to information used to reflect the use of different motion prediction models for the reference target object. In practical applications, the motion prediction model may include a uniform speed model, a single-wheel drive model, a dual-wheel drive model, or other models.

[0088] Uniform velocity model: In the application scenario of this model, it is assumed that the object has a constant velocity in the x and y axes. Based on the constant velocity and time, the position information of the object at different times can be predicted. In practical applications, if only a unified motion prediction model is designed, the uniform velocity model can be used, for example.

[0089] Single-wheel drive model: Assume that the object is driven by a wheel with the same posture as the object. This object can be, for example, a person, an animal, a unicycle, etc. The single-wheel drive model usually involves motion parameters θ U and the forward model F U The motion parameters are shown in the following formula (2), and the forward model is shown in the following formula (3):

[0090] θ U =(V,ω) Formula (2)

[0091]

[0092] Among them, θ in formula (2) U represents the motion parameter, V refers to the motion speed of the reference target object, and ω refers to the angular velocity of the reference target object. φ0 represents the orientation angle at the starting moment, and φ t The single-wheel drive model provides a simple way to model the motion of a steerable rigid body.

[0093] Two-wheel drive model: Assume that the object is driven by two wheels distributed on the central axis, such as a motorcycle, bicycle, or car. The front wheel in the two-wheel drive model is the rotating wheel, and the rear wheel is the driving wheel. The two-wheel drive model also involves motion parameters and forward models. Specifically, the motion parameter θ of the two-wheel drive model is B See the following formula (4) for details. The forward model F B See the following formula (5):

[0094] θ B =(V,β,l r ) Formula (4)

[0095]

[0096] Where V refers to the velocity of the reference target object, β refers to the slip angle of the heading deviation from the reference target object, and l r is the distance between the center of mass of the reference target object and the two wheels. φ0 represents the orientation angle at the starting moment, and φ t Represents the heading angle at time t after the start time.

[0097] See also Figure 3 , Figure 3Schematic diagrams of a single-wheel drive model and a dual-wheel drive model provided by the present disclosure are shown. Figure 3 As shown, Figure 3 a in the figure is a single-wheel drive model. Figure 3 b in the figure is a two-wheel drive model. A single-wheel drive model is, for example, a model of a single-axle system, where the number of wheels on the single axle is not limited. A two-wheel drive model is, for example, a model of a two-axle system, where there is a spacing between the two axles, and the number of wheels on either of the two axles is not limited.

[0098] For the single-wheel drive model, the wheel moves at a speed V and rotates at an angular velocity ω, and its motion parameters are given by the above formula (2). For the two-wheel drive model, the slip angle of the vehicle speed V deviating from the vehicle heading is β, l r is the distance between the center of mass of the reference target object and the two wheels. Its motion parameters are given by the above formula (4).

[0099] In a specific embodiment provided by the present disclosure, the object attribute information of the reference target object is first obtained. Furthermore, the object attribute information is used to determine the reference information of the movement mode of the reference target object. For example, it is determined that the reference target object is single-wheel drive, or it is determined that the reference target object is dual-wheel drive. Accordingly, when the reference target object is single-wheel drive (for example, the reference target object is a unicycle, etc.), the motion prediction model is a single-wheel drive model; when the reference target object is a person, a single-wheel drive model can also be used as a motion prediction model; when the reference target object is dual-wheel drive (for example, the reference target object is a bicycle, a motorcycle, a car, etc.), the motion prediction model is a dual-wheel drive model.

[0100] After determining the motion prediction model corresponding to the reference target object, the initial pose reference information and motion parameter information of the reference target object are input into the motion prediction model, and the motion prediction model outputs the reference pose information of the reference target object in the current frame.

[0101] After obtaining the reference pose information of each reference target object in the current frame, it is convenient to further detect the target object in the current frame based on the reference pose information of each reference target object in the current frame, so as to improve the detection accuracy of the target object through the historical sensor data or other relevant information in the historical frame.

[0102] In practical applications, in addition to predicting the target object in the current frame based on the historical frames, the sensor data in the current frame actually collected in the current frame can also be processed. Specifically, in one or more embodiments provided by the present disclosure, the method further includes:

[0103] Acquire current sensor data corresponding to the current frame;

[0104] At least one initial detection marker corresponding to the at least one target object is generated in the current frame based on the current sensor data.

[0105] The current sensor data specifically refers to the sensor data collected by the image acquisition device in the current frame. When the image acquisition device is a laser radar, the current sensor data corresponding to the current frame is the point cloud data collected by the laser radar; when the image acquisition device is a camera, the current sensor data corresponding to the current frame is the image data collected by the camera.

[0106] According to the current sensor data in the current frame, an initial detection mark corresponding to the target object can be generated in the current frame. For example, when there are three target objects in the current frame, each target object can correspond to an initial detection mark.

[0107] Furthermore, the current sensor data includes current point cloud data;

[0108] Generating at least one initial detection mark corresponding to the at least one target object in the current frame according to the current sensor data, comprising:

[0109] Clustering the current point cloud data to detect at least one current target object;

[0110] Generate initial detection labels corresponding to each current target object.

[0111] When the current sensor data is the current point cloud data, similar to the reference point cloud data, the current point cloud data can be clustered to obtain corresponding initial clustering results. Specifically, each initial clustering result corresponds to a current target object.

[0112] For example, a current frame may include a plurality of current point cloud data. After clustering the current point cloud data in the current frame, four initial clustering results are obtained, and four current target objects can be determined.

[0113] After the current target object is determined in the current frame, an initial detection mark corresponding to each current target object can be generated. Further, the initial detection mark can be an initial detection box. However, in some embodiments, the initial detection mark can also be in the form of a marking point to mark the target object.

[0114] In one or more embodiments provided by the present disclosure, generating an initial detection mark corresponding to each current target object includes:

[0115] Inputting a current frame including each current target object into an object detection model;

[0116] An initial detection result output by the object detection model is obtained, wherein the initial detection result includes the current frame, each initial detection mark in the current frame, and initial detection mark pose information of each initial detection mark.

[0117] Among them, the object detection model specifically refers to a pre-trained machine learning model for realizing target detection. In practical applications, the object detection model can be an artificial intelligence model based on a convolutional neural network. Specifically, the current frame including each target object is input into the object detection model, and the object detection model can detect the current frame and output the corresponding initial detection result. The initial detection result includes the initial detection mark obtained by marking each current target object on the basis of the current frame, and the initial detection mark pose information of each initial detection mark determined on the basis of the current frame.

[0118] Acquiring the detection information of the current frame allows the detection information of the current frame to be combined with the data predicted based on the historical frames during subsequent processing, so that the target detection in the current frame can be more accurate.

[0119] Step 206: Generate a target detection result in the current frame according to each reference detection mark.

[0120] After obtaining the reference detection mark information, the final target detection result can be generated in the current frame according to the position information of each reference detection mark in the current frame. Specifically, it can be determined whether multiple reference target objects are the same target object according to the intersection and union ratio information between multiple reference detection mark information.

[0121] In practical applications, for the same target object, it will participate in the prediction in different historical frames, and the reference detection mark for the same target object will be marked in the current frame. Therefore, it is necessary to detect the final target detection result in the current frame in combination with each reference detection mark.

[0122] In one or more embodiments provided by the present disclosure, when at least one initial detection mark corresponding to at least one target object is generated in the current frame according to the current sensor data, the above step 206 further includes:

[0123] An object detection result is generated in the current frame according to each initial detection mark and each reference detection mark.

[0124] In order to better detect the target object in the current frame, the reference detection marks and the initial detection marks may be combined to generate the target detection result corresponding to the current frame. Specifically, the target detection result indicates the target object detected in the current frame.

[0125] Specifically, generating a target detection result in the current frame according to each initial detection mark and each reference detection mark may include the following steps A, B, and C:

[0126] Step A: determining a current detection mark from each initial detection mark and each reference detection mark, and determining at least one comparison detection mark corresponding to the current detection mark.

[0127] The current detection mark specifically refers to the detection mark processed in the current frame. In practical applications, in order to determine the final target detection result, each detection mark needs to be processed accordingly in sequence. During the processing, in order to facilitate calculation, a detection mark for processing can be determined first. The detection mark processed in the current frame is the initial detection mark.

[0128] After the current detection mark is determined, at least one comparison detection mark corresponding to the current detection mark is further determined. The comparison detection mark specifically refers to a detection mark having a corresponding relationship with the current detection mark. For example, in practical applications, four target objects may be detected at the same time, namely target object 1, target object 2, target object 3 and target object 4. In the case where the current detection mark is one of the detection marks of target object 1, the corresponding at least one comparison detection mark is also a detection mark corresponding to target object 1.

[0129] In a specific embodiment provided by the present disclosure, determining a current detection mark from each initial detection mark and each reference detection mark, and determining at least one comparison detection mark corresponding to the current detection mark, comprises:

[0130] Generate a set of detection marks to be processed according to each initial detection mark and each reference detection mark;

[0131] Determine a current detection mark from the set of detection marks to be processed;

[0132] Calculating an intersection-and-union ratio between the current detection mark and other detection marks in the set of detection marks to be processed;

[0133] The to-be-processed detection mark whose intersection-over-union ratio is greater than or equal to a first preset threshold is used as a comparison detection mark corresponding to the current detection mark.

[0134] In practical applications, each initial detection mark and each reference detection mark can be firstly added as a detection mark to be processed into a set of detection marks to be processed, and then the current detection mark can be determined from the set of detection marks to be processed in sequence. Furthermore, any detection mark to be processed in the set of detection marks to be processed can be selected as the current detection mark, or the current detection mark can be selected from the detection marks to be processed according to a preset condition. The preset condition can be set according to the actual situation, and is not limited in the present disclosure.

[0135] In a specific embodiment provided by the present disclosure, when marking, each pending detection mark includes a detection mark confidence corresponding to the pending detection mark. The detection mark confidence is used to indicate the credibility of the detection mark. In order to facilitate determining the current detection mark from the pending detection mark set, determining the current detection mark from the pending detection mark set includes:

[0136] The current detection mark is determined according to the detection mark confidence corresponding to each detection mark to be processed.

[0137] Specifically, the detection mark confidences corresponding to the pending detection marks can be sorted, and the pending detection mark with the highest confidence can be selected as the current detection mark. In some embodiments, if there are at least two pending detection marks with the highest confidence, any one of them can be randomly selected as the current detection mark.

[0138] The Intersection over Union (IoU) function can be used to calculate the ratio of the intersection and union of two bounding boxes to evaluate the area overlap of two geometric figures. In the target detection algorithm, the IoU can be used to calculate the error between the predicted box and the real box, or to filter redundant overlapping targets in the non-maximum suppression algorithm. In one or more embodiments provided in the present disclosure, weighted non-maximum suppression is used to merge the detection results of the historical frame and the current frame to supplement and enhance the detection of the current frame. Furthermore, after determining the current detection mark, it is also possible to further determine which detection marks belong to the same target object as the current detection mark based on the IoU ratio between the current detection mark and other detection marks.

[0139] Specifically, after determining the current detection mark, the intersection-and-union ratio of the current detection mark and other detection marks in the pending detection set can be calculated. Based on this, the association relationship between each other detection mark and the current detection mark can be obtained. After the calculation is completed, the pending detection mark whose intersection-and-union ratio is greater than or equal to the first preset threshold is used as the comparison detection mark of the current detection mark. At this point, at least one comparison detection mark corresponding to the current detection mark can be obtained. Among them, the first preset threshold can be a preset threshold for determining the comparison detection mark.

[0140] In one or more embodiments provided in the present disclosure, the method further comprises:

[0141] The detection marks to be processed whose intersection-over-union ratio is less than the first preset threshold and greater than or equal to a second preset threshold are removed from the set of detection marks to be processed, wherein the second preset threshold is less than the first preset threshold.

[0142] In practical applications, there may be some IoU values ​​that are relatively small, less than the second preset threshold. In this case, in the specific implementation provided by the present disclosure, it is generally considered that the detection frame to be processed whose IoU value is less than the second preset threshold belongs to another target object. Therefore, when processing the current detection mark, the detection frame to be processed whose IoU value is less than the second preset threshold may not be considered. The second preset threshold is used to determine the preset threshold that does not belong to the same target object as the current detection mark.

[0143] In this case, there may be some pending detection frames whose intersection-over-union ratio with the current detection mark is greater than or equal to the second preset threshold, but less than the first preset threshold. Such pending detection frames cannot be determined to belong to other target objects, nor can they be used for subsequent calculation of the target object corresponding to the current detection mark. Therefore, in the embodiment provided by the present disclosure, the pending detection marks whose intersection-over-union ratio is less than the first preset threshold and greater than or equal to the second preset threshold can be deleted from the pending detection mark set, thereby reducing the amount of calculation for subsequent calculations and improving the calculation efficiency of the target detection algorithm provided by the present disclosure.

[0144] Step B: determining a target detection mark corresponding to the current detection mark according to the current detection mark and each comparison detection mark.

[0145] After the current detection mark and each comparison detection mark are determined, the target detection mark corresponding to the current detection mark can be calculated by the current detection mark and each comparison detection mark. The current detection mark can be a reference detection mark or an initial detection mark.

[0146] The target detection mark corresponding to the current detection mark is calculated through the current detection mark and each comparison detection mark. The initial detection mark in the current frame can be referred to, and the reference detection mark in the historical frame can also be referred to, so that the final target detection can be more accurate.

[0147] In a specific embodiment provided by the present disclosure, determining a target detection mark corresponding to the current detection mark according to the current detection mark and each comparison detection mark includes:

[0148] Obtaining the current detection mark position information of the current detection mark, and obtaining the comparison detection mark position information and the comparison frame weight of each comparison detection mark;

[0149] Determine the target detection mark pose information according to the current detection mark pose information, the comparison detection mark pose information of each comparison detection mark and the comparison frame weight;

[0150] Determine a target detection mark corresponding to the current detection mark according to the target detection mark pose information.

[0151] In the disclosed embodiment, each detection mark corresponds to a set of detection mark posture information. Specifically, the detection mark posture information may include the center point coordinates (x, y, z) of the detection mark, the length, width, height, angle, etc. of the detection mark. Before subsequent calculations, the current detection mark posture information b corresponding to the current detection mark is first obtained, and the comparison detection mark posture information bk corresponding to each comparison detection mark is obtained at the same time.

[0152] The target detection method provided in the present disclosure introduces the marking information in the reference historical frame. However, the historical frame is a frame that has been detected, and it has a certain but limited reference value for the prediction of the current frame. For example, in a time series, the credibility of a historical frame that is closer to the current frame is higher than the credibility of a historical frame that is farther away from the current frame. Based on this, in the target detection method provided in the present disclosure, a confidence weight can be set in advance for each historical frame. The farther the historical frame is from the current frame, the lower its confidence weight. Each reference detection mark detected in the historical frame is set with a detection mark confidence. According to the confidence weight of the historical frame and the detection mark confidence of the reference detection mark, the comparison box weight of each reference detection mark in the current frame can be determined. Specifically, for the confidence weight corresponding to a reference detection mark in the i-th historical frame, refer to the following formula (6):

[0153] w=s·d i Formula (6)

[0154] Where w represents the confidence weight of the i-th historical frame, s is the detection mark confidence of the reference detection mark, and d is the preset weight decay factor. For the i-th historical frame, the corresponding weight decay factor is d i By using the above formula (6), we can calculate the weight of the comparison box corresponding to the comparison detection mark k as w k .

[0155] After determining the current detection mark pose information and the comparison detection mark pose information and the comparison frame weight of each comparison detection mark, the target detection mark pose information of the target detection mark can be calculated. For details, refer to the following formula (7).

[0156] Formula (7)

[0157] Among them, b f represents the target detection mark pose information, b represents the current detection mark pose information, b k represents the position information of the comparison detection mark k, w k Represents the weight of the comparison box for the comparison detection marker k.

[0158] In the method provided by the present disclosure, the current detection mark pose information of the current detection frame and the comparison detection frame mark pose information of each comparison detection frame are averaged. For reference detection marks predicted only from historical frames, their scores will be reduced to avoid affecting the original detection effect of the current frame.

[0159] At this point, the target detection mark pose information corresponding to the current detection mark can be obtained.

[0160] Step C: Eliminate the current detection mark and each comparison detection mark from each initial detection mark and each reference detection mark, and continue to perform the operation of determining the current detection mark from each initial detection mark and each reference detection mark, and determining at least one comparison detection mark corresponding to the current detection mark.

[0161] After determining the target detection mark pose information, it is also necessary to delete the current detection mark and each comparison detection mark used to calculate the target detection mark pose information, so as to avoid these detection marks that have participated in the calculation from repeatedly participating in subsequent calculations, thereby improving the detection accuracy and computational efficiency of the target detection method adopted by the present invention.

[0162] Specifically, removing the current detection mark and each comparison detection mark from each initial detection mark and each reference detection mark includes: removing the current detection mark and each comparison detection mark from the set of detection marks to be processed.

[0163] In the above steps, each initial detection mark and each reference detection mark have been combined into a set of detection marks to be processed. Therefore, the current detection mark and each comparison detection mark corresponding to the current detection mark can be directly removed from the set of detection marks to be processed. This avoids recalculating the detection marks that have already participated in the calculation, saving computing resources.

[0164] After the current detection mark and each comparison detection mark are removed, the next current detection mark and the comparison detection mark corresponding to the current detection mark for calculation will be selected from the set of pending detection marks. The specific selection method is referred to the above related steps and will not be repeated here.

[0165] The method provided by the present disclosure repeats the above steps until the detection mark in the set of detection marks to be processed is empty, that is, until all detection frames have been weighted fused. At this point, multiple target detection marks confirmed after multiple operations can be obtained. Then, the target detection result in the current frame can be obtained.

[0166] The target detection method provided by the present disclosure refers to the historical detection results in the historical frames during the detection process of the target in the current frame. By predicting the historical detection results of the historical frames, the reference detection marks of the target objects in each historical frame in the current frame are predicted. Finally, the final target detection result is generated in the current frame according to the reference detection mark. The target detection method provided by the present disclosure improves the problems of false detection and missed detection caused by defects in the point cloud itself in the process of current target detection results, and further enhances the detection effect. At the same time, the target detection method provided by the present disclosure can be directly integrated with the existing detector without the participation of the neural network model of machine learning, and thus can be directly integrated with the existing detector without the need for additional model parameters, and will not bring about high processing delays, so that it can be applied to more application scenarios.

[0167] The following combination Figure 4 and Figure 5 , taking the application of the target detection method provided by the present disclosure in a roadside detection device as an example, the target detection method is further described. Figure 4 A processing flow chart of a method for detecting a landmark applied to a roadside detection device provided by an embodiment of the present disclosure is shown. Figure 5 A schematic diagram of a target detection method provided by an embodiment of the present disclosure is shown. The target detection method specifically comprises the following steps:

[0168] Step 402: Acquire current point cloud data corresponding to the current frame, and acquire reference point cloud data corresponding to each of a plurality of historical frames, wherein, in a time series, the plurality of historical frames are before the current frame.

[0169] In this embodiment, see Figure 5, determine the current frame T, and the historical frames T-1, T-2, T-3, and T-4 corresponding to the current frame.

[0170] Step 404: clustering the reference point cloud data in each historical frame to obtain at least one reference target object.

[0171] In this embodiment, the reference point cloud data in each historical frame is clustered to obtain the corresponding reference target object in each historical frame. Figure 5 , the reference detection box in each historical frame.

[0172] Step 406: predicting reference pose information of each reference target object in the current frame, and determining a reference detection mark corresponding to each reference target object in the current frame according to each reference pose information.

[0173] In this embodiment, the predicted detection frame of each reference target object in the current frame is predicted based on the pose information of each reference target object in each historical frame. Figure 5 The dotted part of the predicted detection box.

[0174] Step 408: clustering the current point cloud data to detect at least one current target object, and generating an initial detection mark corresponding to each current target object.

[0175] In this embodiment, after clustering the current point cloud data of the current frame T, two current target objects are detected. Figure 5 In the current frame T, an initial detection frame 1' and an initial detection frame 2' corresponding to each current target object are generated in the current frame T.

[0176] Step 410: Generate a set of detection marks to be processed according to each initial detection mark and each reference detection mark.

[0177] In this implementation, each reference detection frame and the initial detection frame may be used to generate a set of detection frames to be processed, so as to be used for subsequent screening and calculation of the detection frames.

[0178] Step 412: Determine a current detection mark from the set of detection marks to be processed, and calculate an intersection-over-union ratio between the current detection mark and other detection marks in the set of detection marks to be processed.

[0179] In this embodiment, the current detection frame is determined according to the detection frame confidence of each detection frame to be processed, and the intersection-over-union ratio between the current detection frame and other detection frames is calculated.

[0180] Step 414: taking the pending detection mark whose intersection-over-union ratio is greater than or equal to the first preset threshold as the comparison detection mark corresponding to the current detection mark.

[0181] In this embodiment, the detection frame to be processed whose intersection-over-union ratio is greater than a first preset threshold is used as a comparison detection frame corresponding to the current detection frame.

[0182] Step 416: remove the pending detection marks whose intersection-over-union ratio is less than the first preset threshold and greater than or equal to a second preset threshold from the pending detection mark set, wherein the second preset threshold is less than the first preset threshold.

[0183] In this embodiment, the detection frames to be processed whose intersection-over-union ratio is less than the first preset threshold and greater than or equal to the second preset threshold are deleted from the set of detection frames to be processed. These deleted detection frames to be processed do not belong to other target objects. Since they cannot be used for the calculation of subsequent target detection results, they need to be eliminated.

[0184] Step 418: Determine a target detection mark corresponding to the current detection mark according to the current detection mark and each comparison detection mark.

[0185] In this embodiment, a weighted fusion calculation may be performed based on the current detection frame and the comparison detection frame corresponding to the current detection frame, thereby generating a target detection mark corresponding to the current detection frame.

[0186] Step 420: remove the current detection mark and each comparison detection mark from the set of detection marks to be processed, and continue to execute step 412 until the detection mark in the set of detection marks to be processed is empty.

[0187] In this embodiment, after determining the target detection mark corresponding to the current detection frame of the current time, the current detection frame and the comparison detection frame can be deleted from the set of detection frames to be processed, and it is determined whether there are still detection frames in the set of detection frames to be processed. If yes, continue to return to step 412, if not, execute the subsequent step 422.

[0188] Step 422: Generate the target detection result of the current frame according to each target detection mark.

[0189] In this embodiment, the target detection result corresponding to the current frame is generated according to each target detection mark generated in the above steps.

[0190] In this embodiment, see Figure 5, by performing target detection on the current frame T, the detection frame 1' and the detection frame 2' in the current frame T are obtained, and multiple historical frames corresponding to the current frame T are obtained. Each historical frame includes historical frame detection data. According to the prediction of the historical detection data in each historical frame, the predicted position of a reference target object in the historical frame in the current frame can be predicted, that is, the predicted detection frame of each reference target object in the current frame can be predicted. Then, by fusing the predicted detection frames and the detection frame of the current frame, the enhanced target detection result for the current frame T, namely, detection frame 1, detection frame 2 and detection frame 3, can be generated.

[0191] The target detection method provided by the embodiment of the present disclosure avoids the situation of false detection and missed detection due to occlusion or detection error in the process of detecting the current frame T. For example, due to occlusion and other issues in the current frame T, the detection frame 3 cannot be detected when only the current frame T is detected. However, by predicting the historical frames, its position in the current frame T can be predicted, thereby obtaining a better target detection result.

[0192] Figure 6 is a flow chart of another target detection method provided by the present disclosure, which can be applied to autonomous driving scenarios, specifically including:

[0193] Step 602: Acquire vehicle historical sensor data corresponding to each of a plurality of historical frames, wherein, in time series, the plurality of historical frames are before the current frame.

[0194] Step 604: Generate a reference detection mark corresponding to at least one detected vehicle in the current frame according to the vehicle historical sensor data of each historical frame.

[0195] Step 606: Generate a target vehicle detection result in the current frame according to each reference detection mark.

[0196] In a specific embodiment provided by the present disclosure, the method further comprises:

[0197] Acquire current sensor data of the vehicle corresponding to the current frame;

[0198] generating at least one initial detection mark corresponding to the at least one detected vehicle in the current frame based on the vehicle current sensor data;

[0199] Wherein, generating a target vehicle detection result in the current frame according to each reference detection mark includes:

[0200] A target vehicle detection result is generated in the current frame according to each initial detection mark and each reference detection mark.

[0201] In a specific implementation provided by the present disclosure, the vehicle current sensor data includes current point cloud data;

[0202] Generating at least one initial detection mark corresponding to the at least one detected vehicle in the current frame according to the current sensor data of the vehicle, comprising:

[0203] Clustering the current point cloud data to detect at least one current detection vehicle;

[0204] An initial detection mark corresponding to each currently detected vehicle is generated.

[0205] In a specific embodiment provided by the present disclosure, generating an initial detection mark corresponding to each currently detected vehicle includes:

[0206] Inputting a current frame including each currently detected vehicle into a vehicle detection model;

[0207] An initial detection result output by the vehicle detection model is obtained, wherein the initial detection result includes the current frame, each initial detection mark in the current frame, and initial detection mark pose information of each initial detection mark.

[0208] In a specific implementation provided by the present disclosure, generating a target vehicle detection result in the current frame according to each initial detection mark and each reference detection mark includes:

[0209] Determine a current detection mark from each initial detection mark and each reference detection mark, and determine at least one comparison detection mark corresponding to the current detection mark;

[0210] Determine a target vehicle detection result corresponding to the current detection mark according to the current detection mark and each comparison detection mark;

[0211] The current detection mark and the comparison detection marks are removed from the initial detection marks and the reference detection marks, and the operation of determining the current detection mark from the initial detection marks and the reference detection marks and determining at least one comparison detection mark corresponding to the current detection mark is continued.

[0212] In a specific embodiment provided by the present disclosure, determining a current detection mark from each initial detection mark and each reference detection mark, and determining at least one comparison detection mark corresponding to the current detection mark, comprises:

[0213] Generate a set of detection marks to be processed according to each initial detection mark and each reference detection mark;

[0214] Determine a current detection mark from the set of detection marks to be processed;

[0215] Calculating an intersection-and-union ratio between the current detection mark and other detection marks in the set of detection marks to be processed;

[0216] The to-be-processed detection mark whose intersection-over-union ratio is greater than or equal to a first preset threshold is used as a comparison detection mark corresponding to the current detection mark.

[0217] In a specific embodiment provided by the present disclosure, the method further comprises:

[0218] The detection marks to be processed whose intersection-over-union ratio is less than the first preset threshold and greater than or equal to a second preset threshold are removed from the set of detection marks to be processed, wherein the second preset threshold is less than the first preset threshold.

[0219] In a specific embodiment provided by the present disclosure, the detection mark to be processed includes the detection mark confidence;

[0220] Determining a current detection mark from the set of detection marks to be processed includes:

[0221] The current detection mark is determined according to the detection mark confidence corresponding to each detection mark to be processed.

[0222] In a specific embodiment provided by the present disclosure, removing the current detection mark and each comparison detection mark from each initial detection mark and each reference detection mark includes:

[0223] The current detection mark and each comparison detection mark are removed from the set of detection marks to be processed.

[0224] In a specific embodiment provided by the present disclosure, determining a target vehicle detection result corresponding to the current detection mark according to the current detection mark and each comparison detection mark includes:

[0225] Obtaining the current detection mark position information of the current detection mark, and obtaining the comparison detection mark position information and the comparison frame weight of each comparison detection mark;

[0226] Determine the target detection mark pose information according to the current detection mark pose information, the comparison detection mark pose information of each comparison detection mark and the comparison frame weight;

[0227] Determine a target vehicle detection result corresponding to the current detection mark according to the target detection mark posture information.

[0228] In a specific embodiment provided by the present disclosure, the historical sensor data includes reference point cloud data;

[0229] Generating a reference detection mark corresponding to at least one detected vehicle in the current frame according to the historical sensor data of each historical frame, comprising:

[0230] For each history frame:

[0231] Clustering the reference point cloud data in the historical frame to obtain at least one reference target vehicle;

[0232] Predicting reference pose information of each reference target vehicle in the current frame;

[0233] A reference detection mark corresponding to each reference target vehicle in the current frame is determined according to each reference pose information.

[0234] In a specific implementation provided by the present disclosure, predicting the reference pose information of each reference target vehicle in the current frame includes:

[0235] For each reference target vehicle:

[0236] Acquire initial reference pose information of the reference target vehicle in the historical frame;

[0237] The initial reference pose information is input into a motion prediction model to obtain reference pose information of the reference target vehicle in the current frame.

[0238] In a specific embodiment provided by the present disclosure, the method further comprises:

[0239] Determining vehicle attribute information of the reference target vehicle;

[0240] A motion prediction model corresponding to the reference target vehicle is determined according to the vehicle attribute information.

[0241] In a specific implementation provided by the present disclosure, inputting the initial reference pose information into a motion prediction model includes:

[0242] Acquiring motion parameter information of the reference target object;

[0243] The initial reference pose information and the motion parameter information are input into a motion prediction model.

[0244] The target detection method provided by the present disclosure is applied to the autonomous driving scenario, and in the process of detecting the vehicle in the current frame, the historical detection results in the historical frame are referenced. By predicting the historical detection results of the historical frames, the reference detection marks of the target vehicles in each historical frame in the current frame are predicted. Finally, the final target detection result is generated in the current frame according to the reference detection mark. The target detection method provided by the present disclosure improves the problems of false detection and missed detection caused by defects in the point cloud itself in the process of the current target detection result, and further enhances the detection effect. At the same time, the target detection method provided by the present disclosure can be directly integrated with the existing detector without the participation of the neural network model of machine learning, and therefore can be directly integrated with the existing detector without the need for additional model parameters, and will not bring high processing delays, so that it can be applied to more application scenarios.

[0245] Figure 7 The block diagram of a computing device 700 according to an embodiment of the present disclosure is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0246] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0247] In one embodiment of the present disclosure, the above components of the computing device 700 and Figure 7Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present disclosure. Those skilled in the art may add or replace other components as needed.

[0248] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 may also be a mobile or stationary server.

[0249] Among them, when the processor 720 executes the computer instructions, the steps of the target detection method are implemented.

[0250] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the target detection method described above are of the same concept, and the details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the target detection method described above.

[0251] An embodiment of the present disclosure further provides a vehicle, which includes a processor, and the processor implements the steps of the target detection method when executing the computer instructions.

[0252] The above is a schematic scheme of a vehicle of this embodiment. It should be noted that the technical scheme of the vehicle and the technical scheme of the above target detection method belong to the same concept, and the details of the technical scheme of the vehicle that are not described in detail can all be referred to the description of the technical scheme of the above target detection method.

[0253] An embodiment of the present disclosure further provides a computer-readable storage medium storing computer instructions, which implement the steps of the target detection method as described above when executed by a processor.

[0254] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the target detection method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the target detection method described above.

[0255] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0256] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0257] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the application are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0258] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0259] The preferred embodiments of the present disclosure disclosed above are only used to help explain the present disclosure. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present disclosure. The present disclosure selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present disclosure, so that those skilled in the art can well understand and use the present disclosure. The present disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A target detection method, characterized in that: include: Acquire historical sensor data corresponding to each of a plurality of historical frames, wherein the plurality of historical frames are before the current frame in time series; generating a reference detection marker corresponding to at least one target object in the current frame based on the historical sensor data of each historical frame; An object detection result is generated in the current frame according to each reference detection mark.

2. The method according to claim 1, characterized in that The method further comprises: Acquire current sensor data corresponding to the current frame; generating, in the current frame, at least one initial detection marker corresponding to the at least one target object based on the current sensor data; The step of generating a target detection result in the current frame according to each reference detection mark includes: An object detection result is generated in the current frame according to each initial detection mark and each reference detection mark.

3. The method according to claim 2, characterized in that The current sensor data includes current point cloud data; Generating at least one initial detection mark corresponding to the at least one target object in the current frame according to the current sensor data, comprising: Clustering the current point cloud data to detect at least one current target object; Generate an initial detection label corresponding to each current target object.

4. The method according to claim 3, characterized in that Generate an initial detection mark corresponding to each current target object, including: Inputting a current frame including each current target object into an object detection model; An initial detection result output by the object detection model is obtained, wherein the initial detection result includes the current frame, each initial detection mark in the current frame, and initial detection mark pose information of each initial detection mark.

5. The method according to claim 2, characterized in that Generating a target detection result in the current frame according to each initial detection mark and each reference detection mark, including: Determine a current detection mark from each initial detection mark and each reference detection mark, and determine at least one comparison detection mark corresponding to the current detection mark; Determine a target detection mark corresponding to the current detection mark according to the current detection mark and each comparison detection mark; The current detection mark and the comparison detection marks are removed from the initial detection marks and the reference detection marks, and the operation of determining the current detection mark from the initial detection marks and the reference detection marks and determining at least one comparison detection mark corresponding to the current detection mark is continued.

6. The method according to claim 5, characterized in that Determining a current detection mark from each initial detection mark and each reference detection mark, and determining at least one comparison detection mark corresponding to the current detection mark, comprising: Generate a set of detection marks to be processed according to each initial detection mark and each reference detection mark; Determine a current detection mark from the set of detection marks to be processed; Calculating an intersection-and-union ratio between the current detection mark and other detection marks in the set of detection marks to be processed; The to-be-processed detection mark whose intersection-over-union ratio is greater than or equal to a first preset threshold is used as a comparison detection mark corresponding to the current detection mark.

7. The method according to claim 6, characterized in that The method further comprises: The detection marks to be processed whose intersection-over-union ratio is less than the first preset threshold and greater than or equal to a second preset threshold are removed from the set of detection marks to be processed, wherein the second preset threshold is less than the first preset threshold.

8. The method according to claim 6, characterized in that The detection mark to be processed includes the detection mark confidence; Determining a current detection mark from the set of detection marks to be processed includes: The current detection mark is determined according to the detection mark confidence corresponding to each detection mark to be processed.

9. The method according to claim 6, characterized in that Eliminating the current detection mark and each comparison detection mark from each initial detection mark and each reference detection mark includes: The current detection mark and each comparison detection mark are removed from the set of detection marks to be processed.

10. The method according to claim 5, characterized in that Determining a target detection mark corresponding to the current detection mark according to the current detection mark and each comparison detection mark includes: Obtaining the current detection mark position information of the current detection mark, and obtaining the comparison detection mark position information and the comparison frame weight of each comparison detection mark; Determine the target detection mark pose information according to the current detection mark pose information, the comparison detection mark pose information of each comparison detection mark and the comparison frame weight; Determine a target detection mark corresponding to the current detection mark according to the target detection mark pose information.

11. The method according to claim 1, characterized in that The historical sensor data includes reference point cloud data; Generating a reference detection mark corresponding to at least one target object in the current frame according to the historical sensor data of each historical frame, comprising: For each history frame: Clustering the reference point cloud data in the historical frame to obtain at least one reference target object; Predicting reference pose information of each reference target object in the current frame; A reference detection mark corresponding to each reference target object in the current frame is determined according to each reference pose information.

12. The method according to claim 11, characterized in that Predicting reference pose information of each reference target object in the current frame includes: For each reference target object: Acquire initial reference pose information of the reference target object in the historical frame; The initial reference pose information is input into a motion prediction model to obtain reference pose information of the reference target object in the current frame.

13. The method according to claim 12, characterized in that The method further comprises: Determining object attribute information of the reference target object; A motion prediction model corresponding to the reference target object is determined according to the object attribute information.

14. The method according to claim 12, characterized in that Inputting the initial reference pose information into a motion prediction model includes: Acquiring motion parameter information of the reference target object; The initial reference pose information and the motion parameter information are input into a motion prediction model.

15. A target detection method, characterized in that: include: Acquire vehicle historical sensor data corresponding to each of a plurality of historical frames, wherein the plurality of historical frames are before the current frame in time sequence; generating a reference detection mark corresponding to at least one detected vehicle in the current frame based on the vehicle historical sensor data of each historical frame; A target vehicle detection result is generated in the current frame according to each reference detection mark.

16. A vehicle comprising a processor, characterized in that When the processor executes the computer instructions, the steps of the method described in any one of claims 1-14 or 15 are implemented.

17. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer instructions, the steps of the method described in any one of claims 1-14 or 15 are implemented.

18. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1-14 or 15 are implemented.

Images