Target detection method and device, computer equipment and storage medium

Abstract

The invention relates to a target detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring laser radar point cloud data and to-be-processed image data; performing identification operation on the to-be-processed image data based on a preset detection model to obtain image surrounding information; analyzing and processing the laser radar point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple pieces of space surrounding information; and performing matching detection based on the image surrounding information, the clustered point cloud data and the plurality of spatial surrounding frames to obtain a target detection list. By adopting the method, surrounding obstacles can be effectively identified and classified, the environmental perception capability of the automatic driving system is improved, the safety and reliability are improved, and higher detection precision is realized by combining 2D visual detection and 3D point cloud clustering technologies.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a target detection method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, the field of artificial intelligence has ushered in an unprecedented prosperity. In recent years, target detection methods based on advanced technologies such as neural networks and transformers have emerged in an endless stream, showing amazing potential and efficiency. These methods can identify and locate specific objects in images, whether they are static pictures or dynamic videos, and can achieve fast and accurate target detection. This progress in AI technology has not only greatly promoted technological innovation in related industries, but also brought convenience to daily life. For example, in the fields of autonomous driving, medical image analysis, security monitoring, etc., the application of target detection technology is becoming more and more extensive.

[0003] In related technologies, 3D target detection algorithms based on vision and point cloud are generally used. Traditional 2D image data is used to detect targets based on images acquired by cameras. Commonly used methods include convolutional neural networks (CNNs) and region proposal networks (RPNs). Combined with the three-dimensional spatial data acquired by 3D sensors (such as lidar and depth cameras), each point in the point cloud contains its three-dimensional coordinates (x, y, z). Point cloud data provides spatial structural information of objects.

[0004] However, among the above-mentioned related technologies, the actual detection effect of the 3D target detection algorithm based on vision and point cloud is strongly correlated with the data set. When the trained model is applied to a completely different data set or environment, the model performance often decreases significantly. Summary of the invention

[0005] Based on this, it is necessary to provide a target detection method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of obstacle detection by utilizing a general visual detection model combined with the spatial position information of surrounding obstacles to address the above-mentioned technical problems.

[0006] In a first aspect, the present application provides a target detection method. The method comprises:

[0007] Obtain laser radar point cloud data and image data to be processed;

[0008] Perform recognition operations on the image data to be processed based on a preset detection model to obtain image surrounding information;

[0009] Analyze and process the LiDAR point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial encirclement information;

[0010] Matching detection is performed based on image encirclement information, clustered point cloud data and multiple spatial bounding boxes to obtain a target detection list.

[0011] In one embodiment, the laser radar point cloud data is analyzed and processed based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information, including:

[0012] Preprocess the laser radar point cloud data to obtain preprocessed point cloud data;

[0013] Based on the Euclidean point cloud clustering algorithm, the pre-processed point cloud data is processed to obtain multiple clustered point cloud data;

[0014] Perform extreme value screening in three dimensions on each clustered point cloud data to obtain multiple clustered point cloud extreme values ​​in three dimensions;

[0015] Generate multiple spatial enclosing information based on multiple clustered point cloud extreme values ​​in three dimensions.

[0016] In one embodiment, matching detection is performed based on image encirclement information, clustered point cloud data, and multiple spatial bounding boxes to obtain a target detection list, including:

[0017] Performing dimension transformation on a plurality of clustered point cloud data based on preset internal and external parameters of the sensor to obtain a plurality of transformed point cloud data;

[0018] Generate transformed enclosing information based on a plurality of transformed point cloud data;

[0019] Based on the preset evaluation algorithm, the converted encirclement information and the image encirclement information are matched and detected to obtain a target detection list.

[0020] In one embodiment, performing matching detection on the converted enclosing information and the image enclosing information based on a preset evaluation algorithm to obtain a target detection list includes:

[0021] Based on the intersection-and-union algorithm, matching calculation is performed on the converted encirclement information and the image encirclement information to obtain a matching calculation result;

[0022] Compare the matching calculation result with the preset matching threshold to obtain a comparison result;

[0023] Based on the comparison result, the image category information is matched with the spatial information to obtain a matching detection list;

[0024] Overlapping is performed on the matching detection list to obtain the target detection list.

[0025] In one embodiment, overlapping screening is performed on the matching detection list to obtain a target detection list, including:

[0026] Performing overlapping calculation on the converted enclosing information corresponding to one of the spatial information in the matching detection list and the other converted enclosing information to obtain an overlapping calculation result;

[0027] Compare the overlap calculation result with a preset overlap threshold to obtain an overlap comparison result;

[0028] The matching detection list is screened based on the overlapping comparison results to obtain a target detection list.

[0029] In one embodiment, before acquiring the laser radar point cloud data and the image data to be processed, the method further includes:

[0030] Get dataset label information;

[0031] Perform data processing on the dataset label information according to the pre-built detection model to obtain the image-text matching results;

[0032] The pre-built detection model is adjusted according to the image-text matching result, feedback data and a preset feedback function to obtain a target detection model and set the target detection model as a preset detection model.

[0033] In one embodiment, before obtaining the data set label information, the method further includes:

[0034] Get dataset label information;

[0035] Perform data processing on the dataset label information according to the pre-built detection model to obtain the image-text matching results;

[0036] The pre-built detection model is adjusted according to the image-text matching result, feedback data and a preset feedback function to obtain a target detection model and set the target detection model as a preset detection model.

[0037] In a second aspect, the present application also provides a target detection device. The device comprises:

[0038] A data acquisition module, used to acquire laser radar point cloud data and image data to be processed;

[0039] An image processing module is used to perform recognition operations on the image data to be processed based on a preset detection model to obtain image surrounding information;

[0040] The point cloud processing module is used to analyze and process the laser radar point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information;

[0041] The matching detection module is used to perform matching detection based on image encirclement information, clustered point cloud data and multiple spatial bounding boxes to obtain a target detection column.

[0042] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0043] Obtain laser radar point cloud data and image data to be processed;

[0044] Perform recognition operations on the image data to be processed based on a preset detection model to obtain image surrounding information;

[0045] Analyze and process the LiDAR point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial encirclement information;

[0046] Matching detection is performed based on image encirclement information, clustered point cloud data and multiple spatial bounding boxes to obtain a target detection list.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0048] Obtain laser radar point cloud data and image data to be processed;

[0049] Perform recognition operations on the image data to be processed based on a preset detection model to obtain image surrounding information;

[0050] Analyze and process the LiDAR point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial encirclement information;

[0051] Matching detection is performed based on image encirclement information, clustered point cloud data and multiple spatial bounding boxes to obtain a target detection list.

[0052] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0053] Obtain laser radar point cloud data and image data to be processed;

[0054] Perform recognition operations on the image data to be processed based on a preset detection model to obtain image surrounding information;

[0055] Analyze and process the LiDAR point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial encirclement information;

[0056] Matching detection is performed based on image encirclement information, clustered point cloud data and multiple spatial bounding boxes to obtain a target detection list.

[0057] The above-mentioned target detection method, device, computer equipment, storage medium and computer program product use the laser radar and camera of the vehicle to collect the laser radar point cloud data and the image data to be processed around the vehicle, use the preset detection model to process the image data to be processed to obtain a 2D detection bounding box, i.e., image encirclement information, and cluster the laser radar point cloud data and determine multiple 3D encirclement boxes, i.e., spatial encirclement information, based on the clustered point cloud data, and then transform the multiple spatial encirclement information and match the transformed encirclement information with the image encirclement information to determine the actual recognition, specifically, matching the type information of the 2D detection encirclement box with the 3D encirclement box, and then removing the overlap of the matched results to obtain the target detection list, and using the point cloud data obtained by the laser radar, the vehicle can perceive the three-dimensional information of the surrounding environment in real time. It can effectively identify and classify surrounding obstacles, improve the environmental perception ability of the automatic driving system, thereby improving safety and reliability, and combine 2D visual detection and 3D point cloud clustering technology to achieve higher detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] FIG1 is a schematic diagram of a process flow of a target detection method in one embodiment;

[0059] FIG2 is a schematic diagram of a flow chart of a matching detection step in one embodiment;

[0060] FIG3 is a block diagram of a target detection device in one embodiment;

[0061] FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] In one embodiment, as shown in FIG1 , a target detection method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0064] Step 102, obtaining laser radar point cloud data and image data to be processed.

[0065] For example, during the driving of the vehicle, the laser radar and camera installed on the vehicle respectively collect the surrounding laser radar point cloud data and images, i.e., image data to be processed.

[0066] Step 104: performing a recognition operation on the image data to be processed based on a preset detection model to obtain image surrounding information.

[0067] Exemplarily, after acquiring the image data to be processed, the image data to be processed is input into a preset detection model, and image features and text features of obstacle targets in the image are extracted and recognized based on the extracted features to obtain the alignment of the image and text, that is, a 2D detection bounding box.

[0068] Step 106, analyzing and processing the laser radar point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information.

[0069] Among them, the preset point cloud clustering algorithm is the Euclidean point cloud clustering algorithm.

[0070] Exemplarily, the laser radar point cloud data is preprocessed, such as noise reduction, outlier removal, downsampling and other processing operations. Then the preprocessed point cloud data is processed using the Euclidean point cloud clustering algorithm to obtain a plurality of clustered point cloud data. After that, based on the three-dimensional coordinate system, each clustered point cloud data is screened for extreme values ​​in three dimensions, and a 3D bounding box, i.e., spatial enclosing information, is generated for each clustered point cloud data according to the screened extreme values. The spatial enclosing information includes 3D spatial data information but does not have category information.

[0071] Step 108 , performing matching detection based on the image enclosing information, the clustered point cloud data and multiple spatial enclosing boxes to obtain a target detection list.

[0072] Exemplarily, after obtaining multiple spatial enclosing information, i.e., 3D enclosing frames, they are processed to be projected in the image, and the spatial enclosing information is mainly processed using the internal and external parameters of the sensor. Then, the projected enclosing frame is matched and judged with the image enclosing information, i.e., the 2D detection enclosing frame, and the spatial enclosing information corresponding to the projected enclosing frame that is similar to the 2D detection enclosing frame is matched with the category information of the image enclosing information, and a list form, i.e., the target detection list, is formed.

[0073] In the above target detection method, the laser radar and camera of the vehicle are used to collect the laser radar point cloud data and the image data to be processed around the vehicle, and the image data to be processed is processed by a preset detection model to obtain a 2D detection bounding box, i.e., image encirclement information. At the same time, the laser radar point cloud data is clustered and multiple 3D bounding boxes, i.e., spatial encirclement information, are determined based on the clustered point cloud data. Then, the multiple spatial encirclement information is transformed and the transformed encirclement information is matched with the image encirclement information to determine the actual recognition. Specifically, the type information of the 2D detection bounding box is matched with the 3D bounding box, and then the overlap of the matched results is removed to obtain the target detection list. Using the point cloud data obtained by the laser radar, the vehicle can perceive the three-dimensional information of the surrounding environment in real time. It can effectively identify and classify surrounding obstacles, improve the environmental perception ability of the automatic driving system, thereby improving safety and reliability, and combine 2D visual detection and 3D point cloud clustering technology to achieve higher detection accuracy.

[0074] In one embodiment, the laser radar point cloud data is analyzed and processed based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information, including:

[0075] Preprocess the laser radar point cloud data to obtain preprocessed point cloud data; perform Euclidean point cloud clustering algorithm on the preprocessed point cloud data to obtain multiple clustered point cloud data; perform extreme value screening in three dimensions on each clustered point cloud data to obtain multiple clustered point cloud extreme values ​​in three dimensions; generate multiple spatial enclosing information based on multiple clustered point cloud extreme values ​​in three dimensions.

[0076] Among them, the preset point cloud clustering algorithm is the Euclidean point cloud clustering algorithm.

[0077] Exemplarily, after obtaining the laser radar point cloud data, data preprocessing is performed on it, that is, filtering, noise reduction, outlier removal and downsampling data operations are performed. Then the preprocessed point cloud data is input into a preset point cloud clustering algorithm, namely the Euclidean point cloud clustering algorithm, for processing and generating clustering results. After clustering, point cloud clusters of different categories are obtained, and a unique label is assigned to each cluster.

[0078] Then, for each cluster, the extreme values ​​of the three dimensions corresponding to the coordinate system in the three-dimensional space are calculated, and the calculated three-dimensional extreme values ​​are used to construct a 3D bounding box in the three-dimensional space, that is, spatial enclosing information, which includes 3D spatial data information but does not have category information.

[0079] In one embodiment, matching detection is performed based on image encirclement information, clustered point cloud data, and multiple spatial bounding boxes to obtain a target detection list, including:

[0080] Based on the preset internal and external parameters of the sensor, multiple clustered point cloud data are dimensionally transformed to obtain multiple transformed point cloud data; based on the multiple transformed point cloud data, transformed encirclement information is generated; based on the preset evaluation algorithm, the transformed encirclement information and the image encirclement information are matched and detected to obtain a target detection list.

[0081] Exemplarily, the intrinsic and extrinsic parameters of the laser radar are used to correctly project the 3D point cloud onto the 2D image plane. Specifically, each point of the spatial encirclement information is converted into homogeneous coordinates, and then the homogeneous left side is converted into the coordinate system corresponding to the image using the extrinsic parameters (i.e., the rotation matrix and the translation vector). After that, the points in the coordinate system corresponding to the image are projected onto the image plane using the intrinsic parameters (i.e., the focal length, the principal point and other parameters), and the encirclement information is obtained.

[0082] After that, the converted enclosing information is overlapped with the image enclosing information, and then the spatial enclosing information corresponding to the enclosing information whose overlapping area exceeds a certain threshold is matched with the category information of the image enclosing information to obtain a target matching list.

[0083] In one embodiment, as shown in FIG2 , matching detection is performed on the converted enclosing information and the image enclosing information based on a preset evaluation algorithm to obtain a target detection list, including:

[0084] Step 202 : performing a matching calculation on the converted enclosing information and the image enclosing information based on an intersection-over-union algorithm to obtain a matching calculation result.

[0085] The preset evaluation algorithm is the intersection-over-union algorithm, and the converted enclosing information is the projected enclosing box of the enclosing box in the 3D space in the image.

[0086] Exemplarily, the intersection-and-union algorithm is used to match the projection bounding box of the 3D space bounding box in the image with the 2D image bounding box, that is, the overlapping part of the projection bounding box and the 2D image bounding box is calculated, and the minimum coordinate and maximum coordinate of the projection bounding box are combined to obtain the intersection-and-union calculation result.

[0087] Step 204: compare the matching calculation result with a preset matching threshold to obtain a comparison result.

[0088] For example, for each projected bounding box, the IoU calculation result is compared with a preset IoU threshold, and if the IoU calculation result exceeds the preset IoU threshold, the match is considered successful. Generally, the preset IoU threshold is set to 0.9.

[0089] Step 206: Match the image category information with the spatial information based on the comparison result to obtain a matching detection list.

[0090] The image surrounding information includes image category information.

[0091] Exemplarily, in the case of successful matching, the image category information in the image enclosing information is matched with the space enclosing information, ie, the 3D space enclosing box, and all matching results are formed into comprehensive information, ie, a matching detection list.

[0092] Step 208: Perform overlapping screening on the matching detection list to obtain a target detection list.

[0093] Exemplarily, the comprehensive information, i.e., all the spatial enclosing information in the matching detection list, is matched pairwise to determine whether there are overlapping enclosing boxes. If there are overlapping boxes, the overlapping boxes are merged, and the original overlapping boxes are replaced with the merged boxes, and the relevant category information is updated. Generally speaking, the merging method can be one of calculating the minimum rectangle enclosing the two boxes and calculating the center point of the two boxes and generating a new box with the center point as the center.

[0094] In one embodiment, the matching detection list includes converted encirclement information; overlapping screening is performed on the matching detection list to obtain a target detection list, including:

[0095] The converted encirclement information corresponding to one of the spatial information in the matching detection list is overlapped with the other converted encirclement information to obtain an overlap calculation result; the overlap calculation result is compared with a preset overlap threshold to obtain an overlap comparison result; the matching detection list is screened based on the overlap comparison result to obtain a target detection list.

[0096] Exemplarily, when processing the spatial enclosing information in the matching detection list, it is first necessary to perform pairwise matching calculations on all the enclosing boxes to determine whether they overlap. Overlap detection is usually achieved by calculating the intersection-over-union (IOU) of each pair of enclosing boxes. If the IOU value exceeds the set threshold, it can be determined that the two boxes overlap. After the overlapping boxes are detected, the boxes need to be merged. The merging method can be to calculate their minimum enclosing box to ensure that the generated new box can completely surround the original overlapping area.

[0097] Once the merge is complete, the next step is to replace the original overlapping boxes. To do this, the information of the merged box needs to be updated to the original detection list.

[0098] In one embodiment, before obtaining the data set label information, the method further includes:

[0099] Obtain data set label information; perform data processing on the data set label information according to a pre-built detection model to obtain a graph-text matching result; adjust the pre-built detection model according to the graph-text matching result, feedback data and a preset feedback function to obtain a target detection model and set the target detection model as a preset detection model.

[0100] Among them, the pre-built detection model includes a visual encoder and a text encoder.

[0101] For example, a basic data set containing most of the obstacle targets in the autonomous driving scene is obtained, and the data is cleaned to obtain the data set label information. Among them, the obstacle targets include vehicles, pedestrians, animals, traffic signs, signal lights, road facilities (cones, water barriers, etc.), natural objects (trees, stones), etc. The data set label information includes the target category, the target 2D box center coordinates, and the target 2D box length and width.

[0102] By using the visual encoder and text encoder to process the image content and label information in the dataset respectively, and then fusing the processed visual and language features, the image content is matched with the corresponding text description.

[0103] In addition, when the model generates detection results, the user provides feedback (positive or negative) on each result, and based on the collected feedback, the model is fine-tuned, that is, the model parameters are updated.

[0104] In one of the embodiments, before generating the data set label information based on the preset basic data set, the method further includes:

[0105] Acquire the current collected data; perform scene recognition based on the current collected data and preset scene data to obtain a scene recognition result; when the scene recognition result is a conventional scene, generate data set label information based on the preset basic data set; when the scene recognition result is an unconventional scene, generate data set label information based on the preset basic data set and the current collected data.

[0106] Exemplarily, scene recognition is performed on images captured by a camera to determine the current scene.

[0107] If the detection scene is a regular town, highway, etc., there is no need to collect data separately. If the scene is a special scene such as a port or mine, it is necessary to collect actual environment data separately.

[0108] In one embodiment, a target detection method is provided, the method comprising the following steps:

[0109] Get the current collection data.

[0110] The scene recognition is performed based on the current collected data and the preset scene data to obtain the scene recognition result.

[0111] When the scene recognition result is a regular scene, data set label information is generated based on a preset basic data set.

[0112] When the scene recognition result is an unconventional scene, data set label information is generated based on the preset basic data set and the currently collected data.

[0113] Get dataset label information.

[0114] The data set label information is processed according to the pre-built detection model to obtain the image-text matching results.

[0115] The pre-built detection model is adjusted according to the image-text matching result, feedback data and a preset feedback function to obtain a target detection model and set the target detection model as a preset detection model.

[0116] Obtain lidar point cloud data and image data to be processed.

[0117] Based on the preset detection model, the image data to be processed is recognized to obtain image surrounding information.

[0118] The laser radar point cloud data is preprocessed to obtain preprocessed point cloud data.

[0119] Based on the Euclidean point cloud clustering algorithm, the preprocessed point cloud data are processed to obtain multiple clustered point cloud data.

[0120] The extreme values ​​of three dimensions are screened for each clustered point cloud data to obtain multiple clustered point cloud extreme values ​​in three dimensions.

[0121] Generate multiple spatial enclosing information based on multiple clustered point cloud extreme values ​​in three dimensions.

[0122] Based on preset internal and external parameters of the sensor, multiple clustered point cloud data are transformed in dimension to obtain multiple transformed point cloud data.

[0123] Generate transformed enclosing information based on the plurality of transformed point cloud data.

[0124] Based on the intersection-and-union algorithm, the converted encirclement information and the image encirclement information are matched and calculated to obtain a matching calculation result.

[0125] The matching calculation result is compared with a preset matching threshold to obtain a comparison result.

[0126] Based on the comparison result, the image category information is matched with the spatial information to obtain a matching detection list.

[0127] The converted enclosing information corresponding to one of the spatial information in the matching detection list is overlapped with the other converted enclosing information to obtain an overlap calculation result.

[0128] The overlap calculation result is compared with a preset overlap threshold to obtain an overlap comparison result.

[0129] The matching detection list is screened based on the overlapping comparison results to obtain a target detection list.

[0130] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0131] Based on the same inventive concept, the embodiment of the present application also provides a target detection device for implementing the target detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target detection device embodiments provided below can refer to the limitations on the target detection method above, and will not be repeated here.

[0132] In one embodiment, as shown in FIG3 , a target detection device is provided, including: a data acquisition module 302 , an image processing module 304 , a point cloud processing module 306 and a matching detection module 308 , wherein:

[0133] The data acquisition module 302 is used to acquire the laser radar point cloud data and the image data to be processed;

[0134] An image processing module 304 is used to perform a recognition operation on the image data to be processed based on a preset detection model to obtain image surrounding information;

[0135] The point cloud processing module 306 is used to analyze and process the laser radar point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information;

[0136] The matching detection module 308 is used to perform matching detection based on the image enclosing information, the clustered point cloud data and multiple spatial enclosing boxes to obtain a target detection list.

[0137] In one embodiment, the point cloud processing module 306 is also used to preprocess the lidar point cloud data to obtain preprocessed point cloud data; perform extreme value screening in three dimensions on each clustered point cloud data to obtain multiple clustered point cloud extreme values ​​in three dimensions; and generate multiple spatial enclosing information based on multiple clustered point cloud extreme values ​​in three dimensions.

[0138] In one of the embodiments, the matching detection module 308 is further used to perform dimensional transformation on multiple clustered point cloud data based on preset internal and external parameters of the sensor to obtain multiple transformed point cloud data; generate transformed encirclement information based on the multiple transformed point cloud data; perform matching detection on the transformed encirclement information and the image encirclement information based on a preset evaluation algorithm to obtain a target detection list.

[0139] In one of the embodiments, the matching detection module 308 is further used to perform matching calculations on the converted enclosing information and the image enclosing information based on an intersection-and-union algorithm to obtain matching calculation results; compare the matching calculation results with a preset matching threshold to obtain a comparison result; match the image category information with the spatial information based on the comparison result to obtain a matching detection list; and perform overlapping screening on the matching detection list to obtain a target detection list.

[0140] In one of the embodiments, the matching detection module 308 is further used to perform overlap calculation on the converted encirclement information corresponding to one of the spatial information in the matching detection list and other converted encirclement information to obtain an overlap calculation result; compare the overlap calculation result with a preset overlap threshold to obtain an overlap comparison result; and filter the matching detection list based on the overlap comparison result to obtain a target detection list.

[0141] In one of the embodiments, the target detection device is used to obtain data set label information; perform data processing on the data set label information according to a pre-built detection model to obtain a graph-text matching result; adjust the pre-built detection model according to the graph-text matching result, feedback data and a preset feedback function to obtain a target detection model and set the target detection model as a preset detection model.

[0142] In one of the embodiments, the target detection device is also used to obtain current collected data; perform scene recognition based on the current collected data and preset scene data to obtain a scene recognition result; when the scene recognition result is a conventional scene, generate data set label information based on a preset basic data set; when the scene recognition result is an unconventional scene, generate data set label information based on the preset basic data set and the current collected data.

[0143] Each module in the above target detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0144] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG4. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data and point cloud data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target detection method is implemented.

[0145] Those skilled in the art will understand that the structure shown in FIG. 4 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0146] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0147] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0148] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0149] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0151] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A target detection method, characterized in that: The method comprises: Obtain laser radar point cloud data and image data to be processed; Performing a recognition operation on the image data to be processed based on a preset detection model to obtain image surrounding information; Analyzing and processing the laser radar point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information; Matching detection is performed based on the image enclosing information, the clustered point cloud data and the plurality of spatial enclosing boxes to obtain a target detection list.

2. The target detection method according to claim 1, characterized in that: The preset point cloud clustering algorithm is a Euclidean point cloud clustering algorithm; the laser radar point cloud data is analyzed and processed based on the preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information, including: Preprocessing the laser radar point cloud data to obtain preprocessed point cloud data; Based on the Euclidean point cloud clustering algorithm, the pre-processed point cloud data is subjected to clustering to obtain a plurality of clustered point cloud data; Performing extreme value screening in three dimensions on each of the clustered point cloud data to obtain multiple clustered point cloud extreme values ​​in three dimensions; A plurality of spatial enclosing information is generated based on the plurality of clustered point cloud extreme values ​​in three dimensions.

3. The target detection method according to claim 1, characterized in that: The performing matching detection based on the image encirclement information, the clustered point cloud data and the plurality of spatial encirclement frames to obtain a target detection list includes: Performing dimension transformation on the plurality of clustered point cloud data based on preset internal and external parameters of the sensor to obtain a plurality of transformed point cloud data; Generate transformed enclosing information based on the plurality of transformed point cloud data; The converted enclosing information and the image enclosing information are matched and detected based on a preset evaluation algorithm to obtain a target detection list.

4. The target detection method according to claim 3, characterized in that: The image encirclement information includes image category information, the preset evaluation algorithm is an intersection-over-union algorithm; performing matching detection on the converted encirclement information and the image encirclement information based on the preset evaluation algorithm to obtain a target detection list includes: Performing a matching calculation on the converted enclosing information and the image enclosing information based on an intersection-over-union algorithm to obtain a matching calculation result; Compare the matching calculation result with a preset matching threshold to obtain a comparison result; Matching the image category information with the spatial information based on the comparison result to obtain a matching detection list; Overlapping screening is performed on the matching detection list to obtain a target detection list.

5. The target detection method according to claim 4, characterized in that: The matching detection list includes converted encirclement information; and the overlapping screening of the matching detection list to obtain the target detection list includes: Performing overlapping calculation on the converted enclosing information corresponding to one of the spatial information in the matching detection list and the other converted enclosing information to obtain an overlapping calculation result; Comparing the overlap calculation result with a preset overlap threshold to obtain an overlap comparison result; The matching detection list is screened based on the overlapping comparison result to obtain a target detection list.

6. The target detection method according to claim 1, characterized in that: Before acquiring the laser radar point cloud data and the image data to be processed, the method further includes: Get dataset label information; Processing the label information of the data set according to the pre-built detection model to obtain a picture-text matching result; The pre-built detection model is adjusted according to the image-text matching result, feedback data and preset feedback function to obtain a target detection model and set the target detection model as the preset detection model.

7. The target detection method according to claim 6, characterized in that: Before obtaining the data set label information, the method further includes: Get the current collected data; Perform scene recognition based on current collected data and preset scene data to obtain scene recognition results; When the scene recognition result is a regular scene, generating data set label information based on a preset basic data set; When the scene recognition result is an unconventional scene, data set label information is generated based on a preset basic data set and the currently collected data.

8. A target detection device, characterized in that: The device comprises: A data acquisition module, used to acquire laser radar point cloud data and image data to be processed; An image processing module, used to perform a recognition operation on the image data to be processed based on a preset detection model to obtain image surrounding information; A point cloud processing module, used to analyze and process the laser radar point cloud data based on a preset point cloud clustering algorithm to obtain clustered point cloud data and multiple spatial enclosure information; A matching detection module is used to perform matching detection based on the image enclosing information, the clustered point cloud data and the multiple spatial enclosing boxes to obtain a target detection list.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images