Target detection methods, devices, electronic equipment and storage media

Abstract

This invention relates to the field of target detection technology, and discloses a target detection method, apparatus, electronic device, and storage medium. The target detection method includes: simultaneously acquiring first point cloud data and a target image; the first point cloud data is point cloud data collected by a lidar in the target direction; the target image is an image collected by an image acquisition device in the target direction; based on the distribution of each point data in the first point cloud data, detecting whether a region of interest exists in the target image, and obtaining a region detection result; if the region detection result indicates that a region of interest exists in the target image, then performing target detection on the region image corresponding to the region of interest in the target image, and obtaining a target detection result for the target image. The target detection method provided by this invention can help improve detection accuracy and efficiency, thereby ensuring the accuracy of the target detection results.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to target detection methods, devices, electronic equipment, and storage media. Background Technology

[0002] Environmental perception is the first step in autonomous driving, serving as the link between the vehicle and its environment. The overall performance of an autonomous driving system largely depends on the quality of its perception system.

[0003] In related technologies, when an autonomous driving system performs a target detection task, it first performs global detection on the image acquired by the image acquisition device, and then performs target detection on the determined region of interest to obtain the target detection result.

[0004] However, if the image acquisition device acquires images with low pixel counts and still uses the above method for target detection, the accuracy of target detection will be affected by unclear details, thus affecting the accuracy of the detection results. Summary of the Invention

[0005] In view of this, the present invention provides a target detection method, apparatus, electronic device and storage medium to solve the problem of target detection results being affected by low detection accuracy.

[0006] In a first aspect, the present invention provides a target detection method, the method comprising:

[0007] At the same time, the first point cloud data and the target image are acquired respectively; the first point cloud data is the point cloud data collected by the lidar in the target direction; the target image is the image collected by the image acquisition device in the target direction.

[0008] Based on the distribution of data at each point in the first point cloud data, the presence of a region of interest in the target image is detected to obtain the region detection result.

[0009] If the region detection result indicates that a region of interest exists in the target image, then target detection is performed on the region image in the target image corresponding to the region of interest to obtain the target detection result of the target image.

[0010] In this method, the distribution of data at each point in the first point cloud data can be combined to quickly detect whether there is a region of interest in the target image to be detected. When it is determined that there is a region of interest in the target image, targeted detection can be performed on the region image corresponding to the region of interest, which helps to improve detection accuracy and target detection efficiency, thereby ensuring the accuracy of target detection results.

[0011] In one optional implementation, based on the distribution of data at each point in the first point cloud data, the presence of a region of interest in the target image is detected to obtain a region detection result, including:

[0012] The first point cloud data is projected onto the target image to obtain the mapping relationship between point data and pixels;

[0013] Identify the size information of the target image;

[0014] The first point cloud data is filtered based on the size information to obtain the second point cloud data;

[0015] Based on the distribution of each point in the second point cloud data and the mapping relationship between point data and pixels, the presence of a region of interest in the target image is detected, and the region detection result is obtained.

[0016] In one optional implementation, the first point cloud data is filtered based on the size information to obtain second point cloud data, including:

[0017] Based on the size information, determine the boundary coordinate range of the target image;

[0018] Based on the distribution of each point in the first point cloud data, the point data within the boundary coordinate interval of the first point cloud data are retained to obtain the second point cloud data.

[0019] In one optional implementation, based on the distribution of point data in the second point cloud data and the mapping relationship between point data and pixels, the presence of a region of interest in the target image is detected to obtain a region detection result, including:

[0020] Clustering is performed on the data points in the second point cloud data to obtain the clustering results;

[0021] Based on the clustering results and the mapping relationship between point data and pixels, the presence of a region of interest in the target image is detected, and the region detection result is obtained.

[0022] In one optional implementation, clustering is performed on the point data in the second point cloud data to obtain clustering results, including:

[0023] Identify ground point cloud data within the second point cloud dataset;

[0024] The ground point cloud data is filtered out from the second point cloud data to obtain the third point cloud data;

[0025] Clustering is performed on the third point cloud data to obtain the clustering results.

[0026] This method can improve the accuracy of target detection while saving the computational load of target detection, thereby improving the efficiency of target detection.

[0027] In one optional implementation, target detection is performed on the region image corresponding to the region of interest in the target image to obtain the target detection result of the target image, including:

[0028] Extract the region image corresponding to the region of interest from the target image;

[0029] Target detection is performed on the region image to obtain the region target detection results;

[0030] Determine the position coordinates of the region image on the target image;

[0031] The target detection results of the region are marked on the position coordinates of the target image to obtain the target detection results of the target image.

[0032] In one alternative implementation, the method further includes:

[0033] If the region detection result indicates that there is no region of interest in the target image, then no target detection processing will be performed on the target image.

[0034] This approach makes target detection in target images more flexible and practical.

[0035] Secondly, the present invention provides a target detection device, the device comprising:

[0036] The acquisition module is used to acquire first point cloud data and target image simultaneously; the first point cloud data is the point cloud data acquired by the lidar in the target direction; the target image is the image acquired by the image acquisition device in the target direction.

[0037] The first detection module is used to detect whether there is a region of interest in the target image based on the distribution of data of each point in the first point cloud data, and to obtain the region detection result.

[0038] The second detection module is used to perform target detection on the region image in the target image corresponding to the region of interest if the region detection result is that there is a region of interest in the target image, so as to obtain the target detection result of the target image.

[0039] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the target detection method described in the first aspect or any corresponding embodiment thereof.

[0040] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the target detection method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0041] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a schematic flowchart of a target detection method according to an embodiment of the present invention;

[0043] Figure 2 This is a flowchart illustrating another target detection method according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the area detection results according to an embodiment of the present invention;

[0045] Figure 4 This is a flowchart illustrating another target detection method according to an embodiment of the present invention;

[0046] Figure 5 This is a flowchart illustrating another target detection method according to an embodiment of the present invention;

[0047] Figure 6 This is a structural block diagram of a target detection device according to an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] In related technologies, when an autonomous driving system performs a target detection task, it first performs global detection on the image acquired by the image acquisition device, then performs target detection on the determined region of interest, and finally obtains the target detection result.

[0051] However, when the image acquisition device acquires images with low pixel counts, if the above method is still used for target detection, the accuracy of target detection will be affected due to unclear details, thus affecting the accuracy of the detection results.

[0052] In view of this, embodiments of the present invention provide a target detection method, comprising: acquiring first point cloud data and a target image at the same time; the first point cloud data is point cloud data collected by a lidar in the target direction; the target image is an image collected by an image acquisition device in the target direction; based on the distribution of each point data in the first point cloud data, detecting whether there is a region of interest in the target image, and obtaining a region detection result; if the region detection result indicates that there is a region of interest in the target image, then performing target detection on the region image corresponding to the region of interest in the target image, and obtaining a target detection result for the target image. The target detection method provided by the present invention can quickly detect whether there is a region of interest to be detected in the target image by combining the distribution of each point data in the first point cloud data. When it is determined that there is a region of interest in the target image, targeted detection can be directly performed on the region image corresponding to the region of interest, thereby helping to improve detection accuracy and target detection efficiency, and thus ensuring the accuracy of the target detection result.

[0053] According to an embodiment of the present invention, a target detection method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0054] This embodiment provides a target detection method that can be used in the aforementioned electronic devices, such as the domain controllers of autonomous vehicles like dump trucks, mining trucks, forklifts, and excavators, or it can be a cloud server. Figure 1 This is a flowchart of a target detection method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0055] Step S101: At the same time, acquire the first point cloud data and the target image respectively.

[0056] In this embodiment of the invention, at the same time, first point cloud data collected by the lidar in the target direction and target images collected by the image acquisition device in the same target direction are acquired, thereby ensuring that the actual scene corresponding to the first point cloud data can cover the actual scene corresponding to the target image. Furthermore, acquiring the first point cloud data and the target image at the same time helps ensure that the object state corresponding to the first point cloud data is consistent with the object state corresponding to the target image. Therefore, during subsequent target detection, the first point cloud data can effectively assist in identifying the Region of Interest (ROI) in the target image, improving target detection accuracy. The image acquisition device may include a camera or scanner, or other acquisition device capable of obtaining two-dimensional images.

[0057] Step S102: Based on the distribution of data of each point in the first point cloud data, detect whether there is a region of interest in the target image and obtain the region detection result.

[0058] In this embodiment of the invention, the density of each point in the first point cloud data can be determined based on its distribution. Since the point data is generated by the laser emitted by the lidar hitting an object and reflecting back, if there are several points in the first point cloud data that are close together, it indicates that an object exists in the area corresponding to these points in the actual scene; if some points in the first point cloud data are too sparse, it indicates that no object exists in the area corresponding to these points in the actual scene.

[0059] The actual scene corresponding to the first point cloud data covers the actual scene corresponding to the target image. Therefore, based on the distribution of each point data in the first point cloud data, it is possible to determine whether the target image includes a region of interest where objects may exist, and thus obtain the region detection result.

[0060] Step S103: If the region detection result indicates that the target image contains a region of interest, then target detection is performed on the region image in the target image corresponding to the region of interest to obtain the target detection result of the target image.

[0061] In this embodiment of the invention, if the region detection result indicates that a region of interest exists in the target image, it signifies the presence of an object to be detected in the target image, requiring target detection. To improve target detection efficiency, targeted target detection is performed on the region image corresponding to the region of interest in the target image, thereby reducing the computational load of target detection and obtaining the target detection result of the target image.

[0062] The target detection method provided in this embodiment can quickly detect whether there is a region of interest to be detected in the target image by combining the distribution of data of each point in the first point cloud data. When it is determined that there is a region of interest in the target image, targeted detection can be performed on the region image corresponding to the region of interest, thereby helping to improve the detection accuracy and target detection efficiency, and thus ensuring the accuracy of the target detection results.

[0063] This embodiment provides a target detection method that can be used in the aforementioned electronic devices, such as the domain controllers of autonomous vehicles like dump trucks, mining trucks, forklifts, and excavators, or it can be a cloud server. Figure 2 This is a flowchart of a target detection method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0064] Step S201: At the same time, acquire the first point cloud data and the target image respectively. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0065] Step S202: Based on the distribution of data at each point in the first point cloud data, detect whether there is a region of interest in the target image and obtain the region detection result.

[0066] Specifically, step S202 includes:

[0067] Step S2021: Project the first point cloud data onto the target image to obtain the mapping relationship between point data and pixels.

[0068] In this embodiment of the invention, the first point cloud data is projected onto the target image using a preset projection formula to establish a correspondence between each point data in the first point cloud data and the pixels in the target image, thereby obtaining the mapping relationship between the point data and the pixels. This allows for the rapid determination of the region of interest in the target image that needs to be detected based on the distribution of each point data in the first point cloud data.

[0069] The preset projection formula is as follows:

[0070]

[0071] (u L ,v L Let K be the pixel coordinates of the point data in the first point cloud data projected onto the imaging plane, where K ∈ R. 3×3 The internal parameter matrix of the image acquisition device, (x L ,y L ,z L Let z be the spatial coordinates of the first point cloud data. cR is the height of the first point cloud data relative to the spatial coordinate system of the LiDAR coordinate system after translation and rotation transformations, relative to the spatial coordinate system of the image acquisition device. L ∈R 3×3 Let t be the relative angle deviation matrix between the lidar spatial coordinate system and the image acquisition device spatial coordinate system. L ∈R 3×3 This is the relative position deviation matrix between the lidar spatial coordinate system and the image acquisition device spatial coordinate system.

[0072] Step S2022: Identify the size information of the target image.

[0073] In this embodiment of the invention, the purpose of identifying the size information of the target image is to determine the effective point data in the first point cloud data, so as to improve the detection effectiveness and avoid the possibility of invalid detection when detecting the region of interest based on the distribution of each point data.

[0074] Step S2023: Filter the first point cloud data according to the size information to obtain the second point cloud data.

[0075] In this embodiment of the invention, based on the size information, the local area point data that completely corresponds to the target image in the first point cloud data can be identified, and then the local area point data is retained to obtain the second point cloud data.

[0076] In some optional implementations, step S2023 above includes:

[0077] Step a1: Determine the boundary coordinate range of the target image based on the size information.

[0078] In this method, the pixel boundaries of the target image can be clearly defined based on the size information. Furthermore, the boundary coordinate range of the target image can be determined based on the pixel coordinates of each boundary vertex. That is, the maximum abscissa u of the target image in the spatial coordinate system of the image acquisition device can be determined based on the pixel coordinates of each boundary vertex. L Minimum x-coordinate u L Maximum ordinate v L and the minimum ordinate v Ln This allows us to obtain the horizontal and vertical boundary coordinate intervals within the target image boundary coordinate interval. For example, the horizontal boundary coordinate interval is [u... L ,u L The vertical boundary coordinate range is [v] Ln ,v L ].

[0079] Step a2: Based on the distribution of each point in the first point cloud data, retain the point data that is within the boundary coordinate interval in the first point cloud data to obtain the second point cloud data.

[0080] In this method, in order to filter out invalid point data in the first point cloud data, the valid point data corresponding to each pixel in the target image is determined by using the boundary coordinate interval based on the distribution of each point data in the first point cloud data, and then the second point cloud data is obtained.

[0081] Specifically, if the pixel x-coordinate u of the midpoint data projected onto the imaging plane from the first point cloud data... L Located in the horizontal boundary coordinate range [u L ,u L The data at that point is projected onto the imaging plane at a pixel ordinate v between [a, b, c] and [b, c]. L Located in the vertical boundary coordinate range [v L ,v Lx If the pixel's x-coordinate is between 1 and 2, then the data at that point is considered valid and can be retained. Conversely, if the pixel's x-coordinate is outside the range of 1 / 2, then the data at that point is considered valid and can be retained. L Not within the horizontal boundary coordinate range [u L ,u L Between ], the pixel ordinate v L Not within the vertical boundary coordinate range [v L ,v Lx Between ], or the x-coordinate of the pixel u L Not within the horizontal boundary coordinate range [u L ,u L Between ] and the pixel ordinate v L Not within the vertical boundary coordinate range [v L ,v Lx If the data point is between ], it is determined to be invalid data and is therefore filtered out.

[0082] In some examples, a target constraint formula can be formulated based on the boundary coordinate range of the target image. Then, the pixel coordinates of each point in the first point cloud data are compared with the target constraint formula. If the target constraint formula is satisfied, the point is retained. Otherwise, if either the horizontal or vertical pixel coordinate of the point data does not satisfy the target constraint formula, it is filtered out.

[0083] The objective constraint expression can be as follows:

[0084]

[0085] Among them, u L u is the maximum abscissa in the spatial coordinate system of the image acquisition device. L v is the minimum abscissa in the spatial coordinate system of the image acquisition device. Lxv is the maximum ordinate in the spatial coordinate system of the image acquisition device. Ln u is the minimum ordinate in the spatial coordinate system of the image acquisition device. L Let v be the x-coordinate of any point in the first point cloud data. L This represents the corresponding pixel ordinate.

[0086] The method described above is used to obtain the second point cloud data, which ensures that each point data has a corresponding pixel on the target image. This improves the detection effectiveness of the region of interest when combining the point data to determine the region of interest, thereby helping to ensure the accuracy of target detection.

[0087] Step S2024: Based on the distribution of each point data in the second point cloud data and the mapping relationship between point data and pixels, detect whether there is a region of interest in the target image and obtain the region detection result.

[0088] In this embodiment of the invention, since each point in the second point cloud data has a corresponding pixel in the target image, based on the distribution of each point in the second point cloud data and the mapping relationship between each point and the pixel, it is possible to determine whether there is a local region in the target image corresponding to a large number of point data points. Therefore, the local region corresponding to the large number of point data points can be identified as a region of interest. The region detection result can include the following conclusions: a region of interest exists in the target image; a region of interest does not exist in the target image.

[0089] In some optional implementations, step S2024 above includes:

[0090] Step b1: Perform clustering processing on the data points in the second point cloud data to obtain the clustering results.

[0091] In this method, in order to determine the correlation between the data points in the second point cloud data, the data points in the second point cloud data are clustered to determine whether the data points in the second point cloud data belong to the same object based on the clustering results, so as to determine the distribution area of ​​the pixels corresponding to multiple data points belonging to the same object on the target image as the region of interest.

[0092] In some alternative implementations, step b1 above includes:

[0093] Step b11: Identify the ground point cloud data in the second point cloud data.

[0094] Step b12: Filter the ground point cloud data from the second point cloud data to obtain the third point cloud data.

[0095] Step b13: Perform clustering processing on the third point cloud data to obtain the clustering results.

[0096] Specifically, to prevent ground point cloud data from affecting the clustering results, the ground point cloud data in the second point cloud data is first identified and then filtered out to obtain a third point cloud data that does not contain ground point cloud data. Among existing technologies, the techniques for identifying ground point cloud data are relatively mature; therefore, any existing technology capable of identifying point cloud data can be used to identify ground point cloud data from the second point cloud data, and no limitation is imposed in this invention.

[0097] Euclidean clustering is employed to cluster the data points in the third point cloud data. Based on the Euclidean distance between the data points and a preset distance threshold, it is determined whether the data points belong to the same category. The distance threshold is the maximum Euclidean distance used to determine whether two data points belong to the same category. If the Euclidean distance between two data points is less than or equal to the distance threshold, they are clustered into the same category. If the Euclidean distance between two data points is greater than the distance threshold, they are not clustered into the same category.

[0098] In one example, the number of clusters in the clustering results can be used to determine whether to perform secondary clustering on each data point to reduce the number of clusters and avoid having too many regions of interest to be detected, which would lead to excessively long target detection time.

[0099] If the number of clusters in the clustering result is less than or equal to a specified threshold, the distribution area of ​​pixels corresponding to multiple point data belonging to the same category on the target image can be used as the region of interest based on the mapping relationship between point data and pixels. If the number of clusters in the clustering result is greater than the specified threshold, the distance threshold is increased to reduce the number of clusters.

[0100] Step b2: Based on the clustering results and the mapping relationship between point data and pixels, detect whether there is a region of interest in the target image and obtain the region detection result.

[0101] In this approach, based on the clustering results, it can be determined whether each point in the third point cloud data belongs to the same category. Based on the mapping relationship between each point and a pixel, the distribution area of ​​pixels corresponding to multiple point data belonging to the same category in the target image can be determined. Then, by detecting the distribution area, the existence of a region of interest in the target image can be obtained, yielding the region detection result.

[0102] Step S203: If the region detection result indicates that the target image contains a region of interest, then target detection is performed on the region image in the target image corresponding to the region of interest to obtain the target detection result of the target image.

[0103] In some optional implementations, step S203 above includes:

[0104] Step S2031: Extract the region image corresponding to the region of interest from the target image.

[0105] In this embodiment of the invention, if the region detection result indicates that the target image contains a region of interest, the obtained region detection result can be as follows: Figure 3 As shown in the figure. The region bounded by the box in the target image is the region of interest.

[0106] To improve target detection efficiency and avoid loss of detail due to image scaling, a region image corresponding to the region of interest is extracted from the target image. This region image is then used for global target detection, which helps improve the accuracy of target detection.

[0107] In some examples, due to potential errors in projecting point data onto pixel coordinates, a mapping error rate is determined to reduce these errors and improve the accuracy and effectiveness of target detection results. This error rate is then used to determine the region magnification ratio, which is used to enlarge the target region image corresponding to the region of interest. The enlarged target region image then serves as the region image to be extracted. For example, if the error rate is 5%, the overall size of the target region image is increased by 5% to cover the error range.

[0108] Step S2032: Perform target detection on the region image to obtain the region target detection result of the region image.

[0109] In this embodiment of the invention, any existing target detection method can be used to perform target detection on the region image to obtain the region target detection result of the region image, and no limitation is made in this invention.

[0110] Step S2033: Determine the position coordinates of the region image on the target image.

[0111] In this embodiment of the invention, since target detection is performed on a region image extracted from the target image, the position coordinates of the region image on the target image are determined in order to annotate the obtained target detection results on the target image.

[0112] Step S2034: Mark the target detection results in the region on the position coordinates of the target image to obtain the target detection results of the target image.

[0113] In this embodiment of the invention, since there can be multiple regions of interest, the target detection results of each region image can be marked on the corresponding position coordinates of the target image, thereby obtaining the complete target detection results of the target image.

[0114] The target detection method provided in this embodiment can improve the accuracy of target detection while saving the computational load of target detection, thereby improving the efficiency of target detection.

[0115] This embodiment provides a target detection method that can be used in the aforementioned electronic devices, such as the domain controllers of autonomous vehicles like dump trucks, mining trucks, forklifts, and excavators, or it can be a cloud server. Figure 4 This is a flowchart of a target detection method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0116] Step S401: At the same time, acquire the first point cloud data and the target image respectively. For detailed explanation, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0117] Step S402: Based on the distribution of data at each point in the first point cloud data, detect whether there is a region of interest in the target image, and obtain the region detection result. For detailed explanation, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0118] Step S403: If the region detection result indicates that a region of interest exists in the target image, then target detection is performed on the region image corresponding to the region of interest in the target image to obtain the target detection result of the target image. For detailed explanation, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0119] Step S404: If the region detection result indicates that there is no region of interest in the target image, then no target detection processing is performed on the target image.

[0120] In this embodiment of the invention, if the number of clusters in the clustering result is large and the data of each point is relatively scattered, it is determined that there is no region of interest in the target image, and therefore no target detection processing is performed on the target image to avoid the ineffective use of computing resources.

[0121] The target detection method provided in this embodiment makes the way to detect targets in target images more flexible and more practical.

[0122] In some optional implementation scenarios, such as Figure 5 As shown, the object detection process can be as follows:

[0123] Step S501: At the same time, for the same target direction, first point cloud data is acquired by lidar and target image is acquired by image acquisition device.

[0124] Step S502: Map the first point cloud data onto the target image to detect whether there is a region of interest in the target image based on the distribution of each point data in the first point cloud data, and obtain the region detection result.

[0125] Step S503: If the region detection result indicates the existence of a region of interest, then extract the region image corresponding to the region of interest from the target image to perform target detection on the region image and obtain the detection result of the region image.

[0126] Step S504: Mark the target detection result of the region image on the position coordinates of the target image corresponding to the region image, and thus obtain the target detection result of the target image.

[0127] Step S505: If the region detection result is that there is no region of interest, then no target detection processing is performed on the target image.

[0128] The target detection method provided by this invention, combined with the first point cloud data acquired by lidar, performs target detection on the target image acquired by the image acquisition device, which can improve the accuracy and efficiency of target detection results from multiple dimensions. Furthermore, utilizing multiple sensors for target detection enhances the practicality of each sensor.

[0129] This embodiment also provides a target detection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0130] This embodiment provides a target detection device, such as Figure 6 As shown, it includes:

[0131] The acquisition module 601 is used to acquire first point cloud data and target image at the same time; the first point cloud data is the point cloud data acquired by the lidar in the target direction; the target image is the image acquired by the image acquisition device in the target direction.

[0132] The first detection module 602 is used to detect whether there is a region of interest in the target image based on the distribution of each point data in the first point cloud data, and to obtain the region detection result.

[0133] The second detection module 603 is used to perform target detection on the region image in the target image corresponding to the region of interest if the region detection result is that there is a region of interest in the target image, so as to obtain the target detection result of the target image.

[0134] In some optional implementations, the first detection module 602 includes: a mapping unit for projecting first point cloud data onto a target image to obtain a mapping relationship between point data and pixels; a first recognition unit for recognizing the size information of the target image; a first filtering unit for filtering the first point cloud data according to the size information to obtain second point cloud data; and a region detection unit for detecting whether there is a region of interest in the target image based on the distribution of each point data in the second point cloud data and the mapping relationship between point data and pixels, to obtain a region detection result.

[0135] In some optional implementations, the first filtering unit includes: a boundary determination unit, used to determine the boundary coordinate range of the target image based on the size information; and a region filtering unit, used to retain the point data in the first point cloud data that are within the boundary coordinate range based on the distribution of each point data in the first point cloud data, to obtain the second point cloud data.

[0136] In some optional implementations, the region detection unit includes: a clustering unit for clustering the point data in the second point cloud data to obtain a clustering result; and a first detection unit for detecting whether there is a region of interest in the target image based on the clustering result and the mapping relationship between the point data and the pixels, to obtain a region detection result.

[0137] In some optional implementations, the clustering unit includes: a second identification unit for identifying ground point cloud data in the second point cloud data; a point data filtering unit for filtering out ground point cloud data from the second point cloud data to obtain third point cloud data; and a clustering execution unit for performing clustering processing on the third point cloud data to obtain clustering results.

[0138] In some optional implementations, the second detection module includes: an extraction unit for extracting a region image corresponding to the region of interest from the target image; a second detection unit for performing target detection on the region image to obtain a region target detection result of the region image; a position determination unit for determining the position coordinates of the region image on the target image; and a labeling unit for labeling the region target detection result on the position coordinates of the target image to obtain a target detection result of the target image.

[0139] In some optional embodiments, the apparatus further includes a third detection module, configured to not perform target detection processing on the target image if the region detection result indicates that the target image does not contain a region of interest.

[0140] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0141] In this embodiment, the target detection device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0142] This invention also provides an electronic device having the above-described features. Figure 6 The target detection device shown.

[0143] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of the present invention, such as... Figure 7 As shown, the electronic device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0144] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0145] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0146] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0147] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0148] The electronic device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0149] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touch screen.

[0150] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0151] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A target detection method, characterized in that, The method includes: At the same time, first point cloud data and target image are acquired respectively; the first point cloud data is point cloud data collected by lidar in the target direction; the target image is an image collected by image acquisition device in the target direction; Based on the distribution of data at each point in the first point cloud data, the presence of a region of interest in the target image is detected to obtain a region detection result. If the region detection result indicates that the target image contains a region of interest, then target detection is performed on the region image in the target image corresponding to the region of interest to obtain the target image target detection result; The step of detecting whether a region of interest exists in the target image based on the distribution of data at each point in the first point cloud data, and obtaining the region detection result, includes: The first point cloud data is projected onto the target image to obtain the mapping relationship between point data and pixels; Identify the size information of the target image; Based on the size information, the first point cloud data is filtered to obtain the second point cloud data; Based on the distribution of each point in the second point cloud data and the mapping relationship between the point data and the pixels, the presence of a region of interest in the target image is detected, and the region detection result is obtained.

2. The method according to claim 1, characterized in that, The step of filtering the first point cloud data based on the size information to obtain the second point cloud data includes: Based on the size information, determine the boundary coordinate range of the target image; Based on the distribution of each point in the first point cloud data, the point data within the boundary coordinate interval in the first point cloud data are retained to obtain the second point cloud data.

3. The method according to claim 2, characterized in that, Based on the distribution of each point in the second point cloud data and the mapping relationship between the point data and pixels, the method detects whether there is a region of interest in the target image, and obtains the region detection result, including: Clustering is performed on the data points in the second point cloud data to obtain the clustering results; Based on the clustering results and the mapping relationship between the point data and pixels, the presence of a region of interest in the target image is detected, and the region detection result is obtained.

4. The method according to claim 3, characterized in that, The clustering process for each point in the second point cloud data to obtain the clustering results includes: Identify the ground point cloud data in the second point cloud data; The ground point cloud data is filtered out from the second point cloud data to obtain the third point cloud data; The third point cloud data is subjected to clustering processing to obtain the clustering result.

5. The method according to claim 4, characterized in that, The step of performing target detection on the region image corresponding to the region of interest in the target image to obtain the target detection result of the target image includes: Extract the region image corresponding to the region of interest from the target image; Target detection is performed on the region image to obtain the region target detection result of the region image; Determine the position coordinates of the region image on the target image; The target detection results of the region are marked on the position coordinates of the target image to obtain the target detection results of the target image.

6. The method according to claim 1, characterized in that, The method further includes: If the region detection result indicates that the target image does not contain a region of interest, then no target detection processing will be performed on the target image.

7. A target detection device, characterized in that, The device includes: The acquisition module is used to acquire first point cloud data and target image simultaneously; the first point cloud data is point cloud data acquired by the lidar in the target direction; the target image is an image acquired by the image acquisition device in the target direction. The first detection module is used to detect whether there is a region of interest in the target image based on the distribution of data of each point in the first point cloud data, and to obtain the region detection result; The second detection module is used to perform target detection on the region image in the target image corresponding to the region of interest if the region detection result is that the target image has a region of interest, so as to obtain the target detection result of the target image; The first detection module further includes: The mapping unit is used to project the first point cloud data onto the target image to obtain the mapping relationship between point data and pixels. The first recognition unit is used to recognize the size information of the target image; The first filtering unit is used to filter the first point cloud data according to the size information to obtain the second point cloud data. The region detection unit is used to detect whether there is a region of interest in the target image based on the distribution of each point data in the second point cloud data and the mapping relationship between the point data and the pixel, and to obtain the region detection result.

8. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the target detection method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the target detection method according to any one of claims 1 to 6.

Images