Detection frame filtering method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a detection frame filtering method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining a detection frame generated by target detection for a laser point cloud, and determining the frame type information corresponding to the detection frame; semantic segmentation information obtained by carrying out semantic segmentation on the laser point cloud is obtained, and the semantic segmentation information comprises semantic categories corresponding to all unit points in the laser point cloud; determining whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point; and if the detection frame is a false detection frame, deleting the detection frame. According to the method, the detection frame generated by target detection is evaluated through the semantic category corresponding to each unit point obtained through semantic segmentation, so that the false detection frame in the detection frame can be determined by combining semantic segmentation, the combination degree of target detection and semantic segmentation is improved, and meanwhile, the detection efficiency is improved by deleting the false detection frame. And the accuracy of the detection frame on which downstream operation depends is higher, so that the detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving, and in particular to a detection frame filtering method, device, electronic device and computer-readable storage medium. Background Art

[0002] In the existing intelligent driving technology, target objects are usually detected sequentially and continuously through target detection and semantic segmentation. In the existing technology, the degree of integration between semantic segmentation and target detection is low, and the information utilization of semantic segmentation is small. There is a large room for improvement in the detection accuracy of the recognition method combining target detection and semantic segmentation. Summary of the invention

[0003] The main purpose of the present invention is to propose a detection frame filtering method, device, electronic device and computer-readable storage medium, aiming to solve the problem that there is a large room for improvement in the detection accuracy of the recognition method combining target detection and semantic segmentation in the prior art.

[0004] To achieve the above object, the present invention provides a detection frame filtering method, the method comprising the steps of:

[0005] Obtaining a detection frame generated by target detection for the laser point cloud, and determining frame category information corresponding to the detection frame;

[0006] Acquire semantic segmentation information obtained by semantically segmenting the laser point cloud, wherein the semantic segmentation information includes a semantic category corresponding to each unit point in the laser point cloud;

[0007] Determine whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point;

[0008] If the detection frame is a false detection frame, the detection frame is deleted.

[0009] Optionally, determining whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point includes:

[0010] For each of the detection frames, determining the semantic category corresponding to the unit point contained in the detection frame;

[0011] For each of the unit points, determining whether the unit point is a noise point according to the frame category information and the semantic category;

[0012] Determine a noise ratio of the number of noise points contained in the detection frame to the number of the unit points contained in the detection frame;

[0013] Determining whether the noise ratio is greater than a preset ratio threshold;

[0014] If the noise ratio is greater than the preset ratio threshold, the detection frame is determined to be a false detection frame.

[0015] Optionally, before determining, for each detection frame, the semantic category corresponding to the unit point contained in the detection frame, the step includes:

[0016] Determine the number of unit points of the unit points included in the laser point cloud;

[0017] Generate detection threads equal to the number of the unit points, wherein the detection threads correspond to the unit points one by one;

[0018] Each of the detection frames is traversed in parallel by the detection threads to determine the unit points contained in each of the detection frames.

[0019] Optionally, the parallel traversing of each detection frame through the detection thread to determine the unit point contained in each detection frame includes:

[0020] Determine the center point coordinates and size information of each detection frame;

[0021] Determine the coordinate range covered by each detection frame according to the center point coordinates and the size information;

[0022] For each of the detection threads, determining the unit point coordinates of the corresponding unit point;

[0023] Traversing each of the detection frames, and determining whether the coordinate range corresponding to the detection frame contains the unit point coordinates;

[0024] If the coordinate range corresponding to the detection frame includes the coordinates of the unit point, it is determined that the detection frame includes the unit point.

[0025] Optionally, if the noise ratio is greater than the preset ratio threshold, determining that the detection frame is a false detection frame includes:

[0026] If the noise ratio is greater than the preset ratio threshold, determining a convex hull formed by unit points contained in the false detection frame;

[0027] Obtaining the object size corresponding to the detection frame;

[0028] Calculating a convex hull ratio between the convex hull and the object size, and determining whether the convex hull ratio is less than a preset convex hull ratio threshold;

[0029] If the convex hull ratio is less than a preset convex hull ratio threshold, the detection frame is determined to be a false detection frame.

[0030] Optionally, if the convex hull ratio is less than a preset convex hull ratio threshold, determining that the detection frame is a false detection frame includes:

[0031] If the convex hull ratio is less than a preset convex hull ratio threshold, determining the point cloud ratio within the edge sub-detection frame of the detection frame;

[0032] Determine whether the point cloud ratio is less than a preset point cloud ratio threshold;

[0033] If the point cloud ratio is less than a preset point cloud ratio threshold, the detection frame is determined to be a false detection frame.

[0034] Optionally, determining the point cloud ratio within the edge sub-detection frame of the detection frame includes:

[0035] For each side of the detection frame on the horizontal plane, determining a preset frame ratio corresponding to the side;

[0036] Determining the edge sub-detection frame corresponding to the side edge according to the side edge and the preset frame ratio, wherein one side of the edge sub-detection frame coincides with the side edge;

[0037] For each edge sub-detection frame, obtaining the size of a sub-point cloud composed of unit points contained in the edge sub-detection frame;

[0038] The ratio of the size of the sub-point cloud to the size of the edge sub-detection frame is used as the corresponding point cloud ratio within the edge sub-detection frame.

[0039] To achieve the above object, the present invention further provides a detection frame filtering device, the detection frame filtering device comprising:

[0040] A first acquisition module is used to acquire a detection frame generated by target detection for the laser point cloud, and determine frame category information corresponding to the detection frame;

[0041] A second acquisition module is used to acquire semantic segmentation information obtained by semantically segmenting the laser point cloud, wherein the semantic segmentation information includes a semantic category corresponding to each unit point in the laser point cloud;

[0042] A first determination module, configured to determine whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point;

[0043] The first deleting module is used to delete the detection frame if the detection frame is a false detection frame.

[0044] To achieve the above-mentioned purpose, the present invention also provides an electronic device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program, when executed by the processor, implements the steps of the detection frame filtering method as described above.

[0045] To achieve the above-mentioned purpose, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the detection frame filtering method as described above are implemented.

[0046] The present invention proposes a detection frame filtering method, device, electronic device and computer-readable storage medium, which obtains the detection frame generated for the laser point cloud by target detection, and determines the frame category information corresponding to the detection frame; obtains the semantic segmentation information obtained by semantically segmenting the laser point cloud, wherein the semantic segmentation information includes the semantic category corresponding to each unit point in the laser point cloud; determines whether the detection frame is a false positive frame according to the frame category information and the semantic category corresponding to the unit point; if the detection frame is a false positive frame, deletes the detection frame. By evaluating the detection frame generated by target detection based on the semantic category corresponding to each unit point obtained by semantic segmentation, it is possible to determine the false positive frame in the detection frame in combination with semantic segmentation, thereby improving the degree of integration between target detection and semantic segmentation. At the same time, by deleting the false positive frame, the accuracy of the detection frame relied on by the downstream operation is higher, thereby achieving improved detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 It is a flowchart of a first embodiment of a detection frame filtering method of the present invention;

[0050] Figure 2 It is a schematic diagram of the overall process of the detection frame filtering method of the present invention;

[0051] Figure 3 It is a schematic diagram of the module structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present application.

[0053] The present invention provides a detection frame filtering method, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a detection frame filtering method of the present invention, the method comprising the steps of:

[0054] Step S10, obtaining a detection frame generated by target detection for the laser point cloud, and determining frame category information corresponding to the detection frame;

[0055] Object detection refers to the technology of identifying and locating specific objects in images or videos.

[0056] In this embodiment, the detection object is a laser point cloud, which is collected by a laser radar installed on the vehicle.

[0057] The detection frame is used to mark or identify the boundary of a specific object during the target detection process. In this embodiment, it is applied to the detection of laser point clouds, so the detection frame can be set to a rectangular shape; the detection frame can be used to locate the target object in the detection object. For example, in application scenarios such as face recognition and vehicle detection, the detection frame can help the algorithm accurately determine the position and size of the target. It can be understood that the detection frame is divided based on different categories of unit points in the laser point cloud determined by target detection. Therefore, the detection frame can determine the frame category information based on the general category of the unit points contained therein, and the frame category information indicates the category of the unit points contained therein based on target detection.

[0058] Step S20, obtaining semantic segmentation information obtained by semantically segmenting the laser point cloud, wherein the semantic segmentation information includes a semantic category corresponding to each unit point in the laser point cloud;

[0059] Semantic segmentation is used to assign a specific category or label to each unit point in the laser point cloud, that is, to assign semantic categories, so that the boundaries and types of different objects in the laser point cloud can be understood, thereby achieving a detailed analysis of the scene; it can be understood that the semantic category is determined based on the type of objects contained in the actual acquisition scene of the laser point cloud, such as semantic categories can include roads, pedestrians, traffic signs, rain, dust, exhaust gas, fog, etc.

[0060] The specific semantic segmentation algorithm can be set based on actual needs.

[0061] A unit point is an independent measurement point in a laser point cloud. The specific number of unit points in a laser point cloud varies based on the actual scene. Generally, a laser point cloud may contain 90,000 to 150,000 unit points. Each unit point has its corresponding semantic category determined based on semantic segmentation. Correspondingly, the number of detection boxes determined by target detection based on the laser point cloud varies based on the actual scene. Generally, target detection will generate hundreds of detection boxes.

[0062] Step S30, determining whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point;

[0063] Step S40: If the detection frame is a false detection frame, the detection frame is deleted.

[0064] If the detection frame is not a false detection frame, the detection frame is retained.

[0065] It is understandable that in target detection, due to environmental interference or detection accuracy and other reasons, the detection frame generated by target detection may have the problem of false detection, that is, the unit points contained in the detection frame cannot roughly correspond to the independent specific object. Therefore, the specific object cannot be determined based on the false detection frame. In order to improve the control accuracy, the false detection frame is deleted in this embodiment to provide more accurate detection frame information for the downstream.

[0066] From the above content, it can be seen that the category of the unit point in the laser point cloud can be clarified through the semantic category, and the detection frame is also divided based on the category of the object corresponding to the unit point. Therefore, by comparing the semantic category corresponding to each unit point with the frame category information of the detection frame, it can be determined whether the detection frame is a false detection frame.

[0067] After deleting the detection box, the retained detection box and other related information can be transmitted to the downstream for processing; in other embodiments, after determining that the detection box is a false detection box, the detection box can be associated with a false detection identifier, and the detection box and the false detection identifier can be transmitted to the downstream without directly performing the deletion operation; the specific processing operation for the false detection box can be selected based on actual needs.

[0068] In this embodiment, the detection frame generated by the target detection is evaluated by relying on the semantic categories corresponding to each unit point obtained by semantic segmentation, so that the false detection frame in the detection frame can be determined in combination with the semantic segmentation, thereby improving the degree of integration between the target detection and semantic segmentation. At the same time, by deleting the false detection frame, the detection frame on which the downstream operations rely is more accurate, thereby achieving improved detection accuracy.

[0069] Further, see also Figure 2 In the second embodiment of the detection frame filtering method of the present invention proposed based on the first embodiment of the present invention, the step S30 includes:

[0070] Step S31, for each of the detection frames, determining the semantic category corresponding to the unit point contained in the detection frame;

[0071] Step S32, for each of the unit points, determining whether the unit point is a noise point according to the frame category information and the semantic category;

[0072] Step S33, determining the noise ratio of the number of noise points contained in the detection frame to the number of unit points contained in the detection frame;

[0073] Step S34, determining whether the noise ratio is greater than a preset ratio threshold;

[0074] Step S35: if the noise ratio is greater than the preset ratio threshold, determine that the detection frame is a false detection frame.

[0075] It can be understood that the unit points contained in the detection frame need to correspond to the objects indicated by the frame category information corresponding to the detection frame; and when a large number of unit points that do not correspond to the objects indicated by the frame category information appear in the detection frame, it means that the detection frame cannot reflect the independent objects in the laser point cloud and cannot fully play the role of the detection frame for indicating specific objects in the laser point cloud; therefore, in this embodiment, the frame category information and the semantic category are used to determine whether the unit point is a noise point.

[0076] It can be understood that the noise points in the unit points can be determined based on the actual unit points. For example, for the unit points in the laser point cloud whose semantic categories are interference objects such as background, ground, dust, fog, reflection, exhaust gas, rain, etc., they can be directly determined as noise points. Furthermore, it can also be determined whether the semantic category corresponding to the unit point matches the frame category information. It can be understood that the frame category information indicates the category that the unit point in the detection frame should have, and the semantic category indicates the actual category of the unit point. Therefore, when the semantic type corresponding to the unit point matches the frame category information, it is considered that the unit point belongs to the object corresponding to the detection frame, and the unit point is not a noise point. On the contrary, when the semantic type corresponding to the unit point does not match the frame category information, it is considered that the unit point does not belong to the object corresponding to the detection frame, and the unit point is a noise point.

[0077] After determining whether each unit point in the detection frame is a noise point, the ratio of the noise points in the detection frame to all the unit points in the detection frame, that is, the noise ratio, can be determined; it can be understood that the larger the noise ratio, the fewer the unit points contained in the detection frame belong to the object corresponding to the detection frame, and conversely, the smaller the noise ratio, the more the unit points contained in the detection frame belong to the object corresponding to the detection frame; therefore, when the noise ratio is too large, it is considered that the unit points contained in the detection frame cannot reflect the specific object, and therefore, the detection frame is determined to be a false detection frame; in this embodiment, a preset ratio threshold is set to indicate that the noise ratio is too large, and the specific value of the preset ratio threshold can be set based on actual needs; when the noise ratio is greater than the preset ratio threshold, it is considered that the noise ratio is too large, and the detection frame is determined to be a false detection frame; when the noise ratio is less than or equal to the preset ratio threshold, it is considered that the noise ratio is small, and the detection frame is determined not to be a false detection frame.

[0078] In this embodiment, the noise ratio can be used to accurately determine whether the detection frame is a false detection frame, thereby distinguishing whether the detection frame is detected due to the aggregation of noise points, and eliminating environmental influences when the environment is unfriendly to laser detection.

[0079] Furthermore, before step S31, the following steps are included:

[0080] Step S36, determining the number of unit points of the unit points contained in the laser point cloud;

[0081] Step S37, generating detection threads equal to the number of the unit points, wherein the detection threads correspond to the unit points one by one;

[0082] Step S38: traverse each of the detection frames in parallel through the detection threads to determine the unit points contained in each of the detection frames.

[0083] It is understandable that after the target detection generates a detection frame, it is necessary to determine the positional relationship between the detection frame and the unit points in the laser point cloud, so as to know which unit points are specifically included in the detection frame.

[0084] In the prior art, the point cloud contained in the detection box is usually determined one by one in turn; this method will cause multiple traversals between the detection box and the point cloud, increasing the time consumption; at the same time, the one-by-one traversal method will also cause the algorithm to take too long; therefore, in order to improve the efficiency of the algorithm, in this embodiment, the detection box is traversed in parallel by generating detection threads consistent with the number of unit points, thereby greatly reducing the time consumption in the process of determining the association between the unit point and the detection box, thereby improving the efficiency of the algorithm.

[0085] When executing parallel traversal, GPU can be used to generate detection threads through CUDA (Compute Unified Device Architecture) of GPU, thereby realizing parallel traversal.

[0086] Specifically, each thread traverses the detection frame based on its corresponding unit point, and needs to determine in which detection frame the unit point is located; after the detection frames where all unit points are located are determined, the correspondence between each unit point and the detection frame is known, so it is possible to reversely determine which unit points are included in each detection frame.

[0087] Furthermore, the step S38 includes:

[0088] Step S381, determining the center point coordinates and size information of each detection frame;

[0089] Step S382, determining the coordinate range covered by each detection frame according to the center point coordinates and the size information;

[0090] Step S383, for each of the detection threads, determining the unit point coordinates of the corresponding unit point;

[0091] Step S384, traversing each of the detection frames, and determining whether the coordinate range corresponding to the detection frame contains the unit point coordinates;

[0092] Step S385: If the coordinate range corresponding to the detection frame includes the coordinates of the unit point, it is determined that the detection frame includes the unit point.

[0093] The detection frame for the laser point cloud is a rectangular parallelepiped, so the coverage area of ​​the detection frame is reflected as a rectangular coordinate range in the coordinate system; therefore, in this embodiment, the coordinate range covered by the detection frame is determined by the center point coordinates and size information of the detection frame.

[0094] The unit point coordinates can be obtained from the relevant information of the laser point cloud; the unit point coordinates can be expressed as (x, y, z). Generally, the unit point also includes its corresponding reflection intensity idensity.

[0095] When the unit point coordinates of the unit point are within the coordinate range corresponding to the detection frame, it is considered that the detection frame contains the unit point; when the unit point coordinates of the unit point are not within the coordinate range corresponding to the detection frame, it is considered that the detection frame does not contain the unit point.

[0096] It should be noted that the detection frame is based on the coordinate system of target detection, and the unit point in the laser point cloud is based on the point cloud coordinate system. Therefore, before comparing the coordinates of the unit point with the coordinate range of the detection frame, the detection frame and the laser point cloud need to be unified into the same coordinate system. Specifically, the heading information of the detection frame can be used as a reference, and the detection frame can be converted into the point cloud coordinate system through rotation and translation formulas.

[0097] Furthermore, the step S35 includes:

[0098] Step S351, if the noise ratio is greater than the preset ratio threshold, determining the convex hull formed by the unit points contained in the false detection frame;

[0099] Step S352, obtaining the object size corresponding to the detection frame;

[0100] Step S353, calculating the convex hull ratio between the convex hull and the object size, and determining whether the convex hull ratio is less than a preset convex hull ratio threshold;

[0101] Step S354: if the convex hull ratio is less than a preset convex hull ratio threshold, the detection frame is determined to be a false detection frame.

[0102] In this embodiment, whether the detection frame is a false detection frame is comprehensively judged in combination with the convex hull condition of the detection frame.

[0103] The convex hull is used to indicate the actual size of the object indicated by the detection frame; the specific structure of the convex hull can be set based on actual needs; in this embodiment, the maximum and minimum values ​​of the unit points contained in the detection frame in the x, y, and z directions are determined, and then a convex hull in the shape of a rectangular parallelepiped is formed based on the maximum and minimum values ​​in the x, y, and z directions, that is, the convex hull is the smallest rectangular parallelepiped containing all the unit points in the detection frame.

[0104] The object size is the size of the object contained in the detection box determined by the target detection calculation.

[0105] After the convex hull is determined, the convex hull ratio between the convex hull and the object size corresponding to the detection box is calculated. Specifically, the convex hull / object size is used as the convex hull ratio.

[0106] It can be understood that the smaller the convex hull ratio is, the greater the deviation between the actual point cloud in the detection frame and the object size determined by the detection frame is, the less the detection frame can reflect the specific object, and the more likely it is to be an object frame; conversely, the larger the convex hull ratio is, the smaller the deviation between the actual point cloud in the detection frame and the object size determined by the detection frame is, the more the detection frame can reflect the specific object, and the less likely it is to be a false detection frame.

[0107] In this embodiment, a preset convex hull ratio threshold is specifically set; when the convex hull ratio is less than the preset convex hull ratio threshold, the detection frame is determined to be a false detection frame; when the convex hull ratio is greater than the preset convex hull ratio threshold, the detection frame is determined not to be a false detection frame.

[0108] It should be noted that the convex hull detection in this embodiment can also be implemented based on CUDA parallel computing; specifically, a thread is generated for each detection box, and the maximum and minimum values ​​of the unit points contained in the detection box in the x, y, and z directions are determined in parallel through CUDA synchronous calculations and CUDA atomic operations; it should be noted that since the coordinates of the point cloud are often of float type, when performing atomic operations for size comparison, CUDA's existing atomicMax and atomicMin operations cannot be used, because these two functions only support comparison of int type data, so the atomicCAS function can be used instead for float type atomic function size comparison.

[0109] Furthermore, the step S354 includes:

[0110] Step S3541, if the convex hull ratio is less than a preset convex hull ratio threshold, determining the point cloud ratio within the edge sub-detection frame of the detection frame;

[0111] In this embodiment, whether the detection frame is a false detection frame is comprehensively judged in combination with the convex hull condition of the detection frame.

[0112] The edge sub-detection frame is used to determine the point cloud situation of the edge position of the detection frame; the specific settings of the edge sub-detection frame are: the step S3541 includes:

[0113] Step S35411, for each side of the detection frame on the horizontal plane, determining a preset frame ratio corresponding to the side;

[0114] Step S35412, determining the edge sub-detection frame corresponding to the side edge according to the side edge and the preset frame ratio, wherein one side of the edge sub-detection frame coincides with the side edge;

[0115] Step S35413, for each edge sub-detection frame, obtaining the size of the sub-point cloud composed of the unit points contained in the edge sub-detection frame;

[0116] Step S35414: taking the ratio of the size of the sub-point cloud to the size of the edge sub-detection frame as the corresponding point cloud ratio within the edge sub-detection frame.

[0117] The horizontal plane is the xy plane in the coordinate system; since the detection frame is a cuboid, the projection of the detection frame on the horizontal plane is a rectangle, and the four sides of the rectangle correspond to the sides of the detection frame, and in the detection frame, the sides are also rectangles.

[0118] The preset border ratio is used to indicate the depth of the edge sub-detection frame; the specific value of the preset border ratio can be set based on actual needs; the preset border ratios corresponding to different sides can be the same or different.

[0119] After the preset border ratio is determined, the height of the edge sub-detection frame is determined. Specifically, the length from the corresponding side to its opposite side is determined, and the product of the length and the preset border ratio is used as the height of the edge sub-detection frame; the rectangle of the side is used as the bottom surface, and the edge sub-detection frame is generated based on the height.

[0120] It can be understood that the edge sub-detection frame is a cuboid located in the detection frame whose bottom surface coincides with the corresponding side edges. Therefore, the edge sub-detection frame includes some unit points in the detection frame.

[0121] Determine the sub-point cloud composed of unit points contained in the edge sub-detection box, and determine the size of the sub-point cloud. The size of the sub-point cloud can be calculated based on the algorithm actually required, such as determining the size of the convex hull corresponding to the sub-point cloud and using it as the size of the sub-point cloud.

[0122] After the sub-point cloud size is determined, the sub-point cloud size / the size of the edge sub-detection box is used as the point cloud ratio.

[0123] Step S3542, determining whether the point cloud ratio is less than a preset point cloud ratio threshold;

[0124] Step S3543: If the point cloud ratio is less than a preset point cloud ratio threshold, the detection frame is determined to be a false detection frame.

[0125] It can be understood that the point cloud ratio indicates the degree of point cloud contained in the edge sub-detection frame; and the edge sub-detection frame is set at the side of the detection frame. Therefore, when the point cloud ratio is larger, it means that the side of the detection frame corresponds to more useful information, and the detection frame can better reflect the complete object; and when the point cloud ratio is smaller, it means that the side of the detection frame corresponds to less useful information, and the detection frame contains more invalid information. Therefore, the detection frame is considered to be a false detection frame.

[0126] Specifically, in this embodiment, a preset point cloud ratio threshold is set; the specific value of the preset point cloud ratio threshold can be set based on actual needs.

[0127] When the point cloud ratio is less than the preset point cloud ratio threshold, the detection box is considered to be a false detection box;

[0128] When the point cloud ratio is greater than or equal to the preset point cloud ratio threshold, the detection box is considered not to be a false detection box.

[0129] In practical applications, the detection box can be comprehensively judged whether it is a false positive box by combining the noise ratio, convex hull ratio and point cloud ratio; for example, first determine whether the detection box is a false positive box based on the noise ratio. When the detection box is determined not to be a false positive box based on the noise ratio, no subsequent judgment is performed; when the detection box is determined to be a false positive box based on the noise ratio, further determine whether the detection box is a false positive box based on the convex hull ratio and the point cloud ratio. When the convex hull ratio and the point cloud ratio both determine that the detection box is a false positive box, the detection box is considered to be a false positive box. When the convex hull ratio or the point cloud ratio confirms that the detection box is not a false positive box, the detection box is considered not to be a false positive box. The specific judgment conditions combining the noise ratio, convex hull ratio and point cloud ratio can also be set based on actual needs. It should be noted that after determining the noise ratio, convex hull ratio and point cloud ratio, when judging whether the detection box is a false positive box, due to the small amount of data for judgment, this part of the calculation can be operated by the CPU.

[0130] In this embodiment, the maximum and minimum values ​​of the unit points contained in the detection frame in the x, y, and z directions are determined, and then an edge sub-detection frame in the shape of a rectangular parallelepiped is formed based on the maximum and minimum values ​​in the x, y, and z directions, that is, the edge sub-detection frame is the smallest rectangular parallelepiped containing all the unit points in the detection frame.

[0131] The object size is the size of the object contained in the detection box determined by the target detection calculation.

[0132] After the edge sub-detection frame is determined, the edge sub-detection frame ratio between the edge sub-detection frame and the object size corresponding to the detection frame is calculated. Specifically, the edge sub-detection frame / object size is used as the edge sub-detection frame ratio.

[0133] It can be understood that the smaller the ratio of the edge sub-detection frame, the greater the deviation between the actual point cloud in the detection frame and the object size determined by the detection frame, the less the detection frame can reflect the specific object, and the more likely it is to be an object frame; conversely, the larger the ratio of the edge sub-detection frame, the smaller the deviation between the actual point cloud in the detection frame and the object size determined by the detection frame, the more the detection frame can reflect the specific object, and the less likely it is to be a false detection frame.

[0134] In this embodiment, a preset edge sub-detection frame ratio threshold is specifically set; when the edge sub-detection frame ratio is less than the preset edge sub-detection frame ratio threshold, the detection frame is determined to be a false detection frame; when the edge sub-detection frame ratio is greater than the preset edge sub-detection frame ratio threshold, the detection frame is determined not to be a false detection frame.

[0135] In this embodiment, the convex hull and the edge sub-detection frame are combined to determine whether the detection frame is a false detection frame, so that the accuracy of determining the false detection frame can be improved based on more marginal information.

[0136] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0138] The present application also provides a detection frame filtering device for implementing the above detection frame filtering method, the detection frame filtering device comprising:

[0139] A first acquisition module is used to acquire a detection frame generated by target detection for the laser point cloud, and determine frame category information corresponding to the detection frame;

[0140] A second acquisition module is used to acquire semantic segmentation information obtained by semantically segmenting the laser point cloud, wherein the semantic segmentation information includes a semantic category corresponding to each unit point in the laser point cloud;

[0141] A first determination module, configured to determine whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point;

[0142] The first deleting module is used to delete the detection frame if the detection frame is a false detection frame.

[0143] The detection frame filtering device evaluates the detection frame generated by target detection by relying on the semantic categories corresponding to each unit point obtained by semantic segmentation, so that the false detection frame in the detection frame can be determined in combination with semantic segmentation, thereby improving the degree of integration between target detection and semantic segmentation. At the same time, by deleting the false detection frame, the detection frame on which the downstream operation depends is more accurate, thereby achieving improved detection accuracy.

[0144] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the second acquisition module in this embodiment can be used to execute step S20 in the embodiment of the present application, the first determination module in this embodiment can be used to execute step S30 in the embodiment of the present application, and the first deletion module in this embodiment can be used to execute step S40 in the embodiment of the present application.

[0145] Furthermore, the first determining module includes:

[0146] A first determination submodule is used to determine, for each detection frame, the semantic category corresponding to the unit point contained in the detection frame;

[0147] A second determination submodule, configured to determine, for each of the unit points, whether the unit point is a noise point according to the frame category information and the semantic category;

[0148] A third determination submodule is used to determine a noise ratio of the number of noise points contained in the detection frame to the number of unit points contained in the detection frame;

[0149] A first judgment submodule, used to judge whether the noise ratio is greater than a preset ratio threshold;

[0150] The fourth determination submodule is used to determine that the detection frame is a false detection frame if the noise ratio is greater than the preset ratio threshold.

[0151] Furthermore, the first determining module further includes:

[0152] A fifth determination submodule, used to determine the number of unit points of the unit points included in the laser point cloud;

[0153] A first generating submodule is used to generate detection threads of the number of the unit points, wherein the detection threads correspond to the unit points one by one;

[0154] The first traversal submodule is used to traverse each of the detection frames in parallel through the detection threads to determine the unit points contained in each of the detection frames.

[0155] Furthermore, the first traversal submodule includes:

[0156] A first determining unit, used to determine the center point coordinates and size information of each detection frame;

[0157] A second determining unit, configured to determine a coordinate range covered by each of the detection frames according to the center point coordinates and the size information;

[0158] A third determining unit, configured to determine, for each of the detection threads, the unit point coordinates of the corresponding unit point;

[0159] A first traversal unit, used for traversing each of the detection frames, and determining whether the coordinate range corresponding to the detection frame contains the unit point coordinates;

[0160] The fourth determining unit is configured to determine that the detection frame contains the unit point if the coordinate range corresponding to the detection frame contains the coordinates of the unit point.

[0161] Furthermore, the fourth determining submodule includes:

[0162] a fifth determining unit, configured to determine a convex hull formed by unit points contained in the false detection frame if the noise ratio is greater than the preset ratio threshold;

[0163] A first acquisition unit, used to acquire the object size corresponding to the detection frame;

[0164] A first calculation unit, configured to calculate a convex hull ratio between the convex hull and the object size, and determine whether the convex hull ratio is less than a preset convex hull ratio threshold;

[0165] The sixth determining unit is configured to determine that the detection frame is a false detection frame if the convex hull ratio is less than a preset convex hull ratio threshold.

[0166] Further, the sixth determining unit includes:

[0167] A first determining subunit, configured to determine a point cloud ratio within an edge sub-detection frame of the detection frame if the convex hull ratio is less than a preset convex hull ratio threshold;

[0168] A first judging subunit, used to judge whether the point cloud ratio is less than a preset point cloud ratio threshold;

[0169] The second determination subunit is used to determine that the detection frame is a false detection frame if the point cloud ratio is less than a preset point cloud ratio threshold.

[0170] Further, the first determining subunit includes:

[0171] For each side of the detection frame on the horizontal plane, determining a preset frame ratio corresponding to the side;

[0172] Determining the edge sub-detection frame corresponding to the side edge according to the side edge and the preset frame ratio, wherein one side of the edge sub-detection frame coincides with the side edge;

[0173] For each edge sub-detection frame, obtaining the size of a sub-point cloud composed of unit points contained in the edge sub-detection frame;

[0174] The ratio of the size of the sub-point cloud to the size of the edge sub-detection frame is used as the corresponding point cloud ratio within the edge sub-detection frame.

[0175] Reference Figure 3 In terms of hardware structure, the electronic device may include components such as a communication module 10, a memory 20, and a processor 30. In the electronic device, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.

[0176] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other electronic devices, servers or Internet of Things devices, such as televisions, etc.

[0177] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as obtaining semantic segmentation information obtained by semantically segmenting the laser point cloud), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0178] The processor 30 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, so as to monitor the electronic device as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.

[0179] although Figure 3 Although not shown, the electronic device may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 3The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0180] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 3 The memory 20 in the electronic device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a television, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0181] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0182] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0183] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A detection frame filtering method, characterized in that: The detection frame filtering method comprises: Obtaining a detection frame generated by target detection for the laser point cloud, and determining frame category information corresponding to the detection frame; Acquire semantic segmentation information obtained by semantically segmenting the laser point cloud, wherein the semantic segmentation information includes a semantic category corresponding to each unit point in the laser point cloud; Determine whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point; If the detection frame is a false detection frame, the detection frame is deleted.

2. The detection frame filtering method according to claim 1, characterized in that: The determining whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point comprises: For each of the detection frames, determining the semantic category corresponding to the unit point contained in the detection frame; For each of the unit points, determining whether the unit point is a noise point according to the frame category information and the semantic category; Determine a noise ratio of the number of noise points contained in the detection frame to the number of the unit points contained in the detection frame; Determining whether the noise ratio is greater than a preset ratio threshold; If the noise ratio is greater than the preset ratio threshold, the detection frame is determined to be a false detection frame.

3. The detection frame filtering method according to claim 2, wherein: The determining, for each of the detection frames, of the semantic category corresponding to the unit point contained in the detection frame includes: Determine the number of unit points of the unit points included in the laser point cloud; Generate detection threads equal to the number of the unit points, wherein the detection threads correspond to the unit points one by one; Each of the detection frames is traversed in parallel by the detection threads to determine the unit points contained in each of the detection frames.

4. The detection frame filtering method according to claim 3, characterized in that: The parallel traversal of each detection frame by the detection thread to determine the unit point contained in each detection frame includes: Determine the center point coordinates and size information of each detection frame; Determine the coordinate range covered by each detection frame according to the center point coordinates and the size information; For each of the detection threads, determining the unit point coordinates of the corresponding unit point; Traversing each of the detection frames, and determining whether the coordinate range corresponding to the detection frame contains the unit point coordinates; If the coordinate range corresponding to the detection frame includes the coordinates of the unit point, it is determined that the detection frame includes the unit point.

5. The detection frame filtering method according to claim 2, wherein: If the noise ratio is greater than the preset ratio threshold, determining that the detection frame is a false detection frame includes: If the noise ratio is greater than the preset ratio threshold, determining a convex hull formed by unit points contained in the false detection frame; Obtaining the object size corresponding to the detection frame; Calculating a convex hull ratio between the convex hull and the object size, and determining whether the convex hull ratio is less than a preset convex hull ratio threshold; If the convex hull ratio is less than a preset convex hull ratio threshold, the detection frame is determined to be a false detection frame.

6. The detection frame filtering method according to claim 5, characterized in that: If the convex hull ratio is less than a preset convex hull ratio threshold, determining that the detection frame is a false detection frame includes: If the convex hull ratio is less than a preset convex hull ratio threshold, determining the point cloud ratio within the edge sub-detection frame of the detection frame; Determine whether the point cloud ratio is less than a preset point cloud ratio threshold; If the point cloud ratio is less than a preset point cloud ratio threshold, the detection frame is determined to be a false detection frame.

7. The detection frame filtering method according to claim 6, characterized in that: Determining the point cloud ratio within the edge sub-detection frame of the detection frame includes: For each side of the detection frame on the horizontal plane, determining a preset frame ratio corresponding to the side; Determining the edge sub-detection frame corresponding to the side edge according to the side edge and the preset frame ratio, wherein one side of the edge sub-detection frame coincides with the side edge; For each edge sub-detection frame, obtaining the size of a sub-point cloud formed by unit points contained in the edge sub-detection frame; The ratio of the size of the sub-point cloud to the size of the edge sub-detection frame is used as the corresponding point cloud ratio within the edge sub-detection frame.

8. A detection frame filtering device, characterized in that: The detection frame filtering device comprises: A first acquisition module is used to acquire a detection frame generated by target detection for the laser point cloud, and determine frame category information corresponding to the detection frame; A second acquisition module is used to acquire semantic segmentation information obtained by semantically segmenting the laser point cloud, wherein the semantic segmentation information includes a semantic category corresponding to each unit point in the laser point cloud; A first determination module, configured to determine whether the detection frame is a false detection frame according to the frame category information and the semantic category corresponding to the unit point; The first deleting module is used to delete the detection frame if the detection frame is a false detection frame.

9. An electronic device, characterized in that: The electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the detection frame filtering method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the detection frame filtering method according to any one of claims 1 to 7 are implemented.

Images