A target screening method and system based on a point cloud processing algorithm in port driving and a medium

Abstract

The application provides a target screening method and system based on a point cloud processing algorithm in port driving, and a medium, comprising determining a target frame to be processed based on point cloud data; projecting the target frame to be processed to a target frame semantic graph to determine a pixel area of the target frame to be processed in the target frame semantic graph; judging whether the target frame to be processed needs to be removed based on semantic values of pixel points in the pixel area; and processing point cloud data corresponding to the target frame to be processed based on a judgment result. The application can be efficient, fast and suitable for complex multi-category scenes, and can comprehensively consider multiple dimensions such as calculation complexity, multi-category processing and target priority processing, and has strong expansibility, simple engineering landing and practicability.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to a target selection method, system, and medium based on point cloud processing algorithm in port operation. Background Technology

[0002] Point cloud data can be used for obstacle identification and target detection. In fields such as autonomous driving or robotics, a core issue is how to perceive surrounding objects. In related technologies, the collected point cloud data can be projected onto a top view, and the bounding box of the top view can be obtained using two-dimensional (2D) detection technology. However, this will result in the loss of the original information of the point cloud during quantization, and it is difficult to detect occluded objects when detecting them from 2D images.

[0003] Non-Maximum Suppression (NMS) is a crucial post-processing technique in object detection, primarily used to select the bounding box that best matches the true location of the target from multiple overlapping candidate boxes output by the model. Its core idea is to improve detection accuracy and efficiency by suppressing redundant detection results and retaining the candidate box with the highest confidence and the best match to the true target.

[0004] In object detection tasks, models typically generate a large number of candidate boxes, which may cover different regions of the same object or contain partial regions of multiple objects. Using these candidate boxes directly without filtering will result in a large number of duplicate or erroneous bounding boxes in the detection results, impacting detection performance. Non-Maximum Suppression (NMS) calculates the overlap between candidate boxes (usually using Intersection over Union (IoU) as a metric) and confidence scores to progressively eliminate redundant boxes, ultimately retaining the optimal detection results.

[0005] Two-dimensional object detection in computer vision is currently the main application area of ​​Natural Search Mechanics (NMS). However, in three-dimensional object detection scenarios such as autonomous driving in ports or from the perspective of BEVs (Battery Electric Vehicles), two-dimensional NMS methods may have performance disadvantages. For example, in scenarios such as autonomous driving in ports, targets are mostly independent individuals, and the targets to be detected are diverse in type, have large size differences, and significant differences in confidence distribution, which leads to low recognition accuracy. Summary of the Invention

[0006] This invention addresses the problems in the prior art by providing a target screening method, system, and medium based on point cloud processing algorithms for port operation. It solves the problem of low recognition accuracy caused by the two-dimensional NMS method in the prior art when there are many types of targets to be detected, large differences in size, and significant differences in confidence distribution.

[0007] The technical solution adopted in this invention is as follows:

[0008] Firstly, this application provides a target selection method based on point cloud processing algorithms for port driving, including:

[0009] Acquire point cloud data in the port environment;

[0010] Based on point cloud data, determine the target bounding box to be processed;

[0011] Project the target box to be processed onto the target box semantic map to determine the pixel region of the target box to be processed in the target box semantic map;

[0012] Based on the semantic values ​​of the pixels within the pixel region, it is determined whether the target box to be processed needs to be removed;

[0013] Based on the judgment result, the point cloud data corresponding to the target box to be processed is processed;

[0014] Cluster analysis is performed on the processed point cloud data to form a valid set of target point clouds;

[0015] Extract features from the point cloud in the effective target point cloud set to generate target feature information;

[0016] Based on target feature information, the categories of valid targets are determined and the valid targets after category determination are output.

[0017] Preferably, determining whether the target box to be processed needs to be removed based on the semantic values ​​of the pixels within the pixel region includes:

[0018] Determine whether there are non-zero pixels in the pixel region;

[0019] Since there are no non-zero pixels in the pixel region, it is determined that the target box to be processed does not need to be removed;

[0020] In response to the presence of non-zero pixels in the pixel region, an overlapping target box that overlaps with the target box to be processed is determined based on the non-zero pixels;

[0021] Based at least on the target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes, it is determined whether the target bounding box to be processed needs to be removed.

[0022] Preferably, if it is determined that the target box to be processed does not need to be removed, the semantic value of the pixel in the pixel area is assigned as the identifier value corresponding to the target box to be processed, and the projection and judgment operation is performed on the next target box to be processed.

[0023] Preferably, determining whether the target box to be processed needs to be removed, based at least on the target categories corresponding to the target box to be processed and the overlapping target boxes, includes:

[0024] Determine the intersection area and the merged area of ​​the target box to be processed and the overlapping target boxes;

[0025] The degree of overlap is determined based on the intersection area and the merged area;

[0026] Based on the target categories corresponding to the target bounding boxes to be processed and the overlapping target bounding boxes, an overlap threshold is determined.

[0027] Based on the overlap degree and the overlap threshold, it is determined whether the target box to be processed needs to be removed.

[0028] Preferably, determining the overlap threshold based on the target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes includes:

[0029] If the target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes are the same, the overlap threshold is set to be less than the baseline threshold.

[0030] If the target categories corresponding to the target boxes to be processed and the overlapping target boxes are different, the overlap threshold is determined based on the degree of proximity of the target categories corresponding to the target boxes to be processed and the overlapping target boxes.

[0031] Preferably, determining whether the target box to be processed needs to be removed based on the overlap degree and the overlap threshold includes:

[0032] In response to the overlap being less than the overlap threshold, it is determined that the target box to be processed does not need to be removed;

[0033] In response to the overlap being not less than the overlap threshold, it is determined that the target box to be processed or the overlapping target box needs to be removed.

[0034] Preferably, if it is determined that the target box to be processed or the overlapping target box needs to be removed, a first confidence level of the target box to be processed and a second confidence level of the overlapping target box are determined based on the target categories corresponding to the target box to be processed and the overlapping target box, respectively.

[0035] Based on the first confidence level and the second confidence level, target boxes that need to be removed are determined from the target boxes to be processed and the overlapping target boxes.

[0036] Preferably, the method further includes:

[0037] In response to the fact that the target box to be removed is the overlapping target box, the coverage area of ​​the overlapping target box on the target box semantic map is obtained;

[0038] The semantic value of the pixels within the coverage area is assigned to 0.

[0039] Secondly, this application provides a point cloud processing system for implementing a target selection method based on a point cloud processing algorithm in port driving as described in the first aspect, comprising:

[0040] The first determination module is configured to determine the target bounding box to be processed based on point cloud data;

[0041] The second determining module is configured to project the target box to be processed onto the target box semantic map and determine the pixel region of the target box to be processed in the target box semantic map;

[0042] The judgment module is configured to determine whether the target box to be processed needs to be removed based on the semantic values ​​of the pixels in the pixel region.

[0043] The processing module is configured to process the point cloud data corresponding to the target box to be processed based on the judgment result.

[0044] Thirdly, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes a target selection method based on a point cloud processing algorithm in port driving as described in the first aspect.

[0045] As can be seen from the above technical solutions, this application has the following advantages:

[0046] 1. By determining the target bounding box to be processed based on point cloud data and projecting it onto the target bounding box semantic map, the pixel region of the target bounding box in the semantic map can be determined quickly and intuitively. Then, based on the semantic value of the pixels in that region, it is intelligently determined whether the target bounding box needs to be removed. This effectively improves the accuracy and efficiency of target bounding box removal, avoids the subsequent processing of invalid data, thereby reducing the overall computational complexity and optimizing the performance of point cloud processing.

[0047] 2. By determining whether there are non-zero pixels within a pixel region, the initial target box removal screening is completed quickly. When there are no non-zero pixels in the region, it can be quickly determined that there is no need to remove the target box, reducing redundant calculations. When there are non-zero pixels, overlapping target boxes are further determined and a precise judgment is made based on the target category, thereby improving the intelligence and accuracy of the target box removal process.

[0048] 3. By assigning the semantic values ​​of pixels within a pixel region to the corresponding identifier values ​​of the target box when it is clear that target boxes do not need to be removed, the recalculation and misjudgment of subsequent target boxes are avoided. This approach not only ensures the accuracy of target box recognition but also greatly improves the continuity of the processing and overall efficiency.

[0049] 4. By dynamically determining the overlap degree based on the intersection area and merge area of ​​the target bounding box and overlapping target bounding boxes, and by setting different overlap degree thresholds according to the target category, the method can more accurately determine whether a target bounding box needs to be removed. This method effectively avoids misjudgments or omissions caused by overlapping target bounding boxes, significantly improving the accuracy and reliability of point cloud data processing results.

[0050] 5. By employing a strategy of setting different overlap thresholds based on the target bounding box category, the overlap threshold is set lower for target bounding boxes of the same category, effectively avoiding redundant identification of the same category; while for target bounding boxes of different categories, the threshold is dynamically determined based on the degree of proximity between the categories. This differentiated processing approach for target categories significantly improves the accuracy and adaptability of the method in complex scenarios, enhancing the application effect.

[0051] 9. Through reasonable coordination among the first determination module, the second determination module, the judgment module, and the processing module, the functions and advantages of the target screening method based on point cloud processing algorithm in port driving are realized. The modular design not only ensures the stability and efficiency of the system implementation, but also effectively improves the application value and scalability of the method, and significantly reduces the computational burden and manual intervention in the data processing process. Attached Figure Description

[0052] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is an exemplary block diagram of a point cloud processing system shown in some embodiments;

[0054] Figure 2 This is an exemplary flowchart of a target selection method based on point cloud processing algorithms in port driving, as shown in some embodiments.

[0055] Figure 3 This is an exemplary flowchart illustrating how to determine whether a target box to be processed needs to be removed, as shown in some embodiments.

[0056] Figure 4 This is an exemplary flowchart illustrating how to determine whether a target bounding box needs to be removed based on overlap and an overlap threshold, as shown in some embodiments. Detailed Implementation

[0057] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this patent, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this patent.

[0058] Existing Non-Maximum Suppression (NMS) methods perform well in 2D image tasks, but they suffer from several limitations in 3D or BEV (Battery, Electric Vehicle) perspective tasks. For example, in 3D and BEV perspective tasks, changes in target occlusion relationships and semantic information loss, coupled with increased target category differences and reduced bounding box overlap, make traditional NMS methods less applicable. Furthermore, in scenarios such as autonomous driving in ports, detecting large machinery like gantry cranes, quay cranes, and forklifts is challenging due to their wide confidence distribution and significant category differences. A low-confidence category A might be more reliable than a high-confidence category B, and since their bounding boxes don't overlap, traditional NMS methods cannot effectively handle this. Additionally, traditional NMS requires sorting and pairwise comparisons of bounding boxes, resulting in high time complexity, large data processing volume, and low data processing efficiency. While some improved NMS methods can address multi-category and size-difference issues, they still suffer from high computational complexity and are unsuitable for real-time scenarios. Therefore, this application provides a target selection method and system based on point cloud processing algorithm in port driving, which can be efficient, fast and applicable to complex multi-category scenarios. It comprehensively considers multiple dimensions such as computational complexity, multi-category processing and target priority processing, and has strong scalability, simple engineering implementation and practicality.

[0059] In some embodiments, such as Figure 1 As shown, the point cloud processing system 100 may include a first determining module 110, a second determining module 120, a judging module 130, and a processing module 140.

[0060] The first determining module 110 is a module used to acquire point cloud data in the port environment and determine the target bounding box to be processed. In some embodiments, the first determining module 110 can be configured to determine the target bounding box to be processed based on the point cloud data.

[0061] The second determining module 120 is a module used to determine the pixel region of the target box to be processed. In some embodiments, the second determining module 120 may be configured to project the target box to be processed onto a target box semantic map and determine the pixel region of the target box to be processed in the target box semantic map.

[0062] The determination module 130 is a module that determines whether the target box to be processed needs to be removed. In some embodiments, the determination module 130 can be configured to determine whether the target box to be processed needs to be removed based on the semantic values ​​of pixels within a pixel region.

[0063] In some embodiments, the determination module 130 may also be configured to: determine whether there are non-zero pixels in the pixel region; determine that the target box to be processed does not need to be removed in response to the absence of non-zero pixels in the pixel region; determine the overlapping target box that overlaps with the target box to be processed based on the non-zero pixels in response to the presence of non-zero pixels in the pixel region; and determine whether the target box to be processed needs to be removed based on the target categories corresponding to the target box to be processed and the overlapping target box, at least based on the target categories corresponding to the target box to be processed and the overlapping target box.

[0064] In some embodiments, the judgment module 130 may also be configured to: if it is determined that the target box to be processed does not need to be removed, assign the semantic value of the pixel in the pixel area to the identifier value corresponding to the target box to be processed, and perform projection and judgment operations on the next target box to be processed.

[0065] In some embodiments, the determination module 130 may also be configured to: determine the intersection area and the merged area of ​​the target box to be processed and the overlapping target box; determine the overlap based on the intersection area and the merged area; determine the overlap threshold based on the target categories corresponding to the target box to be processed and the overlapping target box respectively; and determine whether the target box to be processed needs to be removed based on the overlap and the overlap threshold.

[0066] In some embodiments, the determination module 130 may also be configured to: if the target categories corresponding to the target box to be processed and the overlapping target box are the same, set the overlap threshold to be less than the baseline threshold; if the target categories corresponding to the target box to be processed and the overlapping target box are different, determine the overlap threshold based on the degree of proximity of the target categories corresponding to the target box to be processed and the overlapping target box.

[0067] In some embodiments, the determination module 130 may also be configured to: determine that the target box to be processed does not need to be removed in response to the overlap being less than the overlap threshold; and determine that the target box to be processed or the overlapping target box needs to be removed in response to the overlap being not less than the overlap threshold.

[0068] In some embodiments, the determination module 130 may also be configured to: if it is determined that a target box to be processed or an overlapping target box needs to be removed, determine a first confidence level of the target box to be processed and a second confidence level of the overlapping target box based on the target categories corresponding to the target boxes to be processed and the overlapping target boxes respectively; and determine the target boxes to be removed from the target boxes to be processed and the overlapping target boxes based on the first confidence level and the second confidence level.

[0069] In some embodiments, the determination module 130 may also be configured to: in response to the need to remove target boxes as overlapping target boxes, obtain the coverage area of ​​the overlapping target boxes on the target box semantic map; and assign the semantic value of the pixels within the coverage area to 0.

[0070] The processing module 140 refers to a module used for processing point cloud data. In some embodiments, the processing module 140 can be configured to process the point cloud data corresponding to the target bounding box to be processed based on the judgment result.

[0071] For further explanation of the above, please refer to [link / reference]. Figures 2 to 4 Related descriptions.

[0072] It should be understood that Figure 1 The system and its modules shown can be implemented in various ways. It should be noted that the above description of the point cloud processing system 100 and its modules is for convenience only and should not limit this specification to the scope of the illustrated embodiments. It is understood that those skilled in the art, after understanding the principles of the system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from these principles. In some embodiments, Figure 1 The first determining module 110, the second determining module 120, the judging module 130, and the processing module 140 disclosed herein can be different modules within a single system, or a single module can implement the functions of two or more of the aforementioned modules. For example, the modules can share a single storage module, or each module can have its own separate storage module. Such variations are all within the scope of protection of this specification.

[0073] Figure 2 This is an exemplary flowchart of a target selection method based on point cloud processing algorithms in port driving, as shown in some embodiments of this specification. Figure 2 As shown, process 200 includes the following steps. In some embodiments, process 200 may be executed by point cloud processing system 100.

[0074] Step 210: Determine the target bounding box to be processed based on the point cloud data. In some embodiments, the first determining module 110 may perform step 210.

[0075] For more information about the first determination module 110, please refer to [link / reference]. Figure 1 Related descriptions.

[0076] Point cloud data refers to a collection of multiple point data. Point data refers to data related to point cloud points. In some embodiments, multiple point cloud points can be combined to form a point cloud map, where a point cloud point is the smallest unit that combines to form a point cloud map.

[0077] In some embodiments, point cloud data can be acquired by a detector. The detector may include at least one of a radar detector, a radar camera, a sensor, a laser scanner, etc.

[0078] In some embodiments, the first determining module 110 may include a detector. In some embodiments, the first determining module 110 may be communicatively connected to the detector. The detector may upload point cloud data to the first determining module 110.

[0079] In some embodiments, the point cloud map corresponding to the point cloud data may include multiple frames of point cloud map. Each frame of point cloud map may include at least a portion of the point data from the point cloud data.

[0080] In some embodiments, point cloud data may include data collected by the detector within a unit of time. For example, the unit of time may include 100ms, 200ms, etc. In some embodiments, the detector may be mounted on a vehicle and move synchronously with the vehicle.

[0081] In some embodiments, point data may include at least one of point cloud point location information, color information, target category, intensity information, and time information.

[0082] Location information refers to information related to the position of point cloud points in a coordinate system. For example, the coordinates of the point cloud points. In some embodiments, the coordinate system may include at least one of a two-dimensional coordinate system, a three-dimensional coordinate system, etc. In some embodiments, the coordinate system may include a radar coordinate system, a vehicle coordinate system, a bird's-eye view (BEV) spatial coordinate system (referred to as the BEV view image coordinate system), etc. As an example only, the vehicle coordinate system is a coordinate system with the vehicle's center point as the origin, the vehicle's direction of travel as the x-axis, the direction perpendicular to the vehicle's direction of travel as the y-axis, and the direction perpendicular to the xy-plane as the z-axis.

[0083] Color information refers to information related to the color of a point. Examples include colors like red and blue. In some embodiments, color information can be represented using RGB values. For example, RGB values ​​can include any value from 0 to 255.

[0084] Target category refers to information related to the category of a target object when classifying multiple target objects. A target object is an object that needs to be detected and identified in point cloud data. For example, a target object may include at least one of the following: a forklift in a port, a container truck trailer, pedestrians, and vehicles. In some embodiments, the target category of a target object may include at least one of the following: a stationary target, a non-stationary target, etc. Stationary targets may include at least one of the following: terrain, vegetation, buildings, etc. Non-stationary targets may include at least one of the following: pedestrians, vehicles, etc. For more information on target categories, please refer to the relevant description in step 332.

[0085] Intensity information is information related to the intensity of a point. For example, intensity information may include at least one of the point's pixel value, brightness value, etc.

[0086] Time information refers to information related to the time of point data. For example, the time when the point data was acquired.

[0087] The target boxes to be processed are the target boxes that are waiting to be processed.

[0088] In some embodiments, the first determining module 110 can determine the target bounding box to be processed in various ways. For example, it can be obtained based on a first preset algorithm. In some embodiments, the first preset algorithm may include a 3D target detection algorithm, etc.

[0089] 3D object detection algorithms are algorithms that can determine the bounding box of a target object. Examples include at least one of the PointPillars algorithm and the CenterPoint algorithm.

[0090] In some embodiments, the input to the 3D object detection algorithm may include point cloud data, and the output may include the bounding box to be processed and the bounding box information of the bounding box to be processed.

[0091] Target bounding box information refers to information related to the target bounding box to be processed.

[0092] In some embodiments, the target bounding box information may include at least one of the following: center point, size, heading, target category, confidence level, etc.

[0093] The center point refers to the geometric center of the target bounding box to be processed. In some embodiments, the center point can be characterized based on its location information, such as the coordinates of the center point in the vehicle coordinate system.

[0094] Size refers to information related to the dimensions of the target bounding box. For example, when the target bounding box is a rectangle, the size can include the length and width of the target bounding box.

[0095] Heading refers to information related to the direction of vehicle travel. In some embodiments, heading can be represented by a heading angle. The heading angle is the angular difference between the vehicle's direction of travel and the vehicle's longitudinal axis.

[0096] The target category of the bounding box refers to the type or category of the target object within the bounding box. For example, the target category can be at least one of vehicles, pedestrians, or goods.

[0097] Confidence level refers to the degree of certainty regarding the determination of a target bounding box to be processed. Confidence level can be represented by a probability value.

[0098] In some embodiments, the first determining module 110 can classify multiple target boxes to be processed based on their target categories. The first determining module 110 can sort the multiple target boxes to be processed for each target category based on their confidence levels, and subsequently process the target boxes to be processed sequentially according to the sorted order.

[0099] In some implementations of this specification, the target bounding box information is helpful for obtaining relevant information about the target bounding box, such as the target bounding box to be processed, and facilitates subsequent determination of whether the target bounding box needs to be processed (such as being removed or retained).

[0100] In some embodiments, the target boxes to be processed as determined above may include multiple redundant boxes.

[0101] Redundant boxes refer to target boxes that do not meet the requirements. For example, the target box information of redundant boxes does not meet the requirements. In some embodiments, the first determining module 110 can perform preliminary screening on the target boxes to be processed determined to determine the target boxes to be processed after removing redundant boxes.

[0102] In some embodiments, the first determining module 110 can filter target boxes that meet preset filtering conditions to achieve the effect of removing redundant boxes.

[0103] Preset filtering criteria refer to preset conditions used to remove redundant boxes from the target boxes to be processed. In some embodiments, preset filtering criteria may be related to confidence levels. For example, preset filtering criteria may be to remove target boxes to be processed with confidence levels less than a confidence threshold.

[0104] In some embodiments, the first determining module 110 may obtain the confidence threshold in a variety of ways. For example, at least one of obtaining human input, obtaining it from historical data, etc.

[0105] In some embodiments, the confidence threshold may be related to the target category. The confidence thresholds for different target categories may be the same or different. For example, the confidence threshold for a non-stationary target may be lower than the confidence threshold for a stationary target.

[0106] In some embodiments, the first determining module 110 may construct a first preset table based on historical target categories and historical confidence thresholds in historical data. The first preset table may include historical target categories, historical confidence thresholds, and the correspondence between historical target categories and historical confidence thresholds. In some embodiments, the first determining module 110 may query the first preset table based on the current target category to determine the same or similar historical target categories, and use the historical confidence threshold corresponding to the historical target category as the confidence threshold corresponding to the current target category.

[0107] In some embodiments, the first determining module 110 may obtain preset filtering conditions in a variety of ways. For example, at least one of obtaining manual input, obtaining from historical data, etc.

[0108] Processing each frame of point cloud image can generate tens of thousands or even hundreds of thousands of bounding boxes, including a large number of low-confidence redundant boxes. Processing all bounding boxes using Non-Maximum Suppression (NMS) would be computationally intensive and time-consuming. By identifying candidate bounding boxes and then determining the boxes to be processed from these candidates, a preliminary screening of the candidate bounding boxes can be performed, reducing the number of boxes to be processed and thus decreasing the amount of data processing required.

[0109] Using a confidence threshold as a filtering criterion can remove candidate bounding boxes with confidence scores below the threshold. Depending on different situations or needs, such as different application scenarios, different network performance, and different threshold selections, the number of candidate bounding boxes can be reduced to tens, hundreds, or thousands, thereby reducing the amount of subsequent data processing and improving data processing efficiency and accuracy.

[0110] Step 220: Project the target box to be processed onto the target box semantic map to determine the pixel region of the target box in the target box semantic map. In some embodiments, the second determining module 120 may perform step 220.

[0111] For more information about the second determining module 120, please refer to [link / reference]. Figure 1 Related descriptions.

[0112] A target bounding box semantic graph is a graph used to assign semantic labels to pixels.

[0113] In some embodiments, the initial target bounding box semantic map can be a blank image, including multiple pixels with a pixel value of 0.

[0114] In some embodiments, the target bounding box semantic map can be an image in an image coordinate system from the BEV perspective. In some embodiments, the origin of the image coordinate system from the BEV perspective can be the same as the origin of the vehicle coordinate system. For example, the origin can be the center point of the vehicle, etc.

[0115] In some embodiments, the resolution of the target bounding box semantic map can be set to R. That is, the resolution of each pixel in the target bounding box semantic map in the image coordinate system under the BEV viewpoint is R meters × R meters. In some embodiments, the resolution is a preset value. The second determining module 120 can determine the resolution in a variety of ways. For example, at least one of obtaining manual input, obtaining from historical data, etc.

[0116] In some embodiments, the second determining module 120 can set the range to be investigated from the BEV's perspective. The range to be investigated can be determined in various ways, such as based on the detection range of the detector acquiring point cloud data. For example, taking a radar detector as an example, the forward range to be investigated FR is 100m, and the backward range to be investigated BR, the leftward range to be investigated LR, and the rightward range to be investigated RR are each 50m. FR, BR, LR, and RR refer to the distances forward, backward, leftward, and rightward, respectively, with the center of the vehicle as the origin of the coordinate system and the vehicle's driving direction or the direction the front of the vehicle is facing.

[0117] Semantic identifiers are identifiers assigned to target boxes to be processed. Semantic identifiers can be used to distinguish information about different target boxes to be processed. For example, if there are i target boxes to be processed, the second determining module 120 can use i as the semantic identifier of the i-th target box to be processed.

[0118] In some embodiments, the semantic identifier may be associated with the target category corresponding to the target bounding box to be processed. Different target categories may correspond to different semantic identifiers.

[0119] A pixel region refers to the area occupied by at least one pixel in the semantic map of the target bounding box in the image coordinate system under the BEV viewpoint. In some embodiments, the pixel region may include the area occupied by the corresponding pixel in the semantic map of the target bounding box in the image coordinate system under the BEV viewpoint after the target bounding box to be processed has been projected.

[0120] In some embodiments, the second determining module 120 can project the target box to be processed onto the target box semantic map, and use the area corresponding to the projection as the pixel area corresponding to the target box to be processed.

[0121] Projection refers to converting the coordinates of the target bounding box in the vehicle coordinate system into coordinates in the semantic map of the target bounding box in the image coordinate system from the BEV perspective. For example, it involves converting the spatial coordinates of any point in the vehicle coordinate system... Converted to image coordinates in the semantic map of the target bounding box in the BEV perspective image coordinate system. .

[0122] In some embodiments, the second determining module 120 may be based on spatial coordinates. The image coordinates are calculated using the second preset algorithm. The second preset algorithm may include the following formula:

[0123]

[0124] Where R is the resolution of the target bounding box semantic map. In some embodiments, the second determining module 120 can determine the resolution in various ways. For example, at least one of obtaining human input, obtaining from historical data, or obtaining based on experience. For more information on LR and BR, please refer to the relevant descriptions above.

[0125] In some embodiments, taking a rectangular target box as an example, the second determining module 120 can obtain the four target box vertices in the target box semantic map of the rectangular target box in the image coordinate system under the BEV perspective.

[0126] In some embodiments, the second determining module 120 can determine the vertices of the target bounding box in various ways. For example, the second determining module 120 can obtain the vertex matrix formed by the vertices of the target bounding box to be processed under non-rotational translation:

[0127]

[0128] Where L is the length of the target bounding box to be processed, and W is the width of the target bounding box to be processed.

[0129] The second determining module 120 can left-multiply the vertex matrix and the rotation matrix to rotate the target bounding box to be processed. Then, the rotated target bounding box is translated into the target bounding box semantic map in the image coordinate system under the BEV viewpoint. In some embodiments, the rotation matrix may include:

[0130]

[0131] in, This refers to the heading angle. For more information on the heading angle, please refer to the description in step 210. For more information on the image coordinate system, projection, and pixel region from the BEV perspective, please refer to the previous descriptions.

[0132] In some embodiments, the second determining module 120 can determine the target box coverage area obtained from the above four target box vertices as the pixel area of ​​the target box to be processed in the target box semantic map.

[0133] Step 230: Based on the semantic values ​​of pixels within the pixel region, determine whether the target box to be processed needs to be removed. In some embodiments, the determination module 130 may execute step 230.

[0134] For more information about the judgment module 130, please refer to [link / reference]. Figure 1 Related descriptions.

[0135] Semantic values ​​refer to the meaning assigned to a pixel. In some embodiments, semantic values ​​can correspond to semantic identifiers. For example, after the second determining module 120 projects the i-th target box to be processed onto the target box semantic map, it determines the pixel region corresponding to the i-th target box to be processed. The second determining module 120 can assign the semantic value of the pixels in the pixel region to i, indicating that the pixels in the pixel region are occupied by the i-th target box to be processed, and the pixel region can be used as the i-th pixel region. This forms a correspondence between the target box semantic map and the target boxes to be processed.

[0136] In some embodiments, the determination module 130 determines whether the target box to be processed needs to be removed based on the semantic values ​​of pixels within a pixel region using various methods. For example, the determination module 130 can determine whether there are any non-zero pixels in the semantic values ​​of pixels within the pixel region. If no non-zero pixels exist in the pixel region, the determination module 130 can determine that the target box corresponding to that pixel region does not need to be removed. If non-zero pixels exist in the pixel region, the determination module 130 can determine overlapping target boxes that overlap with the target box to be processed based on the non-zero pixels; and at least based on the target categories corresponding to the target box to be processed and the overlapping target boxes, determine whether the target box to be processed needs to be removed. For more details, please refer to [link to relevant documentation]. Figure 3 and / or Figure 4 Related descriptions.

[0137] Step 240: Based on the judgment result, process the point cloud data corresponding to the target bounding box to be processed. In some embodiments, the processing module 140 may execute step 240.

[0138] For more information about processing module 140, please refer to [link / reference]. Figure 1 Related descriptions.

[0139] The judgment result refers to whether the target box to be processed determined by the judgment module 130 needs to be removed.

[0140] In some embodiments, the judgment module 130 may transmit the judgment result to the processing module 140.

[0141] In some embodiments, the processing module 140 can process target boxes that do not need to be removed and / or target boxes that need to be removed based on the judgment result. For example, if the first target box does not need to be removed, the processing module 140 can draw the first target box in the corresponding pixel area of ​​the target box semantic map. That is, the processing module 140 can retain the first target box in the corresponding pixel area of ​​the target box semantic map. Retention may include retaining the semantic value corresponding to each pixel in the pixel area. As another example, if the second target box needs to be removed, the processing module 140 can determine and record the semantic identifier 2 of the second target box and continue to perform the operation of judging whether the next target box needs to be removed.

[0142] In some embodiments, the point cloud processing system can traverse multiple target boxes to be processed once until all target boxes to be processed have been judged by the judgment module 130 to determine whether they need to be removed.

[0143] The target filtering method based on point cloud processing algorithm in port driving provided in some embodiments of this specification determines whether the target box to be processed needs to be removed based on semantic value. When making the determination, it is only necessary to traverse all the target boxes to be processed once, avoiding pairwise comparison of the target boxes to be processed, improving data processing efficiency, effectively shortening the data processing time, and thus improving the efficiency and quality of detection work.

[0144] The time complexity of common sorting algorithms is O(NlogN), and pairwise comparisons are required to remove duplicates after sorting. The target selection method based on point cloud processing algorithms for port driving provided in some embodiments of this specification uses NMS to determine whether there are target boxes that need to be removed. It does not require sorting the target boxes according to confidence levels and only requires one traversal, resulting in a time complexity of O(N), which is relatively low.

[0145] Figure 3 This is an exemplary flowchart illustrating how to determine whether a target box to be processed needs to be removed, based on some embodiments of this specification. Figure 3 As shown, process 300 includes the following steps. In some embodiments, process 300 may be executed by the determination module 130.

[0146] Step 310: Determine if there are any non-zero pixels in the pixel region. For more information on pixel regions, see [link to relevant documentation]. Figure 2 Related descriptions.

[0147] A non-zero pixel is a pixel within a pixel region whose pixel value is not 0. In some embodiments, a non-zero pixel may be a pixel within the pixel region containing the target object or any other meaningful information.

[0148] In some embodiments, the determination module 130 can determine whether there are non-zero pixels in the pixel region in multiple ways. In some embodiments, the determination module 130 can obtain a set of pixels on the semantic map of the target box within the coverage area of ​​the target box under the BEV viewpoint image coordinate system, based on the four vertices of the target box to be processed. The determination module 130 can examine each pixel in the set of pixels to determine whether there are non-zero pixels in the pixel region.

[0149] Step 321: In response to the absence of non-zero pixels in the pixel region, it is determined that the target box to be processed does not need to be removed.

[0150] For more information on the target bounding box to be processed, please refer to [link / reference]. Figure 2 Related explanations.

[0151] In some embodiments, the target selection method based on point cloud processing algorithm in port driving further includes, if it is determined that the target box to be processed does not need to be removed, the judgment module 130 can assign the semantic value of the pixel in the pixel area to the identifier value corresponding to the target box to be processed, and perform projection and judgment operations on the next target box to be processed.

[0152] For more information on pixels, semantic values, and projection, please refer to [link / reference]. Figure 2 Related explanations.

[0153] An identifier is a label used to distinguish and mark target box information. An identifier value is a data value associated with an identifier. In some embodiments, the identifier value can be numbers, letters, etc.

[0154] Assignment refers to assigning an identifier value to a pixel within a pixel region. In some embodiments, the numerical value (i) is used as the semantic value of the target box to be processed. If it is determined that the target box to be processed does not need to be removed, the determination module 130 can assign the semantic value of the pixel within the pixel region to the identifier value (i), indicating that the pixel region is occupied by the target box i to be processed.

[0155] The judgment operation refers to determining whether a bounding box to be processed needs to be removed based on the semantic values ​​of pixels within a pixel region. For a detailed explanation of the judgment operation, please refer to [link to relevant documentation]. Figure 2 Explanation of step 230.

[0156] Some embodiments in this specification, by identifying the processed pixel regions, can avoid repeatedly judging and calculating the same region in subsequent processing, saving computing resources and improving processing efficiency.

[0157] Step 322: In response to the presence of non-zero pixels in the pixel region, determine the overlapping target box that overlaps with the target box to be processed based on the non-zero pixels.

[0158] An overlapping target box refers to a target box within a pixel region that overlaps with the target box to be processed. In some embodiments, the number of overlapping target boxes can be one or more.

[0159] In some embodiments, the determination module 130 may determine the overlapping target box in a variety of ways based on non-zero pixel points.

[0160] For example, in some embodiments, the judgment module 130 can traverse the pixel area covered by the target box to be processed, count the semantic values ​​of all non-zero pixels and their corresponding pixel counts, and form a statistical result.

[0161] Here, pixel count refers to the number of pixels with the same semantic value. The statistical result can be a tuple {(j1,k1),(j2,k2),...,(jM,kM)}, where j1, j2, ..., jM ∈ {1, 2, ..., N}. Here, j1, j2, ..., jM represent the semantic values ​​of different non-zero pixels, and kM is the pixel count, representing the number of non-zero pixels with a semantic value of jM among all non-zero pixels within the pixel region.

[0162] In some embodiments, when the number of elements in the tuple corresponding to the statistical result is 0, it means that there is no overlapping target box that overlaps with the target box to be processed within the pixel area.

[0163] In some embodiments, M can be 1, indicating that within the pixel area covered by the target box to be processed, there is only one non-zero pixel with a corresponding semantic value j1, and the corresponding pixel count is k1, that is, there is only one overlapping target box in the target box to be processed.

[0164] In some embodiments, M can also be any other value N, indicating that within the pixel area covered by the target box to be processed, there are N non-zero pixels with corresponding semantic values ​​j1, j2...jN, and the corresponding pixel counts are k1, k2...kN, respectively. That is, there are N overlapping target boxes in the target box to be processed.

[0165] In some embodiments, the determination module 130 can determine the overlapping target boxes and their number sequentially based on the semantic values ​​j1, j2...jM in the statistical results. For example, assuming the statistical results are {(1,5),(2,6),(3,7),(4,8)}, it can be determined that there are 4 types of non-zero pixels in the target processing box, corresponding to 4 overlapping target boxes; through these 4 types of non-zero pixels with semantic values ​​of 1, 2, 3, and 4, the 4 overlapping target boxes in the target processing box can be determined.

[0166] Step 332: Determine whether the target box to be processed needs to be removed, based at least on the target categories corresponding to the target boxes to be processed and the overlapping target boxes.

[0167] The target category refers to the type or category to which the target object in the pixel region belongs. For example, the target category could be vehicle, pedestrian, or cargo.

[0168] In some embodiments, the determination module 130 can obtain the target category based on multiple methods. For example, the determination module 130 can query the corresponding target box information in the target box sequence by using the semantic values ​​corresponding to the target box to be processed and the overlapping target boxes, and obtain the target category corresponding to the target box to be processed and the overlapping target boxes, respectively, through the target box information.

[0169] Here, the target box sequence refers to a pre-defined sequence composed of multiple target boxes. Target box information can include the target category of the target box. The target category of a target box refers to the category information of the target object represented by each target box. For example, if the target category of target box 1 is "vehicle," then all target objects represented by target box 1 under this target category are vehicles. Target box 1 under this target category could represent the 11th, 13th, 15th, etc.

[0170] For more information about the target bounding box and target bounding box information, please refer to [link / reference]. Figure 2 Related explanations.

[0171] In some embodiments, the determination module 130 may determine whether the target box to be processed needs to be removed in a variety of ways, based at least on the target categories corresponding to the target box to be processed and the overlapping target boxes respectively.

[0172] In some embodiments, the determination module 130 may determine the intersection area and the merged area of ​​the target box to be processed and the overlapping target box; determine the overlap based on the intersection area and the merged area; determine the overlap threshold based on the target categories corresponding to the target box to be processed and the overlapping target box; and determine whether the target box to be processed needs to be removed based on the overlap and the overlap threshold.

[0173] For a detailed process on how to determine whether a target bounding box needs to be removed, please refer to [link / reference]. Figure 4 Related explanations.

[0174] In some embodiments of this specification, determining whether to remove a target box by judging the presence of non-zero pixels in the pixel region can further improve the accuracy of the determination of whether a target box needs to be removed, thus optimizing the system's detection capability. Furthermore, by deciding whether to remove a target box according to predefined steps, the execution logic of each step can be clearly defined, simplifying the processing flow of complex tasks and improving the overall efficiency and maintainability of the system.

[0175] Figure 4 This is an exemplary flowchart illustrating, according to some embodiments of this specification, a method for determining whether a target bounding box needs to be removed based on overlap and an overlap threshold. For example... Figure 4 As shown, process 400 includes the following steps. In some embodiments, process 400 may be executed by the determination module 130.

[0176] Step 410: Determine the intersection area and merged area of ​​the target box to be processed and the overlapping target boxes.

[0177] For more information on the target bounding boxes to be processed, please refer to the description in step 210. For more information on overlapping target bounding boxes, please refer to the description in step 322.

[0178] The intersection area refers to the area of ​​the overlapping portion of two planar figures. For example, the area of ​​the overlapping portion between the target bounding box and the overlapping target bounding box.

[0179] In some embodiments, the determination module 130 can determine a first pixel count of the pixels corresponding to the overlapping portions of the target box to be processed and the overlapping target box. The determination module 130 can calculate a first product of the first pixel count and the resolution corresponding to the pixel, and use the first product as the intersection area.

[0180] The merged area refers to the sum of the areas of the overlapping and non-overlapping portions of the target box to be processed and the overlapping target boxes.

[0181] In some embodiments, the determination module 130 may count a second pixel count to determine the pixels corresponding to the non-overlapping portions of the target box to be processed and the overlapping target boxes. The determination module 130 may calculate a second product of the second pixel count and the resolution corresponding to the pixels, and use the sum of the second product and the intersection area as the merged area.

[0182] For more information on pixel counting, please refer to the description in step 322. For more information on resolution, please refer to the description in step 220.

[0183] Step 420: Determine the degree of overlap based on the intersection area and the merged area.

[0184] Overlap refers to the degree to which the target bounding box being processed overlaps with overlapping target bounding boxes. In some embodiments, overlap can be expressed as the Intersection over Union (IoU).

[0185] In some embodiments, the determination module 130 can calculate the ratio of the intersection area to the merged area, and use this ratio as the overlap degree. In some embodiments, the same target box to be processed can correspond to multiple different overlapping target boxes. The determination module 130 can calculate the intersection area between the multiple overlapping target boxes and the target box to be processed, and calculate the merged area between the multiple overlapping target boxes and the target box to be processed, respectively. The determination module 130 can calculate the ratio of the intersection area to the corresponding merged area, thereby determining the overlap degree corresponding to the multiple overlapping target boxes.

[0186] Step 430: Determine the overlap threshold based on the target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes, respectively.

[0187] For more information on target categories, please refer to the description in step 332.

[0188] The overlap threshold is a preset value used for comparison with overlap.

[0189] In some embodiments, the determination module 130 can determine the overlap threshold in various ways. For example, the determination module 130 can construct a second preset table based on the historical target categories corresponding to the target boxes to be processed and the overlapping target boxes in historical data, and the corresponding historical overlap thresholds. The second preset table may include historical target categories, historical overlap thresholds, and their correspondences. Based on the current target category, the determination module 130 can determine the same or similar historical target categories by querying the second preset table, and determine the historical overlap threshold corresponding to the historical target category as the current overlap threshold.

[0190] In some embodiments, the determination module 130 may also obtain the overlap threshold through other means. For example, at least one of obtaining manual input, obtaining it from historical data, etc.

[0191] In some embodiments, the target selection method based on point cloud processing algorithm in port driving further includes: if the target categories corresponding to the target box to be processed and the overlapping target box are the same, the judgment module 130 can set the overlap threshold to be less than a baseline threshold; if the target categories corresponding to the target box to be processed and the overlapping target box are different, the judgment module 130 can determine the overlap threshold based on the degree of proximity of the target categories corresponding to the target box to be processed and the overlapping target box.

[0192] The baseline threshold is a fixed parameter value used to determine the overlap threshold between the target bounding box and the overlapping target bounding boxes. For example, the baseline threshold can be set to 0.5.

[0193] When the target categories corresponding to the target boxes A and B are the same, the overlap threshold between the target boxes A and B is less than the baseline threshold of 0.5. For example, the overlap threshold is 0.4, 0.3, etc.

[0194] In some embodiments, the benchmark threshold may be determined based on historical experience.

[0195] Adjacency refers to the degree of similarity or association between the two target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes.

[0196] In some embodiments, the degree of proximity can be quantified by the semantic or hierarchical relationship between target categories. For example, two target categories are considered to have a high degree of proximity if they belong to the same semantic group or category set, and vice versa.

[0197] In some embodiments, the degree of proximity can be determined in a variety of ways. For example, it can be determined by comparing the semantic similarity between target categories.

[0198] In some embodiments, the determination module 130 may determine the overlap threshold by querying a third preset table based on the degree of proximity of the target categories corresponding to the target boxes to be processed and the overlapping target boxes.

[0199] The third preset table refers to the correspondence table between the degree of adjacency and the threshold of overlap. The third preset table can be determined based on manual presets, etc.

[0200] In some embodiments, the overlap threshold can be positively correlated with the degree of proximity. For example, if the locking station and locking frame are usually placed close together, indicating a high degree of proximity, then the overlap threshold can be set to a higher value. As another example, if the truck cab and trailer have a large overlapping area in their structure, indicating a high degree of proximity, then the overlap threshold can be set to the highest value.

[0201] In some embodiments of this specification, the determination of the overlap threshold depends on the category relationship and proximity between the target boxes. In this way, appropriate overlap thresholds can be set for different target boxes to be processed and overlapping target boxes, which helps to further improve the accuracy of determining whether the target boxes to be processed need to be removed, and further improve the efficiency and quality of the detection work.

[0202] Step 440: Based on the overlap and the overlap threshold, determine whether the target box to be processed needs to be removed.

[0203] In some embodiments, the determination module 130 can compare the overlap degree with an overlap degree threshold to determine whether the overlapping target boxes need to be removed. For example, if the overlap degree is less than the overlap degree threshold, the determination module 130 can determine that the corresponding overlapping target boxes do not need to be removed.

[0204] In some embodiments, the determination module 130 may determine that the target box to be processed does not need to be removed in response to the overlap degree being less than the overlap degree threshold; the determination module 130 may determine that the target box to be processed or the overlapping target box needs to be removed in response to the overlap degree being not less than the overlap degree threshold.

[0205] In some embodiments, in response to an overlap degree not less than an overlap degree threshold, the determination module 130 may determine in a variety of ways that the target box to be processed or the overlapping target box needs to be removed.

[0206] For example, when it is determined that the target box to be processed or the overlapping target box needs to be removed, the determination module 130 can remove the target box to be processed by default.

[0207] In some embodiments, the target screening method based on point cloud processing algorithm in port driving further includes, if the judgment module 130 determines that the target box to be processed or the overlapping target box needs to be removed, the judgment module 130 can determine the first confidence level of the target box to be processed and the second confidence level of the overlapping target box based on the target categories corresponding to the target box to be processed and the overlapping target box respectively.

[0208] In some embodiments, the determination module 130 may determine the target boxes to be removed from the target boxes to be processed and the overlapping target boxes based on a first confidence level and a second confidence level.

[0209] The first confidence level refers to the confidence level corresponding to the target bounding box to be processed.

[0210] The second confidence level refers to the confidence level corresponding to the overlapping target boxes.

[0211] For more information on confidence levels, please refer to the relevant description in step 210.

[0212] The target boxes to be removed are the target boxes to be processed and the target boxes in the overlapping target boxes that need to be removed.

[0213] In some embodiments, the determination module 130 may determine at least one of the target boxes to be processed or overlapping target boxes as target boxes that need to be removed based on the magnitude of a first confidence level and a second confidence level. For example, when the first confidence level is greater than the second confidence level, the determination module 130 may determine to remove overlapping target boxes. When the first confidence level is less than the second confidence level, the determination module 130 may determine to remove the target box to be processed.

[0214] In some embodiments, the determination module 130 may determine a first priority score for the target box to be processed and a second priority score for the overlapping target boxes based on a first confidence level and a second confidence level, and based on the target categories corresponding to the target boxes to be processed and the overlapping target boxes, respectively. The determination module 130 may determine the target boxes with lower priority scores as those that need to be removed.

[0215] The first priority score is a score that evaluates the priority of the target bounding box to be processed.

[0216] The second priority score is a score that evaluates the priority of overlapping bounding boxes.

[0217] In some embodiments, different target categories and different confidence levels characterize different levels of reliability for the target boxes or overlapping target boxes to be processed. The determination module 130 may pre-set a fourth preset table. The fourth preset table may include preset levels of reliability for different target categories at different confidence levels.

[0218] In some embodiments, the judgment module 130 may obtain the reliability of the target box to be processed and the overlapping target box respectively by querying a fourth preset table based on the first confidence level and the second confidence level, as well as the target categories corresponding to the target box to be processed and the overlapping target box respectively.

[0219] In some embodiments, the judgment module 130 can determine a priority score by performing a weighted summation based on confidence level and reliability level. For example, for a target box to be processed, the judgment module 130 can determine a first priority score based on the weighted value of a first confidence level of the target box to be processed and the reliability level corresponding to that first confidence level. As another example, for overlapping target boxes, the judgment module 130 can determine a second priority score based on the weighted value of a second confidence level of the overlapping target boxes and the reliability level corresponding to that second confidence level. In some embodiments, the weights corresponding to the first confidence level, the second confidence level, and the reliability levels of the target box to be processed and the overlapping target boxes can be preset values. The judgment module 130 can determine the weights in various ways, such as obtaining manual input or obtaining them from historical data.

[0220] In some embodiments, the judgment module 130 can determine which target boxes to be processed or overlapping target boxes to be removed based on the magnitude of a first priority score and a second priority score. For example, when the first confidence level is different from the second confidence level, the judgment module 130 can select the target box with the lower confidence level among the target boxes to be processed and the overlapping target boxes as the target boxes to be removed. As another example, when the first confidence level is the same as the second confidence level, if the first priority score is different from the second priority score, the judgment module 130 can select the target box with the lower priority score among the target boxes to be processed and the overlapping target boxes as the target boxes to be removed. For instance, target boxes to be processed and overlapping target boxes of different target categories may have the same confidence level, but due to the different target categories, the reliability levels corresponding to the target boxes to be processed and the overlapping target boxes are different, and the corresponding first priority scores and second priority scores are different. The confidence level of common obstacle truck trailers and rare and varied forklifts is 0.4. Since the first priority score corresponding to the forklift is greater than the second priority score corresponding to the truck trailer, the judgment module 130 can prioritize removing the target boxes corresponding to the truck trailers when determining which target boxes to remove. For example, regardless of whether the first confidence level and the second confidence level are the same, the judgment module can directly identify the target box to be processed and the overlapping target boxes with the lower priority score as the target box to be removed.

[0221] In some embodiments of this specification, a first priority score and a second priority score can be determined based on the confidence level of the target box to be processed and the overlapping target boxes, and the target category. By comparing the magnitude of the first priority score and the second priority score, target boxes that need to be removed are determined. When determining target boxes that need to be removed, both confidence level and target category can be considered simultaneously, which can improve the accuracy of determining target boxes that need to be removed.

[0222] In some embodiments of this specification, target boxes of different target categories or overlapping target boxes may have the same or different confidence levels. When determining which target boxes need to be removed, considering the confidence level can improve the accuracy of determining which target boxes need to be removed.

[0223] Some embodiments in this specification can effectively determine whether a target box should be removed by comparing the overlap between target boxes with a preset overlap threshold, thereby reducing the existence of overlapping and redundant target boxes and optimizing the accuracy and precision of target detection.

[0224] In some embodiments, in response to the need to remove overlapping target boxes, the determination module 130 can obtain the coverage area of ​​the overlapping target boxes on the target box semantic map and assign the semantic value of the pixels within the coverage area to 0.

[0225] For more information on the semantic graph of the target bounding box, please refer to the relevant description in step 220.

[0226] Coverage refers to the area on the semantic map corresponding to overlapping target boxes. For more information on overlapping target boxes, please refer to the relevant description in step 322.

[0227] In some embodiments, the determination module 130 may use the area formed by the pixels corresponding to the overlapping target boxes as the coverage area.

[0228] In some embodiments, the determination module 130 can assign the semantic value of pixels within the coverage area to 0 in various ways. For example, when the target box to be removed is the j1th overlapping target box, the determination module 130 can determine the semantic identifier j1 of the j1th overlapping target box. The determination module 130 can determine the coverage area based on the vertices of the j1th overlapping target box and assign the semantic value of pixels within the coverage area to 0 on the target box semantic map.

[0229] In some embodiments, the determination module 130 may perform a determination on whether the j1-jM overlapping target boxes need to be removed. That is, the i-th pixel region is compared with the j1-jM pixel regions sequentially until it is determined that neither the i-th target box to be processed nor the jM overlapping target boxes need to be removed, or the i-th target box to be processed is removed. Then, the determination on whether the (i+1)-th target box to be processed needs to be removed is performed.

[0230] For more information on determining whether to remove target boxes, please refer to step 230. Figure 3 , Figure 4 For more information on pixel regions, please refer to the description in step 220.

[0231] By setting the semantic values ​​of pixels within the coverage area of ​​the overlapping target bounding box on the target bounding box semantic map to 0, non-zero pixels within the coverage area on the target bounding box semantic map can be erased, which helps reduce the amount of subsequent data processing.

[0232] The method for determining whether a target bounding box needs to be removed, provided in some embodiments of this specification, calculates the overlap by determining the intersection area and merge area of ​​the target bounding box and overlapping target bounding boxes. Based on the overlap, it determines whether the target bounding box or overlapping target bounding boxes need to be removed. This method does not rely on confidence scores for judgment and can take into account different overlap situations due to different shapes of different target objects, as well as the priority and detection level of target objects. This makes the judgment more flexible and accurate, more suitable for deduplication tasks of target bounding boxes of multiple target categories, and can also provide more accurate post-processing results.

[0233] In some embodiments, the target filtering method based on point cloud processing algorithms in port operation further includes, in response to the fact that the target box to be processed is the last target box to be processed, the judgment module 130 can obtain the target boxes to be retained based on the list to be deleted. The judgment module 130 can, for the pixels in the target box semantic map converted from point cloud data, traverse the semantic value of each pixel to determine the target boxes to be retained, and obtain the correspondence between each pixel and the target boxes to be retained.

[0234] The last pending target box refers to the last pending target box in the sequence. For example, if there are N pending target boxes, they are ordered from 1 to N and processed sequentially, with the Nth pending target box being the last pending target box.

[0235] The list to be deleted is a list consisting of at least one target box that needs to be removed.

[0236] In some embodiments, the list to be deleted may include a sequence of at least one target bounding box that needs to be removed. In some embodiments, the judgment module 130 may determine the target bounding boxes that need to be removed based on the judgment result, and then generate the list to be deleted. In some embodiments, the list to be deleted may include a sequence of semantic identifiers of the target bounding boxes to be removed.

[0237] For more information on the target bounding boxes that need to be removed, please refer to the relevant description in step 440.

[0238] The retained target boxes refer to the target boxes that the judgment module 130 determines do not need to be removed.

[0239] In some embodiments, the determination module 130 may determine the target boxes to be retained based on the target boxes to be removed. For example, the target boxes to be retained may include the remaining target boxes after removing the target boxes to be removed from the list of target boxes to be processed. In some embodiments, the determination module 130 may generate a list of retained target boxes based on the semantic identifiers of the target boxes to be retained. The list of retained target boxes may include a sequence of data composed of the semantic identifiers of the target boxes to be retained.

[0240] In some embodiments, the determination module 130 can traverse all point cloud points to determine the corresponding pixel point projected onto the semantic map of the target box for each point cloud point. The determination module 130 can determine the semantic value corresponding to the pixel point. If the semantic value corresponding to the pixel point is non-zero, it can indicate that the pixel point is a pixel point of the retained target box corresponding to the semantic value, and the determination module 130 can assign the point cloud point to the retained target box.

[0241] For example, if the retained target box is the i-th target box to be processed, the judgment module 130 can traverse all point cloud points, determine the corresponding pixel point projected onto the semantic map of the target box for each point cloud point, and determine all pixels with semantic value i. The point cloud points corresponding to the pixels with semantic value i are then assigned to the i-th target box to be processed.

[0242] The correspondence relationship refers to the relationship between pixels on the target bounding box semantic map and the retained target bounding boxes. In some embodiments, the correspondence relationship may include whether the pixels on the target bounding box semantic map and the retained target bounding boxes belong to the same target category, etc.

[0243] In some embodiments, the determination module 130 can determine whether the pixels in the target box semantic map and the retained target box belong to the same target category based on the semantic values ​​of the pixels in the target box semantic map and the semantic values ​​of the retained target boxes. For example, if the semantic values ​​are the same, the pixels in the target box semantic map and the retained target boxes belong to the same target category. If the semantic values ​​are different, the pixels in the target box semantic map and the retained target boxes do not belong to the same target category.

[0244] For more information about point cloud points, please refer to the relevant description in step 210.

[0245] In some embodiments, the determination module 130 further obtains the point cloud points corresponding to the retained target box based on the pixels corresponding to the retained target box, and then determines the convex hull feature corresponding to the retained target box. The convex hull feature refers to the convex hull formed by the point cloud points corresponding to the retained target box.

[0246] A convex hull is a convex polygon or a convex polyhedron that includes all points in a set of points and has the smallest area. For example, it may contain all point cloud points corresponding to the target bounding box. At least some of the point cloud points may lie on the edges of the convex polygon or on the faces of the convex polyhedron. Other point cloud points may lie inside the convex polygon or the convex polyhedron. In some embodiments, the edges or faces of the convex hull may include at least one edge or face of the target bounding box.

[0247] In some embodiments, the determination module 130 can determine the convex hull feature of the retained target box based on the retained target box and its corresponding point cloud points. For example, at least one edge or at least one face of the retained target box can be modified to form a convex hull, which can include point cloud points outside the retained target box.

[0248] In some embodiments, the determination module 130 may determine the convex hull feature corresponding to the retained target box based on a third preset algorithm. The third preset algorithm may include at least one of Graham scan algorithm, fast convex hull algorithm, etc.

[0249] In some embodiments, when the target bounding box is a planar shape, the convex hull feature can be determined using algorithms such as Graham's scan. In some embodiments, when the target bounding box is a three-dimensional shape, the convex hull feature can be determined using algorithms such as fast convex hull algorithm.

[0250] By determining the convex hull features corresponding to the retained target boxes, the retained target boxes can be updated to include all point cloud points of the same target category, thereby improving the segmentation accuracy of the retained target boxes.

[0251] This specification provides one or more embodiments of a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer can execute a target selection method based on point cloud processing algorithms in port operation.

[0252] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0253] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0254] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0255] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0256] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A target selection method based on point cloud processing algorithm in port navigation, characterized in that, include: Acquire point cloud data in the port environment; Based on point cloud data, determine the target bounding box to be processed; Project the target box to be processed onto the target box semantic map to determine the pixel region of the target box to be processed in the target box semantic map; A target bounding box semantic graph is a graph used to assign semantic labels to pixels; Based on the semantic values ​​of the pixels within the pixel region, it is determined whether the target box to be processed needs to be removed; Based on the judgment result, the point cloud data corresponding to the target box to be processed is processed; Cluster analysis is performed on the processed point cloud data to form a valid set of target point clouds; Extract features from the point cloud in the effective target point cloud set to generate target feature information; Based on target feature information, the categories of valid targets are determined and the valid targets after category determination are output.

2. The target selection method based on point cloud processing algorithm in port driving according to claim 1, characterized in that, The step of determining whether the target box to be processed needs to be removed based on the semantic values ​​of the pixels within the pixel region includes: Determine whether there are non-zero pixels in the pixel region; Since there are no non-zero pixels in the pixel region, it is determined that the target box to be processed does not need to be removed; In response to the presence of non-zero pixels in the pixel region, an overlapping target box that overlaps with the target box to be processed is determined based on the non-zero pixels; Based at least on the target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes, it is determined whether the target bounding box to be processed needs to be removed.

3. The target selection method based on point cloud processing algorithm in port driving according to claim 2, characterized in that, If it is determined that the target box to be processed does not need to be removed, the semantic value of the pixel in the pixel area is assigned to the identifier value corresponding to the target box to be processed, and the projection and judgment operation is performed on the next target box to be processed.

4. The target selection method based on point cloud processing algorithm in port driving according to claim 2, characterized in that, The step of determining whether the target box to be processed needs to be removed, based at least on the target categories corresponding to the target box to be processed and the overlapping target boxes, includes: Determine the intersection area and the merged area of ​​the target box to be processed and the overlapping target boxes; The degree of overlap is determined based on the intersection area and the merged area; Based on the target categories corresponding to the target bounding boxes to be processed and the overlapping target bounding boxes, an overlap threshold is determined. Based on the overlap degree and the overlap threshold, it is determined whether the target box to be processed needs to be removed.

5. The target selection method based on point cloud processing algorithm in port driving according to claim 4, characterized in that, The step of determining the overlap threshold based on the target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes includes: If the target categories corresponding to the target bounding box to be processed and the overlapping target bounding boxes are the same, the overlap threshold is set to be less than the baseline threshold. If the target categories corresponding to the target boxes to be processed and the overlapping target boxes are different, the overlap threshold is determined based on the degree of proximity of the target categories corresponding to the target boxes to be processed and the overlapping target boxes.

6. The target selection method based on point cloud processing algorithm in port driving according to claim 4, characterized in that, The step of determining whether the target box to be processed needs to be removed based on the overlap degree and the overlap threshold includes: In response to the overlap being less than the overlap threshold, it is determined that the target box to be processed does not need to be removed; In response to the overlap being not less than the overlap threshold, it is determined that the target box to be processed or the overlapping target box needs to be removed.

7. The target selection method based on point cloud processing algorithm in port driving according to claim 6, characterized in that, If it is determined that the target box to be processed or the overlapping target box needs to be removed, a first confidence level of the target box to be processed and a second confidence level of the overlapping target box are determined based on the target categories corresponding to the target box to be processed and the overlapping target box, respectively. Based on the first confidence level and the second confidence level, target boxes that need to be removed are determined from the target boxes to be processed and the overlapping target boxes.

8. The target selection method based on point cloud processing algorithm in port driving according to claim 7, characterized in that, The method further includes: In response to the fact that the target box to be removed is the overlapping target box, the coverage area of ​​the overlapping target box on the target box semantic map is obtained; The semantic value of the pixels within the coverage area is assigned to 0.

9. A point cloud processing system, characterized in that, A target selection method based on point cloud processing algorithm in port driving as described in any one of claims 1 to 8 includes: The first determination module is configured to determine the target bounding box to be processed based on point cloud data; The second determining module is configured to project the target box to be processed onto the target box semantic map and determine the pixel region of the target box to be processed in the target box semantic map; The judgment module is configured to determine whether the target box to be processed needs to be removed based on the semantic values ​​of the pixels in the pixel region. The processing module is configured to process the point cloud data corresponding to the target box to be processed based on the judgment result.

10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions. When the computer reads the computer instructions in the storage medium, the computer executes a target selection method based on point cloud processing algorithm in port driving as described in any one of claims 1 to 8.

Images