A visual recognition-based method and system for classifying cargo information

Abstract

This application relates to the field of data processing technology, and in particular to a method and system for classifying cargo information based on visual recognition. The method includes the following steps: acquiring a complete point cloud image of the cargo; for any data point in the point cloud image, calculating its local density and local curvature in a neighborhood composed of different numbers of neighboring points to construct a location quality index; constructing a seed point priority based on the local density and local curvature of the data point in the optimal neighborhood, wherein local density is positively correlated with seed point priority, and local curvature is negatively correlated with seed point priority; determining seed points based on the priority of the data points; setting an adaptive threshold; and using an Euclidean clustering algorithm to segment the point cloud image of the cargo based on the adaptive threshold. This application has the effect of reducing the generation of invalid clusters, improving the stability of the cluster structure, and facilitating subsequent differentiation of cargo.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method and system for classifying cargo information based on visual recognition. Background Technology

[0002] With the development of intelligent manufacturing and smart logistics, some production workshops have introduced automated warehousing systems. In these systems, the efficient and accurate classification of goods information has become a key factor in ensuring the efficiency and precision of logistics operations. To achieve automatic identification and classification of goods, current technologies generally use LiDAR to collect 3D point clouds of goods. A complete point cloud is formed by stitching together point clouds from multiple perspectives, and then the complete point cloud is projected to obtain a feature grayscale image. The grayscale information in the feature grayscale image is the magnitude of the normal vector of the goods point cloud along the z-axis of the LiDAR, lacking point cloud distance information and unable to distinguish goods in different columns of the same layer. For example, a column of goods might be missing from the top layer. To solve this problem, clustering algorithms are typically used to cluster the point cloud image before feature grayscale projection. After clustering, the point cloud image is divided into two parts, and different colors can be used to distinguish the missing top layer and other areas.

[0003] In traditional Euclidean clustering algorithms, seed points are typically selected randomly from the complete point cloud image or traversed in the order of the point cloud index. However, this method is prone to causing oversegmentation (one object is split into two) by setting edge points as seed points; or selecting isolated points in low-density regions as seed points, resulting in invalid clusters. This leads to an unstable final cluster structure, affecting the operation of scenarios such as intelligent warehousing or automated sorting. Summary of the Invention

[0004] To address the problem of poor cluster structure stability in traditional clustering algorithms, which affects the operation of subsequent intelligent warehousing and automated sorting, this application provides a visual recognition-based cargo information classification method and system.

[0005] Firstly, this application provides a visual recognition-based method for classifying goods information, employing the following technical solution:

[0006] A visual recognition-based cargo information classification method is proposed, which involves acquiring a complete point cloud image of the cargo; for any data point in the point cloud image, calculating its local density and local curvature in a neighborhood composed of different numbers of neighboring points to construct a location quality index, where the number of neighboring points is within a preset neighborhood interval, and determining the optimal number based on the location quality index, and using the neighborhood composed of the optimal number of data points as the optimal neighborhood; constructing a seed point priority based on the local density and local curvature of the data point in the optimal neighborhood, where local density is positively correlated with seed point priority and local curvature is negatively correlated with seed point priority; determining seed points based on the priority of the data points; setting an adaptive threshold, and using an Euclidean clustering algorithm to segment the point cloud image of the cargo based on the adaptive threshold.

[0007] The beneficial effects are as follows: For each data point in a point cloud image, its local density and local curvature are first calculated. The calculation of local density and local curvature is based on the number of neighboring points of the data point, and different numbers of neighboring points will lead to different local curvatures and local densities of the data points. Therefore, in this application, a location quality index is constructed based on local density and local curvature. The optimal number of neighboring points is selected according to the location quality index to form the neighborhood of the data point. Then, the local density and local curvature corresponding to this neighborhood are used as the basis for the priority of the data point as a seed point. Seed points are selected based on the priority and then clustering is performed. Compared with the traditional method of randomly selecting seed points for clustering, the technical solution in this application can reduce the number of seed points falling in sparse regions and edge positions, thereby reducing the generation of invalid clusters and reducing the occurrence of oversegmentation.

[0008] Optionally, the step of constructing the seed point priority based on the local density and local curvature of the data point in the optimal neighborhood includes: for any data point, taking the local density of the data point at the optimal quantity as the optimal density, and taking the local curvature of the data point at the optimal quantity as the optimal curvature; and taking the product of the result of the optimal curvature processed by the exponential function and the local density as the priority of the data point.

[0009] The beneficial effects are: seed points are selected based on optimal density and optimal curvature, and seed points should be avoided from falling on locations with high curvature and low density. Therefore, in this method, optimal density is directly proportional to priority, while optimal curvature is inversely proportional to priority.

[0010] Optionally, the steps for determining the optimal number based on the location quality index include: comparing the location quality indices corresponding to different neighborhoods of the data points, obtaining the maximum value of the location quality index, and taking the number of neighborhood points corresponding to the maximum value as the optimal number.

[0011] The beneficial effect is that by selecting the optimal location quality index within a certain range and then using the corresponding number of neighborhood points as the optimal number, the selection of the number of neighborhood points becomes more reasonable.

[0012] Optionally, the calculation steps for local density include: obtaining the neighborhood volume and using the ratio of the number of neighborhood points to the neighborhood volume as the local density.

[0013] The beneficial effect is that it enables the calculation of local density based on the neighborhood volume and the number of neighboring points in the neighborhood.

[0014] Optionally, the PCA decomposition algorithm can be used to calculate the local curvature of the data points at different numbers of neighboring points.

[0015] Optionally, the steps for constructing the location quality index include: for a neighborhood consisting of any number of neighboring points, taking the mean of the local densities of all data points as the global density, and taking the maximum value of the local curvature among all data points as the maximum curvature; taking the ratio of the global density to the local density as the density ratio of the data points, taking the ratio of the maximum curvature to the local curvature as the curvature ratio, and taking the product of the density ratio and the curvature ratio as the location quality index.

[0016] The beneficial effect is that, in the process of determining the optimal number, if data points If a certain neighborhood has a local density of high density and a relatively low density, then a smaller number of neighborhood points should be selected to avoid over-smoothing and preserve local geometric details. If a point If the local density is low and the ratio is high, then a larger value should be selected. The value ensures the stability of local features within the neighborhood of nearest neighbor points and resists noise interference.

[0017] Optionally, the preset range for the number of neighboring points is: .

[0018] The beneficial effect is that it is mainly used here to control the range of the number of neighboring points, preventing the number of neighboring points from being too small or too large and affecting subsequent calculations.

[0019] Optionally, seed points can be selected in the neighborhood of each data point based on the priority of each data point using non-maximum suppression.

[0020] Optionally, the step of setting the adaptive threshold includes: for any seed point, obtaining the Euclidean distance between the seed point and all neighboring points in the optimal neighborhood, using the priority corresponding to the neighboring points in the optimal neighborhood as the weight of the data point, and using the weighted average of multiple neighboring points as the adaptive threshold.

[0021] The beneficial effects are as follows: Each seed point adaptively sets its clustering range based on the density characteristics of its actual location, solving the adaptation difficulty of traditional fixed-radius methods in non-uniform point clouds. In sparse regions, the clustering radius is automatically increased to avoid mistakenly dividing the same object into multiple clusters. In dense regions, the clustering radius is appropriately reduced due to priority weighting to avoid aggregating too many irrelevant points. In complex industrial applications such as intelligent warehousing and stacking recognition, where objects vary in size, are arranged randomly, and have uneven point cloud density distribution, the adaptive clustering threshold mechanism significantly improves the system's adaptability and error tolerance to various working conditions.

[0022] Secondly, this application provides a cargo information classification system based on visual recognition, employing the following technical solution:

[0023] A visual recognition-based cargo information classification system includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the system implements the aforementioned visual recognition-based cargo information classification method.

[0024] The beneficial effect is that the above-mentioned visual recognition-based cargo information classification method generates a computer program and stores it in the memory so that it can be loaded and executed by the processor. Thus, the system can be made based on the memory and the processor, making it convenient to use.

[0025] This application has the following technical advantages:

[0026] Prioritizing each data point based on its local density and curvature reduces the likelihood of selecting seed points in sparsely distributed areas, thus minimizing the generation of invalid clusters. It also reduces the likelihood of seed points falling on the edges of the point cloud image, thereby reducing over-segmentation. Attached Figure Description

[0027] Figure 1 This is a flowchart of a visual recognition-based cargo information classification method according to this application. Detailed Implementation

[0028] This application discloses a visual recognition-based cargo information classification method. It calculates the priority of each data point as a seed point based on its local density and curvature in a point cloud image, thus avoiding the selection of seed points at sparsely dense or highly curvature edge locations. Compared to traditional methods that randomly select seed points, this method reduces the occurrence of over-segmentation of the point cloud image and the generation of invalid clusters, thereby improving the accuracy of subsequent clustering and enhancing the classification effect.

[0029] Reference Figure 1A visual recognition-based cargo information classification method includes steps S1-S5.

[0030] S1: Obtain a complete point cloud image of the cargo.

[0031] First, point cloud images from different viewpoints are acquired using LiDAR. These images are then downsampled using a random sampling algorithm to reduce file size, accelerate subsequent processing, and preserve structural information to the greatest extent possible, maintaining the basic outline of the cargo. Considering the overall placement of the stacks and the distances between them, the radar's illumination angle is limited, leading to incomplete data in individual stack point cloud images. Therefore, coarse and fine registration are performed on the multi-view point cloud images using RANSAC (Random Sample Consensus) and ICP (Iterative Closest Point). The rotation matrix between two point cloud images is calculated, and the resulting rotation matrix is ​​used to stitch the multiple frames together to obtain a complete point cloud image of the cargo. RANSAC and ICP are standard techniques in this field and will not be elaborated upon further.

[0032] S2: For any data point in the point cloud image, calculate its local density and local curvature in the neighborhood formed by different numbers of neighboring points to construct a location quality index. The number of neighboring points is within a preset neighborhood interval. Determine the optimal number based on the location quality index, and take the neighborhood formed by the optimal number of data points as the optimal neighborhood.

[0033] For any data point, its local density and local curvature are calculated based on neighborhoods formed by different numbers of neighboring points. For any given data point, the local density varies depending on the number of neighboring points selected. Therefore, this method iterates through the local density and local curvature corresponding to different neighborhoods of the data point within a certain range. In this embodiment, the preset range for the number of neighboring points is... .

[0034] Specifically, the formula for calculating local density is: In the formula, For point In its Local density within a neighborhood composed of nearest neighbors; For point Its first Distance between neighboring points; This indicates the number of neighboring points.

[0035] The steps for calculating local curvature include: calculating the neighborhood centroid using the PCA decomposition method based on the three-dimensional coordinates of the data points, and constructing... Covariance matrix, extract three eigenvalues The local curvature is calculated based on three eigenvalues. The formula for calculating the local curvature can be expressed as:

[0036] In the formula, For point In its Local curvature in a neighborhood composed of neighboring points; The first feature value extracted by the PCA algorithm; The second feature value extracted by the PCA algorithm; This is the third feature value extracted by the PCA algorithm.

[0037] Local curvature reflects the degree of surface curvature in a point cloud image and is a key feature for distinguishing flat regions from edges. When When it approaches 0, it indicates that the point and The nearest neighbor is on a flat surface, when Approaching Time indicates point Located in areas with high curvature, such as edges or corners.

[0038] Subsequently, a location quality index is constructed based on local density and local curvature. The steps for constructing the location quality index include: for a neighborhood composed of any number of neighboring points, the mean of the local densities of all data points is taken as the global density, and the maximum value of the local curvature among all data points is taken as the maximum curvature; the ratio of global density to local density is taken as the density ratio of data points, the ratio of maximum curvature to local curvature is taken as the curvature ratio, and the product of density ratio and curvature ratio is taken as the location quality index.

[0039] Specifically, the formula for calculating the location quality index can be expressed as: In the formula, Representing data points Location quality index; This represents the mean of the local density of all data points, which is also the global density. Indicates that the data points are in Local density in a neighborhood consisting of 10 neighboring points; Indicates that the data points are in Local curvature in a neighborhood formed by neighboring points; This represents the maximum local curvature of all data points, i.e., the maximum curvature.

[0040] In the formula, This is the density ratio, reflecting the relationship between the local density of the current data point and the global density corresponding to all data points, while also performing the standardization operation on the local density. Similarly, Standardized data points Local curvature; The larger the ratio, the higher the value. The smaller the value, the smaller the value should be. Values ​​retain details; while The smaller the ratio, the higher the value. The larger the value, the larger the flat area should be. Value smoothing noise.

[0041] Finally, based on the location quality index, the optimal number of data points is determined, and the neighborhood formed by the optimal number of data points is taken as the optimal neighborhood. In this embodiment, each data point corresponds to multiple location quality indices. The multiple location quality indices are compared, and the maximum value of the location quality index is obtained. The maximum value corresponds to... That is, the number of neighboring points is taken as the optimal number.

[0042] S3: Construct a seed point priority based on the local density and local curvature of the data point in the optimal neighborhood, where local density is positively correlated with seed point priority and local curvature is negatively correlated with seed point priority.

[0043] For any data point, the local density of the data point with the optimal quantity is taken as the optimal density, and the local curvature of the data point with the optimal quantity is taken as the optimal curvature; the product of the result of the optimal curvature processed by the exponential function and the local density is taken as the priority of the data point.

[0044] Specifically, In the formula, For point As a priority of seed points For data points The local density in the optimal neighborhood, i.e., the optimal density; For data points The local curvature in the optimal neighborhood is the optimal curvature.

[0045] In the formula, the higher the optimal density and the lower the optimal curvature, the higher the priority. This reduces the number of seed points that fall in the corners or sparsely distributed areas of the point cloud image, reduces the generation of invalid clusters, and reduces the occurrence of oversegmentation.

[0046] S4: Determine seed points based on the priority of data points.

[0047] After calculating the priority of all data points in the point cloud image as seed points, non-maximum suppression (NMS) is used to select seed points in the neighborhood of each data point based on the priority of each data point. A KD-Tree is then used to find all seed points that meet the conditions, forming a seed point set. The non-maximum suppression algorithm and KD-Tree are standard techniques in this field and will not be elaborated upon here.

[0048] S5: Set an adaptive threshold, and use the Euclidean clustering algorithm to segment the point cloud image of the goods based on the adaptive threshold.

[0049] Traditional Euclidean clustering algorithms employ a uniform adaptive threshold. However, in this scenario, the uneven density of data points in the point cloud image makes it difficult to uniformly apply the adaptive threshold. Sparse regions may fail to form effective clusters, while dense regions are prone to oversegmentation, potentially leading to the incorrect merging or ignoring of misaligned goods. Therefore, in this embodiment, for any seed point, the Euclidean distance between the seed point and all neighboring points in the optimal neighborhood is obtained. The priority of the neighboring points in the optimal neighborhood is used as the weight of the data point, and the weighted average of multiple neighboring points is used as the adaptive threshold.

[0050] Specifically, the formula for calculating the adaptive threshold can be expressed as: In the formula For data points The The priority of each neighboring point as a seed point is given by the following factors: ; This is the normalization function. For point Its first Euclidean distance between neighboring points; It is a linear normalization function.

[0051] use Weighting the Euclidean distance gives points in flat, dense regions a greater weight in the distance calculation, resulting in points... The clustering adaptive threshold improves the accuracy and robustness of subsequent clustering.

[0052] Clustering is performed using an improved adaptive clustering distance threshold to obtain clustering results. After obtaining the clustering results, the complete point cloud image is projected into a two-dimensional feature grayscale image for segmentation to obtain the target goods. These results are then compared with the goods information in the template library to complete the classification. This process is a conventional technique in this field and will not be described in detail in this embodiment.

[0053] This application also discloses a visual recognition-based cargo information classification system, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement a visual recognition-based cargo information classification method according to this application.

[0054] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0055] While this specification has shown and described numerous embodiments of the present application, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of the present application. It should be understood that various alternatives to the embodiments of the present application described herein may be employed in the practice of this application.

[0056] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for classifying information of goods based on visual recognition, characterized by, Acquire a complete point cloud image of the cargo; for any data point in the point cloud image, calculate its local density and local curvature in a neighborhood composed of a different number of neighborhood points, the number of neighborhood points being in a preset neighborhood interval, construct a position quality index based on the local density and the local curvature, determine an optimal number based on the position quality index, and take a neighborhood composed of the optimal number of data points as an optimal neighborhood; construct a seed point priority based on the local density and the local curvature of the data point in the optimal neighborhood, wherein the local density is positively correlated with the seed point priority, and the local curvature is negatively correlated with the seed point priority; determine a seed point based on the priority of the data point; set an adaptive threshold, and segment the point cloud image of the cargo using a Euclidean clustering algorithm based on the adaptive threshold; For a neighborhood composed of any number of neighborhood points, take the mean value of the local density of all data points as a global density, and take the maximum value of the local curvature in all data points as a maximum curvature; Take the ratio of the global density to the local density as a density ratio of the data point, take the ratio of the maximum curvature to the local curvature as a curvature ratio, and take the product of the density ratio and the curvature ratio as a position quality index.

2. The method of claim 1, wherein the method further comprises: The step of constructing a seed point priority based on the local density and the local curvature of the data point in the optimal neighborhood includes: for any data point, taking the local density of the data point under the optimal number as an optimal density, and taking the local curvature of the data point under the optimal number as an optimal curvature; and taking the product of the result of processing the optimal curvature through an exponential function and the local density as the priority of the data point. 3.The method of claim 1, wherein, The step of determining an optimal number based on a position quality index includes: comparing the position quality indexes corresponding to different neighborhoods of the data point, obtaining a maximum value of the position quality index, and taking the number of neighborhood points corresponding to the maximum value as the optimal number.

4. The method of claim 1, wherein the method further comprises: The step of calculating a local density includes: obtaining a neighborhood volume, and taking the ratio of the number of neighborhood points to the neighborhood volume as the local density.

5. The method of claim 1, wherein the method further comprises: The local curvature of the data point under a different number of neighborhood points is calculated using a PCA decomposition algorithm.

6. The method of claim 3, wherein the method further comprises: The preset neighborhood point quantity range is .

7. The method of claim 1, wherein the method further comprises: Seed points are selected in the neighborhood corresponding to each data point based on the priority of each data point using non-maximum suppression. 8.The method of claim 1, wherein, The step of setting an adaptive threshold includes: for any seed point, obtaining the Euclidean distance between the seed point and all neighborhood points in the optimal neighborhood, taking the priority of the neighborhood points in the optimal neighborhood as the weight of the data point, and taking the weighted average of a plurality of neighborhood points as the adaptive threshold.

9. A visual recognition-based cargo information classification system, characterized by, The method comprises: A processor and a memory, the memory storing computer program instructions which, when executed by the processor, implement a visual recognition-based cargo information classification method according to any one of claims 1-8.

Images