Intelligent warehousing article sorting method and system based on AI vision

Abstract

The application relates to the technical field of intelligent warehousing and logistics automation, and discloses an intelligent warehousing article sorting method and system based on AI vision, which comprises the following steps: acquiring original vision information, article stacking height map and environmental obstacle point cloud data, and obtaining a preliminary clear image through enhancement processing; multi-scale features are extracted and an illumination normalization channel is constructed to obtain a corrected spatial feature tensor; the feature tensor is filtered and reorganized by using a shielding mask to obtain an anti-shielding stable feature vector; high-dimensional space matching and rigid body transformation calculation are performed to determine the article category and pose; priority scheduling is performed in combination with the height map to obtain an optimized sorting task queue; adaptive trajectory planning and dynamic conflict resolution are performed in combination with the point cloud data to obtain a conflict-free control sequence. The method can realize accurate identification and efficient obstacle avoidance sorting in a complex scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent warehousing and logistics automation technology, and in particular to an intelligent warehousing item sorting method and system based on AI vision. Background Technology

[0002] AI vision technology is widely used in item recognition and localization, aiming to simulate human eye function through computer vision algorithms to automatically extract and analyze the features of goods. However, the complexity of visual information makes the feature extraction process highly susceptible to interference from external factors such as light, angle, or occlusion, resulting in incomplete or distorted features. Therefore, how to stably extract deep features of goods in complex environments and achieve accurate category recognition has become a key issue in the intelligent sorting process of warehouse goods.

[0003] In existing technologies, traditional mainstream item sorting methods are typically used to handle warehousing tasks, primarily relying on basic image acquisition and feature matching techniques. This approach often reveals significant shortcomings when faced with complex scenarios involving a wide variety of items in different shapes and sizes. Specifically, in a busy warehouse environment, images captured by cameras may be blurry or obscured due to heavy stacking or insufficient lighting. Because existing technologies lack the ability to extract deep features and adapt to dynamic environments, the system struggles to accurately determine the category of an item, especially when feature extraction is unstable, easily leading to recognition failures due to large feature differences. This not only results in incorrect sorting instructions and items being sent to the wrong areas but also causes overall sorting inefficiency and can even trigger chaos in subsequent logistics processes.

[0004] In summary, existing technologies suffer from low sorting and identification accuracy in complex stacking scenarios. Summary of the Invention

[0005] This invention provides an intelligent warehouse item sorting method and system based on AI vision to solve the problem of low sorting and recognition accuracy in complex stacking scenarios.

[0006] Firstly, in order to solve the above-mentioned technical problems, the present invention provides an intelligent warehouse item sorting method based on AI vision, comprising: The raw visual information in the warehouse scene is acquired, along with the stacking height map of the items and the point cloud data of environmental obstacles. The raw visual information is then processed for noise removal and contrast enhancement to obtain a preliminary clear image. Based on the preliminary clear image, multi-scale visual feature extraction and illumination normalization channel construction are performed to obtain the corrected spatial feature tensor. Based on the corrected spatial feature tensor, spatial consistency analysis is performed and an occlusion mask is generated. The corrected spatial feature tensor is then weighted, filtered, and reconstructed using the occlusion mask to obtain an anti-occlusion stable feature vector. Based on the anti-occlusion stable feature vector and the preset item feature library, high-dimensional spatial projection matching is performed to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, standard model retrieval and rigid body transformation calculation are performed to obtain the confirmed item category and three-dimensional grasping pose. Based on the confirmed item category and the three-dimensional grasping pose, combined with the item stacking height map, sorting instructions based on spatial volume and gradient are generated and priority scheduling is performed to obtain an optimized sorting task queue. Based on the optimized sorting task queue, and combined with the environmental obstacle point cloud data, adaptive trajectory planning and dynamic conflict resolution are performed to obtain a conflict-free control sequence.

[0007] Secondly, the present invention provides an intelligent warehouse goods sorting system based on AI vision, comprising: The data acquisition and enhancement module is used to acquire raw visual information in the warehouse scenario, and to acquire the stacking height map of items and point cloud data of environmental obstacles. The raw visual information is then processed for noise removal and contrast enhancement to obtain a preliminary clear image. The feature extraction and correction module is used to perform multi-scale visual feature extraction and illumination normalization channel construction based on the preliminary clear image to obtain the correction spatial feature tensor. An anti-occlusion processing module is used to perform spatial consistency analysis and generate an occlusion mask based on the corrected spatial feature tensor, and to use the occlusion mask to perform weighted filtering and feature recombination on the corrected spatial feature tensor to obtain an anti-occlusion stable feature vector. The identification and localization module is used to perform high-dimensional spatial projection matching based on the anti-occlusion stable feature vector and the preset item feature library to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, standard model retrieval and rigid body transformation calculation are performed to obtain the confirmed item category and three-dimensional grasping pose. The task scheduling optimization module is used to generate sorting instructions and prioritize them based on spatial volume and gradient, according to the confirmed item category and the three-dimensional grasping pose, combined with the item stacking height map, to obtain an optimized sorting task queue. The motion control and obstacle avoidance module is used to perform adaptive trajectory planning and dynamic conflict resolution based on the optimized sorting task queue and the environmental obstacle point cloud data, so as to obtain a conflict-free control sequence.

[0008] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention removes noise and adjusts contrast by using image preprocessing technology, extracts multi-scale feature pyramids and texture invariant descriptors by applying convolutional neural networks, and isolates occlusion interference and performs feature weighted filtering by combining region segmentation algorithm. This method can effectively eliminate visual interference caused by uneven lighting, perspective distortion and stacking of items in the warehouse environment, and stably extract the deep features of items in complex scenes and achieve accurate similarity comparison, thereby improving the accuracy and robustness of item recognition in dynamic and complex environments and avoiding misjudgment and sorting errors caused by feature extraction distortion.

[0009] (2) This invention maps the confirmed item category and three-dimensional grasping pose to the item stacking height map, calculates the space occupied volume and stacking area gradient to generate grasping priority values, and constructs priority queues and obstacle avoidance paths accordingly. This method can intelligently assess the stability of goods stacking and grasping risks, prioritize the handling of goods in stable areas, and plan smooth paths to avoid elevation obstacles, thereby reducing the risk of items tipping over during sorting, optimizing the logical order and path efficiency of sorting operations, and improving the throughput of overall warehouse management.

[0010] (3) This invention obtains the joint space trajectory of the optimized path, generates an oriented bounding box model by combining environmental obstacle point cloud data, performs real-time collision detection, and performs path replanning and fifth-order polynomial and B-spline interpolation smoothing when a conflict is detected. This method can lock the potential collision risk between the robotic arm and obstacles in advance in the dynamically changing warehouse environment, and automatically generate a conflict-free and smooth transition trajectory, thereby ensuring the safe operation of the automated sorting equipment, reducing mechanical vibration and wear, and realizing efficient, stable and conflict-free automated sorting action. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the intelligent warehouse item sorting method based on AI vision provided in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an AI vision-based intelligent warehouse item sorting system provided in the second embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] Reference Figure 1The first embodiment of the present invention provides an intelligent warehouse item sorting method based on AI vision, including the following steps: S11, acquire the original visual information in the warehouse scene, and acquire the stacking height map of items and the point cloud data of environmental obstacles. Perform noise removal and contrast enhancement processing on the original visual information to obtain a preliminary clear image. S12, Based on the preliminary clear image, perform multi-scale visual feature extraction and illumination normalization channel construction to obtain the corrected spatial feature tensor; S13, Based on the corrected spatial feature tensor, perform spatial consistency analysis and generate an occlusion mask. Use the occlusion mask to perform weighted filtering and feature recombination on the corrected spatial feature tensor to obtain an anti-occlusion stable feature vector. S14. Based on the anti-occlusion stable feature vector and the preset item feature library, perform high-dimensional spatial projection matching to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, perform standard model retrieval and rigid body transformation calculation to obtain the confirmed item category and three-dimensional grasping pose. S15, based on the confirmed item category and the three-dimensional grasping pose, and combined with the item stacking height map, sorting instructions based on spatial volume and gradient are generated and priority scheduling is performed to obtain an optimized sorting task queue. S16. Based on the optimized sorting task queue and combined with the environmental obstacle point cloud data, adaptive trajectory planning and dynamic conflict resolution are performed to obtain a conflict-free control sequence.

[0014] In step S11, the original visual information of the warehouse scene is acquired, along with the item stacking height map and environmental obstacle point cloud data. Noise removal and contrast enhancement processing are then performed on the original visual information to obtain a preliminary clear image, including: Raw visual information is collected using warehouse cameras; Based on the original visual information, adaptive illumination analysis and noise localization are performed, and the standard deviation of brightness in the local area is calculated. If it exceeds the preset standard deviation threshold of illumination, median filtering is performed to obtain a denoised visual image. Based on the denoised visual image, contrast enhancement and edge sharpening are performed, and pixel intensity concentration is calculated. If it exceeds a preset pixel intensity concentration threshold, contrast-limited adaptive histogram equalization and edge sharpening calculations are performed to obtain a preliminary clear image. Depth data of the warehouse scene is collected from depth sensing devices, and the data is mapped and converted into formats to generate item stacking height maps and environmental obstacle point cloud data, respectively.

[0015] In one implementation, raw visual information, represented as RGB format color image data, is first acquired using an industrial-grade vision sensor installed above the warehouse operation area. Subsequently, adaptive illumination analysis and noise localization are performed based on the raw visual information. Specifically, the system sets a sliding window of a predetermined size (such as 3×3 or 5×5 pixels), and allows the sliding window to traverse the image pixel by pixel. For each local area covered by the sliding window, the arithmetic mean of the brightness values ​​of all pixels in it is calculated. Then, the square of the difference between the brightness value of each pixel in the window and the arithmetic mean is calculated. The squares of all differences are added together and divided by the total number of pixels in the window to obtain the variance. Finally, the square root of the variance is taken to obtain the standard deviation of the brightness of the local area. The calculated standard deviation of brightness is compared with a preset illumination standard deviation threshold. If the standard deviation of brightness is greater than the preset illumination standard deviation threshold, it is determined that there are high-frequency noise or non-uniform illumination interference in the local area. At this time, median filtering is performed on the area, that is, the brightness values ​​of all pixels in the window are counted and sorted by value. The value in the middle of the sorted sequence is selected to replace the original value of the center pixel of the window, thereby obtaining a smoothed and denoised visual image.

[0016] It should be noted that the preset illumination standard deviation threshold is set based on standards for indoor workplace lighting (such as ISO 8995-1) and the signal-to-noise ratio characteristic curve of industrial cameras. Specifically, technicians need to acquire solid color background images under standard lighting conditions, calculate the inherent thermal noise standard deviation of the sensor, and usually set 2 to 3 times the inherent noise standard deviation as the preset illumination standard deviation threshold to ensure that only significant external interference noise is filtered out without loss of texture details. In another embodiment, an idle scene image can be acquired under standard lighting, the local brightness standard deviation distribution of multiple samples can be statistically analyzed, and its 95th percentile can be set as the preset illumination standard deviation threshold.

[0017] Next, contrast enhancement and edge sharpening operations are performed on the denoised visual image. This process first calculates the pixel intensity concentration. The system counts the number of pixels at each gray level (0-255) in the denoised visual image, generating a gray-level histogram. Then, the five gray levels with the highest pixel count are selected, and the sum of the pixel counts at these five gray levels is divided by the total number of pixels in the entire image to obtain the pixel intensity concentration value. If this value exceeds a preset pixel intensity concentration threshold, it indicates that image details are submerged in the background. At this point, Limit Contrast Adaptive Histogram Equalization (CLAHE) and edge sharpening calculations are initiated. Specifically, the image is divided into multiple non-overlapping rectangular sub-blocks, and the gray-level histogram of each sub-block is calculated. A clipping threshold is set, and the portion of the histogram exceeding this threshold is cropped and evenly distributed to the other parts of the histogram. The cumulative distribution function is recalculated, and finally, a bilinear interpolation algorithm is used to smoothly fuse the pixel gray levels at the sub-block boundaries to eliminate block artifacts.

[0018] Subsequently, the Laplacian operator is used to perform secondary differential sharpening on the image edges. Specifically, a 3×3 convolution kernel template is first defined, with the coefficient at the center position set to a positive number (e.g., 4), the coefficients at the four neighboring positions (top, bottom, left, and right) set to negative numbers (e.g., -1), and the coefficients at the remaining positions set to 0. This template is then convolved with the image matrix, i.e., the sum of the products of each coefficient within the template and the corresponding pixel value in the image is calculated. This result approximately represents the second derivative of the image brightness. Finally, this second derivative result is added to the original pixel values ​​to highlight high-frequency edge information in the image, thus obtaining a preliminarily clear image.

[0019] It should be noted that the setting of the preset pixel concentration threshold depends on the dynamic range index and histogram statistical characteristics of the vision sensor. For a typical 8-bit depth industrial camera, when the histogram is excessively concentrated (e.g., concentration exceeds 0.6, meaning 60% of the pixels are concentrated in a very small number of gray levels), it means that the image dynamic range is compressed. Therefore, this threshold is usually set between 0.6 and 0.75. In specific implementation scenarios, if the warehouse environment uses point light source illumination, resulting in large areas of shadow or highlight in the image, the above histogram statistics may show a bimodal or multimodal distribution. In this case, those skilled in the art can use a local histogram statistical method instead of the above global statistical method, that is, calculate the average concentration of each sub-block of the image as the judgment basis; by collecting high and low contrast scene samples, observing their histogram concentration, and selecting the critical value that can effectively distinguish between "needs enhancement" and "no processing" scenes as the preset pixel concentration threshold. Furthermore, the selection of this threshold should ensure that, under normal operating lighting conditions (illuminance greater than 300 Lux), the concentration value of the background image of the conveyor belt without any items placed is below this threshold. This ensures that the algorithm only enhances low-contrast images caused by degraded image quality, avoiding overprocessing of normal images. The clipping threshold and template parameters of the Laplacian operator in the CLAHE algorithm should be selected based on the image's signal-to-noise ratio and the required edge strength, referring to the recommended parameter ranges of standard image processing libraries such as OpenCV.

[0020] Simultaneously, depth data of the warehouse scene is collected from depth sensing devices (such as ToF cameras or LiDAR), and this depth data is then mapped and format-converted. Specifically, the depth data consists of a massive number of three-dimensional coordinate points (X, Y, Z), typically in millimeters or meters. First, to construct the stacking height map of items, the system divides the horizontal X-axis and Y-axis coordinate regions into several fixed-size grid cells (e.g., ...). The system iterates through all point cloud data, determining the location of each point within a grid cell. Within each grid cell, it selects the value with the largest vertical Z-axis coordinate and uses this maximum Z-axis value as the height feature value of that grid cell. Combining the height feature values ​​of all grids forms a two-dimensional height matrix, thus obtaining the item stacking height map. For generating environmental obstacle point cloud data, the system calls a pre-stored empty warehouse background point cloud model. This model is a static three-dimensional reference data established by multiple scans and averaging data from a depth sensing device in the initial state of the warehouse area without goods and without dynamic interference (such as personnel or vehicles). The system spatially compares the currently collected point cloud data with the empty warehouse background point cloud model, calculating the Euclidean distance from each point in the current point cloud to the corresponding nearest neighbor in the background model. If the Euclidean distance is greater than a preset change threshold, the point is determined to be a newly added item or obstacle point and retained; otherwise, it is determined to be a background point and removed. Finally, environmental obstacle point cloud data containing only dynamic objects or newly added goods is generated.

[0021] It should be noted that the preset variation threshold is set based on the measurement accuracy error of the depth sensor (e.g., ±5mm), and is usually twice the sensor's nominal error (e.g., 10mm) to prevent the background from being misjudged as an obstacle due to measurement fluctuations of the sensor itself. Regarding the Laplacian operator, setting its center coefficient to 4 and its neighborhood to -1 is one of the standard forms of the second-order differential approximation operator derived from discrete mathematics in the field of image processing. This parameter setting can effectively achieve isotropic edge detection. Those skilled in the art can also choose an extended template with a center coefficient of 8 and a full neighborhood of -1 according to the sharpening requirements in the diagonal direction.

[0022] In step S12, based on the preliminary clear image, multi-scale visual feature extraction and illumination normalization channel construction are performed to obtain the corrected spatial feature tensor, including: Based on the preliminary clear image, a pre-trained convolutional neural network model is used to perform deep feature mining and texture descriptor extraction to obtain a basic feature map; Based on the aforementioned basic feature map, feature pyramid construction and multi-scale fusion are performed, and feature scale unification is achieved through upsampling mapping to obtain a multi-scale fused feature map. Based on the multi-scale fusion feature map, channel brightness normalization and viewpoint affine transformation correction are performed to obtain the correction space feature tensor.

[0023] In one implementation, the preliminary clear image output in step S11 is first scaled to the model's preset standard input size (e.g., 512×512 pixels) using bilinear interpolation. This size is determined based on the memory bandwidth and computational core load balance point of mainstream deep learning inference graphics cards (such as NVIDIA Tesla T4) to ensure a real-time processing frame rate of no less than 30fps. Subsequently, the scaled image is input into a pre-trained ResNet-50 residual neural network model for feature extraction. The specific operation of this model is as follows: the image data passes through five convolutional stages (Stage 1 to Stage 5) of the model in sequence. In each convolutional layer, a convolutional kernel matrix of a preset size (e.g., 3×3) slides on the image feature map with a fixed stride. At each position, the weight values ​​in the convolutional kernel matrix are multiplied element-wise with the pixel values ​​in the corresponding receptive field of the image and summed. The bias term value is then added to obtain the convolutional response value at that position. Subsequently, the distribution is adjusted through a batch normalization layer and negative values ​​are removed by the ReLU activation function to obtain the basic feature map.

[0024] Next, a Feature Pyramid (FPN) is constructed and multi-scale fusion is performed based on the base feature map. This process adopts a top-down approach. First, the high-level feature map output from Stage 5 of the ResNet-50 model is selected as the top layer. It is upsampled to increase its size. The upsampling uses a bilinear interpolation algorithm. That is, for each pixel in the target magnified image, the four neighboring pixels around its projection position in the original feature map are found. The geometric distance from the projection point to these four neighboring points is calculated. The inverse of the geometric distance is normalized and used as the weight coefficient. The weighted sum of the feature values ​​of these four neighboring pixels is calculated as the feature value of the target pixel, thereby doubling the size of the feature map. At the same time, a 1×1 convolution kernel is used to reduce the dimensionality of the secondary feature map output from Stage 4 to unify the number of channels (e.g., to 256 channels). Finally, the upsampled high-level feature map and the dimensionality-reduced secondary feature map are added element-wise. This process is repeated until it is fused to the output layer of Stage 2 to obtain a multi-scale fused feature map containing rich semantics and details.

[0025] Finally, channel brightness normalization and viewpoint affine transformation correction are performed based on the multi-scale fused feature map. Specifically, each channel of the feature map is traversed, the feature values ​​of all pixels in that channel are statistically analyzed, and their arithmetic mean and standard deviation are calculated. Then, a standardization calculation is performed, which involves subtracting the arithmetic mean of the channel from the original value of the feature point, and then dividing the difference by the sum of the standard deviation and a small constant. The small constant is used to prevent the denominator from being zero, thereby eliminating the influence of illumination intensity on the feature response amplitude. The viewpoint affine transformation correction operation uses a pre-constructed homography transformation matrix to spatially resample the normalized feature map. The specific process uses the inverse mapping method. Specifically, each pixel coordinate in the target corrected view (i.e., the top view) is traversed, and it is multiplied by the inverse of the homography transformation matrix to calculate the corresponding floating-point coordinates of that point in the original tilted feature map. Then, the feature value at that floating-point coordinate is calculated using the bilinear interpolation algorithm described above and filled into the corresponding position in the target view, finally obtaining a corrected spatial feature tensor with eliminated perspective distortion and uniform illumination.

[0026] It should be noted that the ResNet-50 model selected in this embodiment is a deep residual network containing 50 layers. Its core lies in the introduction of a "skip connection" mechanism, which directly adds the input to the output of the convolutional block, effectively solving the problem of gradient vanishing in deep networks. The model is first pre-trained on the ImageNet general dataset to obtain basic visual extraction capabilities. Then, an image dataset containing items unique to this warehouse scenario (such as turnover boxes of specific colors and cardboard boxes with barcodes) is collected. The model parameters are fine-tuned by transfer learning using the stochastic gradient descent (SGD) algorithm until the loss function converges. The method for constructing the homography transformation matrix is ​​as follows: In the initial stage of system deployment, a calibration reference object containing at least four known physical coordinate feature points (such as checkerboard corner points) is placed on the ground within the camera's field of view. The pixel coordinates (u, v) of these four points in the image coordinate system and their physical coordinates (X, Y) on the ground plane (Z=0) in the world coordinate system are extracted. A system of linear equations containing eight parameters is constructed and solved to obtain a 3×3 homography matrix describing the projection relationship from the image plane to the physical ground plane. The small constant is typically set as follows: This value is set based on the precision standard of computer floating-point arithmetic, which can avoid division by zero errors and will not cause significant deviations in the feature distribution.

[0027] In step S13, spatial consistency analysis is performed based on the corrected spatial feature tensor to generate an occlusion mask. The occlusion mask is then used to perform weighted filtering and feature reorganization on the corrected spatial feature tensor to obtain an anti-occlusion stable feature vector, including: Based on the corrected spatial feature tensor, local response anomaly detection is performed, the feature response value and the neighborhood mean are calculated, and the absolute value of the difference between the feature response value and the neighborhood mean is calculated. The region where the feature response value is lower than the neighborhood mean and the absolute value of the difference exceeds a preset response deviation threshold is determined as the initial interference region. Based on the corrected spatial feature tensor and the initial interference region, the pre-trained semantic segmentation network model is input to perform accurate occlusion region segmentation, resulting in an accurate occlusion region segmentation map. Based on the precise occlusion region segmentation map and the corrected spatial feature tensor, feature mask reverse filtering and global aggregation and recombination are performed to obtain an anti-occlusion stable feature vector.

[0028] In one implementation, spatial consistency analysis is performed based on the corrected spatial feature tensor to generate an occlusion mask. The occlusion mask is then used to perform weighted filtering and feature reorganization on the corrected spatial feature tensor to obtain an anti-occlusion stable feature vector. Specifically, firstly, a local response anomaly detection operation is performed based on the corrected spatial feature tensor. This operation is based on the principle of feature continuity, defining a sliding window of size k×k (e.g., 3×3), and performing step-by-step traversal on each channel feature map of the feature tensor. For each local region covered by the window, the values ​​excluding the center point are calculated. The arithmetic mean of the neighboring pixels is denoted as the neighborhood mean. At the same time, the difference between the feature response value of the center point of the window and the neighborhood mean is calculated, and the absolute value of the difference is taken as the response deviation. The system traverses all pixels and marks the pixel positions that simultaneously meet the two conditions of the center feature response value being less than the neighborhood mean and the response deviation being greater than the preset response deviation threshold as 1, and the other positions as 0, thereby generating a single-channel two-dimensional initial interference region map.

[0029] Next, based on the corrected spatial feature tensor and the initial interference region, a pre-trained semantic segmentation network model is input to perform accurate occlusion region segmentation. This operation first performs channel dimension concatenation. Assuming the dimensions of the corrected spatial feature tensor are (C, H, W), where C is the number of channels, H and W are the spatial dimensions, and the dimensions of the initial interference region map are (1, H, W), the system stacks the two in the channel dimension to construct a combined input tensor with dimensions (C+1, H, W). Subsequently, this combined input tensor is input into the semantic segmentation network of the U-Net architecture. The network performs four downsampling operations through the encoder path to extract multi-scale contextual information, and performs corresponding four upsampling operations and feature concatenation through the decoder path to restore spatial resolution. Finally, the output layer outputs a probability distribution map with dimensions (1, H, W) through the Sigmoid activation function. The probability map is then binarized and thresholded (the threshold is usually set to 0.5) to obtain the accurate occlusion region segmentation map.

[0030] Finally, based on the precise occlusion region segmentation map and the correction spatial feature tensor, feature mask reverse filtering and global aggregation and recombination operations are performed. Specifically, first, a logical NOT operation is performed on the precise occlusion region segmentation map to generate a weight matrix, in which non-occluded regions are 1 and occluded regions are 0; then, the Hadamard product (element-wise multiplication) is used to apply this weight matrix to each channel of the correction spatial feature tensor, forcing the feature values ​​of occluded regions to zero; finally, a global average pooling operation is performed on the processed feature tensor, that is, the average value of all non-zero pixel values ​​in each channel is calculated, compressing the tensor of dimension (C,H,W) into a feature vector of dimension (C,1,1), and flattening it into a one-dimensional anti-occlusion stable feature vector of length C.

[0031] It should be noted that the preset response deviation threshold is set based on the statistical distribution characteristics of the feature map. Specifically, the standard deviation of the local gradient of the feature is calculated on the unoccluded sample set, and twice the standard deviation is taken as the threshold to cover 95% of the normal texture fluctuation range. Regarding the semantic segmentation network model, its backbone network uses ResNet-34 to balance accuracy and inference speed, and it is loaded with pre-trained weights from the ImageNet dataset. This is because ImageNet contains a massive amount of basic visual features (such as edges and corners). Utilizing transfer learning, using the pre-trained weights as initial parameters can avoid gradient instability caused by random initialization and accelerate the convergence of the model in specific warehouse scenarios. The model training process is as follows: First, a dedicated warehouse scenario dataset containing labeled images (labeling shelf occlusion, robotic arm shadows, and stacking blind spots) is constructed, containing no less than 1000 labeled images. The Adam optimizer is used, and the initial learning rate is set to... The learning rate is decayed using a cosine annealing strategy. The loss function employs a weighted combination of binary cross-entropy loss and Dice coefficient loss (e.g., each weighted at 0.5) to address the imbalance between occluded and background samples. The training batch size is set according to the GPU memory specifications (e.g., 16 or 32), and training is iterated until the validation set IoU (Intersection over Union) no longer improves. Channel stitching aims to use explicit geometric mutation information (initial interference region) as prior knowledge to guide the network to focus on potential occlusion locations, assisting the network in distinguishing between real object textures and shadow occlusions.

[0032] In step S14, high-dimensional spatial projection matching is performed based on the anti-occlusion stable feature vector and a preset item feature library to obtain a matching similarity. If the matching similarity exceeds a preset feature similarity threshold, standard model retrieval and rigid body transformation calculation are performed to obtain the confirmed item category and 3D grasping pose, including: Based on the anti-occlusion stable feature vector and the pre-set item feature library, vector space mapping and cosine similarity calculation are performed to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, then a standard 3D model is retrieved and aligned with the feature points to obtain spatial transformation parameters; Based on the spatial transformation parameters and the preset standard grasping pose template, rigid body coordinate transformation and pose mapping operations are performed to obtain the confirmed item category and three-dimensional grasping pose.

[0033] In one implementation, vector space mapping and cosine similarity calculation are performed first. The system's pre-built item feature library is a dynamically updated database containing three core datasets for all registered items: high-dimensional standard feature vectors, high-precision standard point cloud models, and corresponding preset standard grasping poses. When a new type of item is added to the library, its multi-angle images and point clouds are collected using an offline scanning device. Feature vectors are generated using the same feature extraction network as online recognition and stored in the library. During online matching, the system maps the anti-occlusion stable feature vector output in step S13 to the same vector space as the feature library, and iterates through and calculates its cosine similarity with each standard feature vector in the library. This is calculated by dividing the dot product of the two vectors by the product of their magnitudes. The result is a value between -1 and 1. The closer the value is to 1, the more consistent the features are in the direction, i.e., the more similar the semantic categories are. The system selects the highest similarity value as the matching similarity and uses the corresponding item ID as the candidate category. If the matching similarity exceeds a preset feature similarity threshold, the item category is successfully confirmed, and the corresponding standard point cloud model and the predefined set of standard key points on that model are retrieved. If the matching similarity is lower than the preset feature similarity threshold, the system will determine the current item as an 'unknown category' or an 'abnormally damaged item'. At this time, the system triggers an anomaly handling mechanism, marking the item's anti-occlusion stable feature vector and original image as 'samples awaiting manual review' and storing them in the anomaly database; simultaneously, a 'bypass diversion' instruction is generated to control the robotic arm to move the unidentified item to the review area, avoiding forced grasping that could lead to item damage or misclassification, thereby ensuring the safety of the sorting process and closed-loop data management.

[0034] Next, a standard 3D model is retrieved and feature points are aligned. This process uses the Iterative Closest Point (ICP) algorithm for spatial registration. First, the centroid of the currently acquired object surface point cloud (source point cloud) and the centroid of the retrieved standard point cloud model (target point cloud) are calculated. The centroid coordinates of the two sets of point cloud data are subtracted from each other to achieve decentralization. Then, an iterative calculation loop is entered. In each iteration, for each point in the source point cloud, the corresponding point with the closest Euclidean distance in the target point cloud is found, and the covariance matrix between the source point cloud and the corresponding point set is constructed. The covariance matrix is ​​decomposed using the singular value decomposition (SVD) method to obtain the left singular vector matrix, the singular value diagonal matrix, and the right singular vector matrix. The current rotation matrix is ​​calculated by multiplying the right singular vector matrix with the transpose of the left singular vector matrix, and the translation vector is calculated based on the centroid deviation after rotation. The source point cloud is spatially transformed using the calculated rotation matrix and translation vector, and the mean square error (MSE) between the transformed source point cloud and the target point cloud is calculated. If the MSE is less than a preset convergence error threshold (e.g., ...), the calculation is performed. If the iteration count reaches the preset maximum number of iterations (e.g., 50 times), the iteration stops, and the final accumulated rotation matrix (3×3 matrix) and translation vector (3×1 vector) are used as the determined spatial transformation parameters.

[0035] Finally, rigid body coordinate transformation and pose mapping operations are performed based on the spatial transformation parameters and the preset standard grasping pose template. The standard grasping pose template is preset based on the physical properties of the object. Its setting is usually based on the geometric centroid position and surface normal vector distribution of the object. The area above the centroid and with the highest surface flatness is selected as the suction cup adsorption point, or the area where the parallel surface spacing meets the gripper stroke is selected as the gripping point. This pose is fixed relative to the standard model coordinate system. After obtaining the spatial transformation parameters, the system performs a rigid body transformation with pure mathematical operations, that is, performs matrix multiplication operation between the calculated rotation matrix and the local coordinate vector in the standard grasping pose template to obtain the rotated coordinate vector. Then, the rotated coordinate vector is vector-added with the calculated translation vector to obtain the absolute three-dimensional grasping pose of the object in the current world coordinate system.

[0036] It should be noted that the preset feature similarity threshold is set based on the classifier's performance metrics. Typically, before system deployment, a test set containing positive samples (same type of items) and negative samples (different type of items) is constructed, a precision-recall curve (PR curve) is plotted, and the similarity value corresponding to the equilibrium point (e.g., 0.85) is selected as the threshold, balancing recognition accuracy and recall. The preset convergence error threshold and preset maximum number of iterations are common hyperparameters of the ICP algorithm. The error threshold is usually set according to the repeatability accuracy of the robotic arm (it can be set to 1 / 10 of the repeatability accuracy of the robotic arm's end effector, such as 0.1mm), while the maximum number of iterations is set according to the system's real-time requirements to prevent computational blocking in the event of non-convergence. It can be set to 100 iterations to meet real-time requirements (single registration time <50ms) while ensuring registration accuracy.

[0037] It is worth noting that if the iterative nearest point algorithm fails to converge to below the preset convergence error threshold within the preset maximum number of iterations, or if the calculated grasping pose is found to be unreachable by the robotic arm after inverse kinematics check, the system determines that the object positioning has failed. At this time, the system will record the candidate category of the object and the reason for failure, generate a "positioning anomaly" alarm, and may send it to the same review area as "unidentified object" according to the strategy, or try to generate a conservative, possibly non-optimal grasping pose based on the object bounding box and stacking height map.

[0038] In step S15, based on the confirmed item category and the 3D grasping pose, combined with the item stacking height map, sorting instructions based on spatial volume and gradient are generated and prioritized to obtain an optimized sorting task queue, including: Based on the confirmed item category and the three-dimensional grasping pose, an initial sorting instruction sequence is generated, and the initial sorting instruction sequence is mapped to the item stacking height map to obtain the mapping result; Based on the mapping results, local spatial morphology analysis is performed to calculate and obtain the spatial volume and gradient values ​​of the stacked area. Based on the space occupancy volume value and the stacking area gradient value, a multi-factor weighted score and queue reordering are performed to obtain an optimized sorting task queue.

[0039] In one implementation, an initial sorting instruction sequence is first generated based on the confirmed item category and the 3D grasping pose. The system then uses calibrated camera intrinsic and extrinsic parameter matrices to construct a mapping relationship from the world coordinate system to the heightmap image coordinate system, converting the 3D physical coordinates (X, Y) of the center point of each item in the instruction sequence into 2D pixel coordinates (u, v) on the heightmap plane. Subsequently, using these pixel coordinates as the geometric center, a corresponding rectangular region of interest (ROI) is extracted on the heightmap according to the ratio of the item's physical size to the map resolution, yielding the mapping result.

[0040] Next, based on the mapping results, local spatial morphology analysis is performed to calculate and obtain the space-occupied volume value and the gradient value of the stacked area. Specifically, each pixel in the rectangular region of interest is traversed, and its grayscale value (which directly corresponds to the physical height) is read. The grayscale values ​​of all pixels are summed, and the sum is multiplied by the actual physical base area represented by a single pixel to quantify the current stacked volume of the item.

[0041] The calculation process for the gradient values ​​in the stacked region is as follows: First, a convolution operation based on the Sobel operator is performed within the rectangular region of interest. This operation is achieved by moving a 3×3 sliding window pixel by pixel within the region. For each central pixel covered by the window, the grayscale change rate in the horizontal and vertical directions is calculated. The specific process for calculating the grayscale change rate in the horizontal direction is to multiply the three pixel values ​​in the right column of the window by coefficients 1, 2, and 1 respectively, multiply the three pixel values ​​in the left column by coefficients -1, -2, and -1 respectively, and set the coefficient in the middle column to 0. The results of these nine multiplications are then added together to obtain the brightness of the point in the horizontal direction. The specific process for calculating the vertical grayscale change rate is as follows: multiply the three pixel values ​​in the lower row of the window by coefficients 1, 2, and 1 respectively, multiply the three pixel values ​​in the upper row by coefficients -1, -2, and -1 respectively, and set the coefficient for the middle row to 0. Add these nine products together to obtain the brightness difference value of the point in the vertical direction. Then, add the square of the brightness difference value in the horizontal direction to the square of the brightness difference value in the vertical direction, and take the square root of the sum to obtain the gradient magnitude of the center pixel. Finally, calculate the arithmetic mean of the gradient magnitudes of all pixels in the region as the gradient value of the stacked region, which reflects the flatness of the stacked surface.

[0042] Finally, based on the space-occupied volume value and the stacking area gradient value, a multi-factor weighted scoring and queue reordering operation is performed to obtain an optimized sorting task queue. The specific scoring calculation logic adopts a linear weighted model: subtracting the minimum volume of the current batch from the space-occupied volume value and dividing by the volume range yields a normalized volume score; subtracting the minimum gradient of the current batch from the stacking area gradient value and dividing by the gradient range yields a normalized gradient score; subtracting this normalized gradient score from 1 yields a flatness score; finally, multiplying the normalized volume score by a preset volume weight coefficient and adding the flatness score multiplied by a preset flatness weight coefficient gives the grasping priority score; the system then reorders the instruction sequence according to this score from high to low.

[0043] It should be noted that the preset volume weight coefficient and preset flatness weight coefficient are set based on the warehousing operation strategy, and the sum of their values ​​is strictly equal to 1. By analyzing historical sorting data or simulations, the sorting success rate and efficiency under different combinations of (volume, gradient) can be analyzed. Multi-objective optimization methods (such as Pareto front analysis) can be used to determine one or more recommended combinations of weight coefficients for operators to choose according to real-time tasks. For example, under the "prioritize clearing high stacks" strategy, the volume weight coefficient is set to 0.6 to 0.8, which means that the higher the height and the larger the volume of the item, the higher the priority. Under the "fragile item anti-tipping" strategy, the flatness weight coefficient is set to 0.6 to 0.8, which means that the flatter the surface (i.e., the smaller the gradient value) of the item, the higher the priority. The Sobel operator uses a "1, 2, 1" weighting coefficient, a classic setting in the field of image processing. Compared with simple average difference, this coefficient gives higher weight to the central neighborhood pixels, which can effectively smooth noise and enhance the robustness of edge detection. The calculation process is essentially a second-order differential approximation of the spatial surface on a discrete digital image to accurately capture the tilt trend of the stacked object surface.

[0044] In step S16, based on the optimized sorting task queue and combined with the environmental obstacle point cloud data, adaptive trajectory planning and dynamic conflict resolution are performed to obtain a conflict-free control sequence, including: Based on the optimized sorting task queue, the basic motion trajectory is obtained by using a preset analytical inverse kinematics solver to calculate the basic trajectory in joint space. Based on the basic motion trajectory and the environmental obstacle point cloud data, directional bounding box construction and geometric overlap detection are performed to obtain conflict joint configuration parameters; Based on the conflict joint configuration parameters, local path replanning and trajectory smoothing correction are performed to obtain a conflict-free control sequence.

[0045] In one implementation, the basic trajectory calculation in joint space is first performed using a pre-set analytical inverse kinematics solver based on the target 3D grasping pose in the optimized sorting task queue. Specifically, based on the Denavit-Hartenberg (DH) parameter model of the robotic arm, a homogeneous transformation matrix is ​​constructed for each joint i relative to the previous joint i-1. This matrix is ​​obtained by sequentially multiplying the rotation matrix of the joint angle around the Z-axis, the translation matrix of the link offset along the Z-axis, the translation matrix of the link length along the X-axis, and the rotation matrix of the link torsion angle around the X-axis. The homogeneous transformation matrices of all joints of the robotic arm are sequentially multiplied to establish the total transformation equation from the base coordinate system to the end effector, and this total transformation equation is set to be equal to the target pose matrix. The system of equations is solved using algebraic elimination or analytical methods, and the joint angle values ​​that satisfy the target pose are derived in reverse. Subsequently, a fifth-order polynomial interpolation algorithm is used to generate the trajectory, that is, a fifth-order function of time is constructed. Using the six boundary conditions of joint position, joint velocity, and joint acceleration at the start and end times, a system of linear equations containing six unknown coefficients is constructed. Solving the system of equations yields the polynomial coefficients, thereby generating a basic motion trajectory that is continuous and smooth in time.

[0046] Next, based on the basic motion trajectory and the environmental obstacle point cloud data, directional bounding box construction and geometric overlap detection are performed. Specifically, firstly, the geometric center coordinates of all points in the environmental obstacle point cloud data (i.e., the arithmetic mean of all point coordinates) are calculated. The geometric center coordinates are then subtracted from the coordinates of each point to obtain decentralized coordinates. A 3×3 covariance matrix is ​​constructed, where each element is obtained by summing the products of the decentralized points on their corresponding two coordinate axis components and dividing by the total number of points in the point cloud (e.g., the element in the first row and second column is the average of the products of the X and Y coordinate components of all points). The eigenvalues ​​and corresponding eigenvectors of the covariance matrix are then solved using the Jacobi iteration method or singular value decomposition method, and the maximum eigenvalue is selected. The three unit eigenvectors corresponding to the three eigenvalues ​​are used as the three principal directional axes (i.e., length, width, and height) of the bounding box. All original point cloud data are projected onto these three principal directional axes respectively, and the dot product of the point coordinate vector and the unit vector of the principal directional axis is calculated. The maximum and minimum values ​​of all projected values ​​are found on each principal directional axis. Half of the sum of the maximum and minimum values ​​on each principal directional axis is multiplied by the unit vector of that axis. The calculation results of the three axes are then added together and the geometric center coordinates are added to obtain the center position of the bounding box. The difference between the maximum and minimum values ​​on each principal directional axis is used as the dimension length of the bounding box in that direction.

[0047] Subsequently, geometric overlap detection and overlap volume calculation are performed. The system uses the Separating Axis Theorem (SAT) to detect whether the robotic arm model intersects with the obstacle bounding box. If they intersect, the volume of the overlapping part is calculated. This is achieved by using a polyhedron trimming algorithm, taking the plane containing each face of the robotic arm bounding box as the trimming plane, and sequentially cutting the geometry of the obstacle bounding box, retaining the vertex sequence located inside the trimming plane, and generating a new convex polyhedron composed of intersecting vertices. This convex polyhedron is then decomposed into several tetrahedrons (for example, triangulation faces connecting each surface with a vertex inside the polyhedron). The volume of each tetrahedron is calculated using the determinant formula, which is one-sixth of the absolute value of the product of the three vectors formed by the four vertices of the tetrahedron. Finally, the volume values ​​of all tetrahedrons are added together to obtain the final volume value of the overlapping part. If this volume value is non-zero and exceeds the preset collision tolerance threshold, the joint angle at the current moment is marked as a conflict joint configuration parameter.

[0048] Finally, based on the conflict joint configuration parameters, local path replanning and trajectory smoothing correction operations are performed to obtain a conflict-free control sequence. This process employs the Rapid Expanding Random Tree (RRT*) algorithm. Within the joint configuration space, random sampling is performed, starting from the collision-free point before the conflict and ending at the collision-free point after the conflict. The Euclidean distance between the sampled point and the nearest node in the search tree is calculated. Under the premise that the connecting path does not collide, the node with the smallest path cost (i.e., distance) is selected as the parent node, and the sampled point is added to the tree structure. Nodes in the neighborhood of the new node are searched. If the path cost to reach a neighboring node through the new node is smaller, the parent node connection relationship is changed by rewiring. Finally, the optimal obstacle avoidance path connecting the start and end points is extracted, and the inflection points of the new path are smoothed again using the aforementioned fifth-order polynomial interpolation algorithm, generating a conflict-free control sequence that can directly drive the servo motor. Furthermore, if the RRT* algorithm fails to find a conflict-free path connecting the start and end points within the preset maximum number of search iterations (e.g., 2000 times) or the maximum computation time (e.g., 500ms), the system will determine that the current path planning task has failed. At this point, the system executes a 'degradation waiting strategy', controlling the robotic arm to maintain its current posture and pausing its movement. At the same time, it sends a 'congestion alarm' signal to the host computer, waiting for environmental obstacles (such as other mobile AGVs) to leave the collision area before re-triggering the adaptive trajectory planning step; or after multiple replanning failures, it switches to a preset safe retraction trajectory, withdrawing the robotic arm to a safe zero position to prevent collision accidents caused by forced planning.

[0049] It is worth noting that the conflict-free control sequence consists of a series of time-discrete spatial configuration points of the robotic arm joints (i.e., the angle value of each joint) and their corresponding timestamps, as well as optional joint velocity and acceleration information. This sequence can be sent to the servo driver of the robotic arm through a real-time communication interface (such as EtherCAT), and the driver performs interpolation and closed-loop control to accurately execute the planned collision-free trajectory.

[0050] It should be noted that the physical parameters such as link length and torsion angle in the DH parameter model must be strictly set according to the laser tracker calibration data at the time of the robotic arm's manufacture to eliminate manufacturing tolerances. The preset collision tolerance threshold is usually set to... to This value is determined based on the requirements for collision detection sensitivity of collaborative robots in industrial robot safety standards (such as ISO 10218-1:2011) and in conjunction with the measurement accuracy specifications of on-site point cloud acquisition equipment (such as LiDAR).

[0051] Specifically, considering that the sensor may have ±5mm of random Gaussian noise in the depth direction, which will generate a non-physical 'noise volume' at the edge of the bounding box it constructs in 3D space, this can be addressed by continuously acquiring multiple frames of point cloud data in a static scene, calculating the variance of the depth value for each spatial voxel, identifying voxels with variances greater than a specific threshold as noise, and calculating the total volume of all noise voxels. During the system cold start phase, technicians need to acquire unloaded noise point clouds in a static environment and calculate their envelope volume, setting 1.2 to 1.5 times this volume as the collision tolerance threshold. In the fifth-order polynomial interpolation, the velocity and acceleration boundary conditions are typically set to 0 at the start and end points, while intermediate connection points are set to continuous values ​​for the preceding and following trajectory segments to ensure the smooth operation of the robotic arm and prevent motor overload or mechanical vibration caused by sudden acceleration changes (infinite jerk). The formula for calculating the mixed product in the overlapping volume calculation is a standard method for calculating the volume of a tetrahedron in analytical geometry, which can accurately quantify the severity of collisions.

[0052] In summary, this invention achieves accurate identification and conflict-free, efficient sorting in complex stacking scenarios by using AI visual feature extraction and occlusion processing, integrating stacking height maps to optimize grasping priority, and combining real-time trajectory replanning technology.

[0053] Reference Figure 2 The second embodiment of the present invention provides an intelligent warehouse item sorting system based on AI vision, comprising: The data acquisition and enhancement module is used to acquire raw visual information in the warehouse scenario, and to acquire the stacking height map of items and point cloud data of environmental obstacles. The raw visual information is then processed for noise removal and contrast enhancement to obtain a preliminary clear image. The feature extraction and correction module is used to perform multi-scale visual feature extraction and illumination normalization channel construction based on the preliminary clear image to obtain the correction spatial feature tensor. An anti-occlusion processing module is used to perform spatial consistency analysis and generate an occlusion mask based on the corrected spatial feature tensor, and to use the occlusion mask to perform weighted filtering and feature recombination on the corrected spatial feature tensor to obtain an anti-occlusion stable feature vector. The identification and localization module is used to perform high-dimensional spatial projection matching based on the anti-occlusion stable feature vector and the preset item feature library to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, standard model retrieval and rigid body transformation calculation are performed to obtain the confirmed item category and three-dimensional grasping pose. The task scheduling optimization module is used to generate sorting instructions and prioritize them based on spatial volume and gradient, according to the confirmed item category and the three-dimensional grasping pose, combined with the item stacking height map, to obtain an optimized sorting task queue. The motion control and obstacle avoidance module is used to perform adaptive trajectory planning and dynamic conflict resolution based on the optimized sorting task queue and the environmental obstacle point cloud data, so as to obtain a conflict-free control sequence.

[0054] It should be noted that the AI ​​vision-based intelligent warehouse item sorting system provided in this embodiment of the invention is used to execute all the process steps of the AI ​​vision-based intelligent warehouse item sorting method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0055] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a cold-plate intelligent processing control program. When the processor executes the computer program, it implements the steps described in the various embodiments of the cold-plate intelligent processing control method, for example... Figure 1 The step S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above system embodiments, such as the feature extraction and correction module.

[0056] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0057] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0058] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0059] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0060] If the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0061] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0062] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A smart warehouse item sorting method based on AI vision, characterized in that, include: The raw visual information in the warehouse scene is acquired, along with the stacking height map of the items and the point cloud data of environmental obstacles. The raw visual information is then processed for noise removal and contrast enhancement to obtain a preliminary clear image. Based on the preliminary clear image, multi-scale visual feature extraction and illumination normalization channel construction are performed to obtain the corrected spatial feature tensor. Based on the corrected spatial feature tensor, spatial consistency analysis is performed and an occlusion mask is generated. The corrected spatial feature tensor is then weighted, filtered, and reconstructed using the occlusion mask to obtain an anti-occlusion stable feature vector. Based on the anti-occlusion stable feature vector and the preset item feature library, high-dimensional spatial projection matching is performed to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, standard model retrieval and rigid body transformation calculation are performed to obtain the confirmed item category and three-dimensional grasping pose. Based on the confirmed item category and the three-dimensional grasping pose, combined with the item stacking height map, sorting instructions based on spatial volume and gradient are generated and priority scheduling is performed to obtain an optimized sorting task queue. Based on the optimized sorting task queue, and combined with the environmental obstacle point cloud data, adaptive trajectory planning and dynamic conflict resolution are performed to obtain a conflict-free control sequence.

2. The intelligent warehouse item sorting method based on AI vision according to claim 1, characterized in that, The process involves acquiring raw visual information from a warehouse setting, obtaining stacked item height maps and environmental obstacle point cloud data, and then performing noise removal and contrast enhancement on the raw visual information to obtain a preliminary clear image, including: Use warehouse cameras to collect raw visual information; Based on the original visual information, adaptive illumination analysis and noise localization are performed, and the standard deviation of brightness in the local area is calculated. If it exceeds the preset standard deviation threshold of illumination, median filtering is performed to obtain a denoised visual image. Based on the denoised visual image, contrast enhancement and edge sharpening are performed, and pixel intensity concentration is calculated. If it exceeds a preset pixel intensity concentration threshold, contrast-limited adaptive histogram equalization and edge sharpening calculations are performed to obtain a preliminary clear image. Depth data of the warehouse scene is collected from depth sensing devices, and the data is mapped and converted into formats to generate item stacking height maps and environmental obstacle point cloud data, respectively.

3. The intelligent warehouse item sorting method based on AI vision according to claim 1, characterized in that, The step of extracting multi-scale visual features and constructing illumination normalization channels based on the preliminary clear image to obtain a corrected spatial feature tensor includes: Based on the preliminary clear image, a pre-trained convolutional neural network model is used to perform deep feature mining and texture descriptor extraction to obtain a basic feature map; Based on the aforementioned basic feature map, feature pyramid construction and multi-scale fusion are performed, and feature scale unification is achieved through upsampling mapping to obtain a multi-scale fused feature map. Based on the multi-scale fusion feature map, channel brightness normalization and viewpoint affine transformation correction are performed to obtain the correction space feature tensor.

4. The intelligent warehouse item sorting method based on AI vision according to claim 1, characterized in that, The step involves performing spatial consistency analysis and generating an occlusion mask based on the corrected spatial feature tensor, and then using the occlusion mask to perform weighted filtering and feature reorganization on the corrected spatial feature tensor to obtain an anti-occlusion stable feature vector, including: Based on the corrected spatial feature tensor, local response anomaly detection is performed, the feature response value and the neighborhood mean are calculated, and the absolute value of the difference between the feature response value and the neighborhood mean is calculated. The region where the feature response value is lower than the neighborhood mean and the absolute value of the difference exceeds a preset response deviation threshold is determined as the initial interference region. Based on the corrected spatial feature tensor and the initial interference region, the pre-trained semantic segmentation network model is input to perform accurate occlusion region segmentation, resulting in an accurate occlusion region segmentation map. Based on the precise occlusion region segmentation map and the corrected spatial feature tensor, feature mask reverse filtering and global aggregation and recombination are performed to obtain an anti-occlusion stable feature vector.

5. The intelligent warehouse item sorting method based on AI vision according to claim 1, characterized in that, The process involves performing high-dimensional spatial projection matching based on the anti-occlusion stable feature vector and a pre-set item feature library to obtain a matching similarity. If the matching similarity exceeds a preset feature similarity threshold, standard model retrieval and rigid body transformation calculation are performed to obtain the confirmed item category and 3D grasping pose, including: Based on the anti-occlusion stable feature vector and the pre-set item feature library, vector space mapping and cosine similarity calculation are performed to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, then a standard 3D model is retrieved and the feature points are aligned to obtain spatial transformation parameters. Based on the spatial transformation parameters and the preset standard grasping pose template, rigid body coordinate transformation and pose mapping operations are performed to obtain the confirmed item category and three-dimensional grasping pose.

6. The intelligent warehouse item sorting method based on AI vision according to claim 1, characterized in that, The step of generating and prioritizing sorting instructions based on spatial volume and gradient, according to the confirmed item category and the 3D grasping pose, combined with the item stacking height map, to obtain an optimized sorting task queue includes: Based on the confirmed item category and the three-dimensional grasping pose, an initial sorting instruction sequence is generated, and the initial sorting instruction sequence is mapped to the item stacking height map to obtain the mapping result; Based on the mapping results, local spatial morphology analysis is performed to calculate and obtain the spatial volume and gradient values ​​of the stacked area. Based on the space occupancy volume value and the stacking area gradient value, a multi-factor weighted score and queue reordering are performed to obtain an optimized sorting task queue.

7. The intelligent warehouse item sorting method based on AI vision according to claim 1, characterized in that, The step of performing adaptive trajectory planning and dynamic conflict resolution based on the optimized sorting task queue and the environmental obstacle point cloud data to obtain a conflict-free control sequence includes: Based on the optimized sorting task queue, the basic motion trajectory is obtained by using a preset analytical inverse kinematics solver to calculate the basic trajectory in joint space. Based on the basic motion trajectory and the environmental obstacle point cloud data, directional bounding box construction and geometric overlap detection are performed to obtain conflict joint configuration parameters; Based on the conflict joint configuration parameters, local path replanning and trajectory smoothing correction are performed to obtain a conflict-free control sequence.

8. An intelligent warehouse goods sorting system based on AI vision, characterized in that, include: The data acquisition and enhancement module is used to acquire raw visual information in the warehouse scenario, and to acquire the stacking height map of items and point cloud data of environmental obstacles. The raw visual information is then processed for noise removal and contrast enhancement to obtain a preliminary clear image. The feature extraction and correction module is used to perform multi-scale visual feature extraction and illumination normalization channel construction based on the preliminary clear image to obtain the correction spatial feature tensor. An anti-occlusion processing module is used to perform spatial consistency analysis and generate an occlusion mask based on the corrected spatial feature tensor, and to use the occlusion mask to perform weighted filtering and feature recombination on the corrected spatial feature tensor to obtain an anti-occlusion stable feature vector. The identification and localization module is used to perform high-dimensional spatial projection matching based on the anti-occlusion stable feature vector and the preset item feature library to obtain the matching similarity. If the matching similarity exceeds the preset feature similarity threshold, standard model retrieval and rigid body transformation calculation are performed to obtain the confirmed item category and three-dimensional grasping pose. The task scheduling optimization module is used to generate sorting instructions and prioritize them based on spatial volume and gradient, according to the confirmed item category and the three-dimensional grasping pose, combined with the item stacking height map, to obtain an optimized sorting task queue. The motion control and obstacle avoidance module is used to perform adaptive trajectory planning and dynamic conflict resolution based on the optimized sorting task queue and the environmental obstacle point cloud data, so as to obtain a conflict-free control sequence.

Images