A warehouse inventory counting system and method using image recognition

Abstract

The application relates to the technical field of goods inventory, in particular to an image-identified warehouse article counting system and method. The image-identified warehouse article counting method comprises the following steps: collecting a pre-picking image of a to-be-inventoried goods pile from a first visual angle, and constructing a goods position initial belief map with uncertainty annotation based on the pre-picking image; when the goods in the goods pile are removed from the goods position and are in a carrying state, for the goods position identified as high uncertainty in the initial belief map, a local area image of the corresponding goods is collected and instance segmentation is carried out to obtain local live data; the initial belief map is updated through a neural network fusion module, and the state information of the hidden goods position is completed; and the total number of goods is counted according to the updated belief map. Through the technical scheme of pile type classification delimitation, belief map quantitative representation and neural fusion reasoning, the application realizes goods counting in a complex scene.

Description

Technical Field

[0001] This application relates to the field of inventory technology, and more specifically, to a warehouse item counting system and method based on image recognition. Background Technology

[0002] Against the backdrop of the rapid development of intelligent manufacturing and modern logistics industries, automated storage and retrieval systems (AS / RS) have become core infrastructure for warehousing systems in various industries, including manufacturing, e-commerce retail, and cold chain logistics, due to their core advantages of high space utilization, superior storage and retrieval efficiency, and the ability to achieve fully automated control throughout the entire process. Inventory counting, as a core component of AS / RS operation and management, is crucial for ensuring accurate inventory records, optimizing warehousing scheduling strategies, and mitigating inventory management risks. Its efficiency and accuracy directly determine the overall operational efficiency and management sophistication of the warehousing system.

[0003] Traditional warehouse inventory counting relies on manual counting and verification of each location, which is not only costly in terms of manpower and time, but also extremely inefficient. Furthermore, in the context of high-bay racking and dense storage in automated warehouses, manual counting poses significant safety hazards and is prone to errors due to human negligence, failing to meet the high-frequency, high-precision inventory requirements of modern warehousing. With the rapid development of machine vision and deep learning technologies, image recognition-based automated inventory counting solutions are gradually becoming the industry mainstream. These solutions acquire images of stacked goods using image acquisition devices deployed on stacker cranes, and combine these with visual algorithms such as object detection and image segmentation to automatically count goods, significantly reducing reliance on manual labor and improving efficiency. However, existing image recognition-based warehouse inventory counting solutions can only make simple state inferences about the location of obscured goods within the stack based on manually preset geometric rules and basic stack type priors, affecting the final inventory counting accuracy. Summary of the Invention

[0004] The purpose of this application is to address the aforementioned deficiencies in the prior art by providing an image recognition method and system for counting warehouse items, solving the core problem of low counting accuracy in complex occlusion scenarios in the prior art, and realizing automated warehouse inventory.

[0005] To achieve the above objectives, this application provides the following technical solution: In a first aspect, this application provides a warehouse item counting method based on image recognition, comprising the following steps: S1. Acquire images of the stack of goods to be inventoried from a first-person perspective before picking up the goods, and construct an initial belief map of the goods location with uncertainty annotations based on the images before picking up the goods. S2. When the goods in the stack are moved out of the storage location and are in a transport state, for the goods positions marked as high uncertainty in the initial belief map, local area images of the corresponding goods are collected and instance segmented to obtain local real-time data. S3. Acquire images of the stack of goods after placement from a second perspective and perform target detection to obtain the second perspective detection results; input the initial belief map, the local real-time data, and the second perspective detection results into a pre-trained neural network fusion module; update the initial belief map through the neural network fusion module and complete the state information of the occluded goods position; S4. Based on the updated belief map, count the total number of goods.

[0006] Furthermore, in step S2, a temporal visual fingerprint is extracted and generated for each instance segmented goods; in step S3, the temporal visual fingerprint corresponding to each goods is input into the neural network fusion module, and during the inference process, the temporal visual fingerprint is used to determine whether each goods belongs to the same physical goods, and to correct the goods identity association information.

[0007] Furthermore, the temporal visual fingerprint is generated based on local key points and descriptors extracted from images of the outer surface of the goods, or packaging texture and printing pattern features encoded by a lightweight convolutional neural network.

[0008] Furthermore, in step S1, the specific process of constructing the initial belief map of cargo location with uncertainty annotations is as follows: The image before pickup is classified into stack types, and the spatial grid division rules for the cargo location corresponding to the stack are determined based on the standard stack types obtained from the classification. Then, visible cargo targets are detected in the image before pickup, and an initial belief map of cargo location with uncertainty annotation is constructed based on the target detection results and the spatial grid division rules.

[0009] Furthermore, the spatial grid division rule is as follows: Based on the number of stacking layers and the number of goods per column of the standard stack type obtained from the classification, a two-dimensional grid of the same size is generated that corresponds one-to-one with the physical storage space of the stack. Each grid cell uniquely corresponds to an independent standard goods storage location in the stack. The row and column arrangement of the grid and the total number of cells match the stacking arrangement of the goods and the rated total number of goods for that standard stack type.

[0010] Furthermore, in step S1, the initial belief map represents the stack space in the form of a grid, with each grid cell associated with a multi-dimensional state vector. The state vector includes at least: an initial probability value of the presence of goods at the location generated based on the target detection results, a list of other goods identifiers that may obscure the location, an estimated value representing the degree of obstruction at the location, and an uncertainty label value used to quantify the reliability of the location state judgment.

[0011] Furthermore, in step S2, the local real-time data serves as a verification benchmark for correcting the state of the initial belief map in step S3. The local real-time data includes at least the actual outline dimensions, surface features, and integrity information of the corresponding goods.

[0012] Furthermore, in step S3, during the inference process, the neural network fusion module first uses the local real-time data as a benchmark to correct the probability value and uncertainty label value of the corresponding position in the initial belief map, and then combines the detection results from the second perspective to perform fusion calculation.

[0013] Furthermore, in step S3, the neural network fusion module is a graph neural network based on an attention mechanism, and the fusion calculation process of the neural network fusion module is performed based on the stacking occlusion pattern and physical structure prior learned during pre-training.

[0014] Secondly, this application provides an image recognition warehouse item counting system for implementing the aforementioned image recognition warehouse item counting method, the system comprising: The image acquisition unit is used to acquire images of the stack or goods from corresponding perspectives in three stages: before picking up the stack, during the transport of goods, and after the goods are placed. The initial perception and reasoning unit is used to construct an initial belief map of the cargo location with uncertainty annotations based on the pre-pickup image; The local sensing unit is used to perform instance segmentation of local areas of cargo images and generate local real-time data; The neural fusion reasoning unit has a built-in pre-trained neural network fusion module, which is used to perform fusion reasoning of multi-source data, update the cargo position belief map and complete the state information of the occluded position. The decision output unit is used to complete the total number of goods statistics and inventory results output based on the updated belief map.

[0015] Compared with the prior art, this application has the following advantages: 1. This application achieves cargo counting in complex scenarios through a technical solution of stack type classification and delimitation, belief graph quantization representation, and neural fusion reasoning. First, stack type classification determines spatial grid division rules that correspond one-to-one with the physical space of the cargo stack, providing physical anchors for cargo location detection and state representation, avoiding misalignment of detection results. Then, a multi-dimensional state vector belief graph with occlusion degree and uncertainty annotations is constructed, achieving probabilistic quantization representation of the full position state of the cargo stack. Finally, through a reasoning process of first correcting local real-world data and then fusing second-view detection results, combined with stack occlusion patterns and physical structure priors learned by a graph neural network pre-trained based on an attention mechanism, the state completion of occluded positions is achieved. Compared to existing technologies that rely on manually preset rules, this solution improves the counting accuracy in complex occlusion scenarios.

[0016] 2. This application solves the problem of duplicate counting and missed detection caused by cargo displacement and viewpoint differences through a cross-stage identity association verification design based on temporal visual fingerprints, ensuring the consistency and uniqueness of inventory counting. In the unobstructed scenario of cargo transportation, a unique temporal visual fingerprint is generated for each individual cargo after instance segmentation. During the neural network fusion inference process, fingerprint matching is used to verify the identity association of cargo detection results at different stages before pickup and after delivery, and between spatially adjacent cargo, to determine whether the detection target is the same physical cargo. This corrects the identity association errors caused by slight cargo displacement, occlusion changes, and viewpoint differences, thereby completely avoiding counting errors caused by changes in spatial position. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating the method described in this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand this application, the technical solutions of this application will be clearly and completely described below in conjunction with embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0019] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0021] The following detailed explanation of specific implementation methods further illustrates this application: This embodiment takes the inventory of a standard 6-layer, 5-column, 30-item stack commonly used in warehouses as an example. The inventory process for other standard stack types can be adjusted accordingly based on this embodiment. The image acquisition hardware in this embodiment is installed on the stacker crane platform and consists of 3 global shutter fixed-focus industrial cameras, a matching ring light source, and a gigabit network image acquisition card. All cameras are calibrated for internal parameters and stereo calibration through a camera calibration board.

[0022] The calibration employs the Zhang Zhengyou calibration method. During intrinsic parameter calibration, each camera acquires at least 20 images of the calibration board at different heights, angles, and distances. The intrinsic parameter matrix, radial distortion coefficient, and tangential distortion coefficient of the camera are solved using a calibration algorithm. Stereo calibration involves acquiring synchronous calibration images from different cameras in the same scene and solving for the extrinsic parameter matrices (rotation matrix R and translation vector T) between the cameras to complete the spatial pose calibration between them. The acceptable calibration results are: reprojection error ≤ 0.5 pixels for single-camera intrinsic parameter calibration and ≤ 1.0 pixel for dual-camera stereo calibration. Cameras that do not meet these standards must be recalibrated until the accuracy requirements are met. The acquisition and updating of the extrinsic parameter matrix are synchronized with the monthly equipment maintenance cycle of the stacker crane. When a collision occurs on the stacker crane or the camera installation position changes, stereo calibration must be re-performed immediately. Cameras include: Camera No. 1 (dedicated to first-person view acquisition) is installed in the center of the front of the loading platform, with the shooting direction perpendicular to the front of the stack, and is used to acquire a front view image of the stack before picking up the goods. Camera No. 2 (dedicated to local area acquisition) is installed at the center of the top of the loading platform, with the shooting direction vertically downward. It is used to acquire unobstructed local images of the cargo in the transport state. The shooting direction of Camera No. 2 can be adjusted within the range of 0°-90° via an electric pan-tilt head (0° is vertically downward, and 90° is horizontally forward). Camera No. 3 (dedicated to second-view acquisition) is installed on the left side of the loading platform to capture side view images of the stack of goods after loading. All cameras are connected to the edge computing terminal, and the brightness of the matching ring light source can be automatically adjusted according to the ambient light in the warehouse to ensure image acquisition quality under different lighting conditions.

[0023] See Figure 1 Specifically, the implementation steps include the following: S1. Collect images of the stack of goods to be inventoried from a first-person perspective before picking up the goods, and construct an initial belief map of the goods location with uncertainty annotations based on the images before picking up the goods.

[0024] In practice, the warehouse WCS stacker crane control system first sends an inventory task command to the stacker crane, controlling the stacker crane to travel to the aisle coordinates corresponding to the location to be inventoried, raising and lowering the loading platform to the center height of the stack. After the stacker crane comes to a complete stop, camera 1 is triggered to capture a frontal image of the stack to be inventoried before picking up the goods. Three frames are acquired in a single acquisition, and the sharpness of each frame is calculated using the Laplacian variance algorithm. The frame with the highest sharpness is selected as the base image for subsequent processing to avoid image quality problems caused by motion blur and defocus.

[0025] Then, perform standardization preprocessing on the filtered base images, as follows: The first step is to perform Gaussian denoising using a 3×3 Gaussian filter kernel to eliminate salt and pepper noise and random noise in the image, smoothing image details while preserving the edge features of the goods. The second step is to use the pre-calibrated camera intrinsic parameter matrix and distortion coefficients to complete the radial and tangential distortion correction of the image, restore the true geometric size and proportion of the stack, and avoid mesh division errors caused by perspective distortion. The third step is to enhance contrast by limiting contrast adaptive histogram equalization (CLAHE), with the clipLimit parameter set to 2.0 and the grid size set to 8×8, to improve the image detail recognition in low-light and backlight environments and ensure the accuracy of subsequent detection and classification.

[0026] The preprocessed image is input into the pre-trained ResNet50 pallet classification network. The fully connected layers of the network output classification probabilities for four standard pallet types. The category with the highest probability is taken as the final classification result. In this embodiment, the network outputs a classification result of 6 layers, 5 columns, and 30 standard pallets. If the highest probability of the classification result is lower than 0.99 but greater than or equal to 0.8, it is determined to be a non-full pallet / slightly deviated pallet type. The subsequent spatial grid division and inventory process is performed based on the standard pallet type corresponding to the highest probability. If the highest probability of the classification result is lower than 0.8, it is determined to be a severely non-standard pallet type. A non-standard pallet type warning is sent to the WMS system, and the location is marked as a location requiring manual verification. The automatic inventory process is terminated to avoid counting errors caused by severely non-standard pallets.

[0027] Based on the 6-layer, 5-column, 30-piece standard stack type obtained from the classification, the corresponding spatial grid division rules are determined, and the specific execution process is as follows: The first step is to extract the overall outer contour of the stack using the Canny edge detection algorithm, and then fit the four outer boundary lines of the stack (top, bottom, left, and right) using the Hough line transform. The pixel coordinate range of the stack in the image is then calculated as follows: in, and These are the minimum and maximum pixel coordinates in the horizontal direction, respectively. and These are the minimum and maximum pixel coordinates in the vertical direction, respectively; The second step involves dividing the outer rectangle of the pallet into a 6-row, 5-column two-dimensional grid, based on the standard pallet type's 6 stacking layers and 5 columns of goods per column. The horizontal pixel size of each grid cell is... The vertical pixel size of a single grid cell is ; The third step is to assign a unique physical location number to each grid cell. The numbering rule is from bottom to top and from left to right. The bottom layer is numbered 1-5 from left to right, the second layer is numbered 6-10, and so on, with the top layer being numbered 26-30. Each grid cell uniquely corresponds to an independent standard goods storage location in the stack. The row and column arrangement of the grid and the total number of 30 cells are perfectly matched with the stacking arrangement of 30 standard stacks of goods and the rated total number of goods to be stored.

[0028] The pre-processed image before pickup is input into a pre-trained YOLOv8s object detection model. This model achieves mAP@0.5≥99% for goods detection on the test set. The confidence threshold during model inference is set to 0.5, and the non-maximum suppression (NMS) threshold is set to 0.45. The model outputs the pixel coordinates of the detection box for each visible item. The system calculates the category confidence score and the center pixel coordinates of the detection boxes. It then performs location mapping on the detection results, calculating the grid cell to which the center coordinates of each detection box belong, and binding the detection result to the corresponding grid cell to achieve the mapping from image detection results to the physical location of the goods. If multiple detection boxes exist within a single grid cell, the detection box with the highest confidence score is taken as the valid detection result for that grid cell.

[0029] Finally, based on the completed 6x5 two-dimensional grid, a 5-dimensional state vector is constructed for each grid cell, ultimately forming an initial belief map of the cargo location with uncertainty annotations. The specific calculation rules for each dimension are as follows: First dimension: Initial probability value, i.e., the initial probability that goods exist at this location, with a range of values. If the confidence level of the bounding box to the grid cell is ≥0.9 and the intersection-union ratio (IOU) between the bounding box and the grid cell is ≥0.8, the initial probability value is set to 0.95; if the confidence level of the bounding box is in the range of 0.5-0.9, or the IOU is in the range of 0.5-0.8, the initial probability value is set to 0.6; if the grid cell has no bounding box, the initial probability value is set to 0.5. The second dimension: Initial probability of no stock, i.e., the initial probability that there is no stock at this location, with a range of values. Each of these values ​​corresponds one-to-one with the initial probability value of having stock. The third dimension: the occlusion list, which records other cargo identifiers that may occlude the location. The depth value is calculated using a pinhole camera model based on a calibrated monocular camera, standard warehouse locations, and the fixed physical dimensions of the cargo. First, the baseline depth of the location is calculated using the known physical width of the standard pallet and the camera's calibrated pixel focal length. Then, the depth value corresponding to each B grid cell is determined based on the fixed arrangement of goods in a standard stack. Finally, the depth value of the cargo corresponding to the A detection frame is calculated using the physical dimensions of a standard single box of goods. A smaller depth value indicates a closer target to the camera. If the depth value of the center coordinates of the A detection frame is less than the depth value of the B grid cell, and the projection of the A detection frame onto the image covers more than 30% of the area of ​​the B grid cell, then the unique identifier of the cargo corresponding to the A detection frame is added to the occlusion list of the B grid cell. Fourth dimension: Occlusion estimate, representing the degree of occlusion at this location, with a range of values. The calculation method is the sum of the proportion of the occlusion area of ​​all detection boxes in the occlusion list to the grid cell, with a maximum value of 1 and a value of 0 if there is no occlusion. Fifth dimension: Uncertainty label value, quantifying the reliability of the positional state judgment, with a value range of... It is positively correlated with the estimated degree of occlusion, and the calculation formula is: in For values ​​marked as uncertain, This is an estimate of the degree of occlusion. The estimated degree of occlusion for a completely unobstructed location. =0, corresponding to an uncertainty label value of 0.05; estimated occlusion level at a completely occluded location. =1, corresponding to an uncertainty label value of 0.9.

[0030] The values ​​across all dimensions are interconnected, and the initial probability values ​​for the presence and absence of goods are complementary parameters. After constructing the state vectors of all grid cells, grid cells with uncertainty values ​​exceeding a preset threshold of 0.7 or occlusion estimation values ​​exceeding 0.5 are automatically marked as high-uncertainty locations for subsequent targeted acquisition and correction. After the initial perception and inference unit completes the marking of high-uncertainty locations, it synchronously generates a list of acquisition instructions containing the grid numbers, physical coordinates, and corresponding camera shooting parameters for all high-uncertainty locations. This list is sent in real-time to the image acquisition unit and the local perception unit via the MQTT protocol, serving as the trigger for subsequent local image acquisition and instance segmentation.

[0031] S2. When goods in the stack are moved out of their storage location and are in a transport state, for the goods locations marked as high uncertainty in the initial belief map, local area images of the corresponding goods are collected and instance segmented to obtain local real-time data.

[0032] In practice, after the initial belief map is constructed, a picking instruction is sent to the stacker crane through the WCS system. The stacker crane forks are extended and inserted into the fork holes at the bottom of the pallet, and the entire pallet of goods is moved out of the storage location and completely retracted into the loading platform, so that the goods are completely in the transport state of the stacker crane. At this time, the goods are completely detached from the rack, without the obstruction of rack columns or adjacent stacks, which is an unobstructed acquisition scenario. After the forks are retracted, the image is captured after 1 second of stillness, when the goods are completely still. The determination of whether goods have been moved out of their storage location and are in a transport state employs a dual confirmation mechanism of WCS system status feedback and hardware arrival signal. First, after the stacker crane forks complete the picking action and retract to the origin of the loading platform, the stacker crane PLC controller sends a "forks retracted to position" switch signal to the WCS system. Simultaneously, the WCS system sends a "goods transported to position" status command to the edge computing terminal. At the same time, the through-beam photoelectric sensor installed in the loading platform detects whether the pallet has completely entered the safe area of ​​the loading platform. When the sensor outputs an unobstructed high-level signal, the hardware-level arrival confirmation is completed. Only when the WCS system status feedback and the photoelectric sensor signal are satisfied simultaneously does the system determine that the goods are in a valid transport state, triggering subsequent local image acquisition actions to avoid invalid acquisition in abnormal scenarios such as incomplete picking or goods position deviation.

[0033] For the goods corresponding to the high uncertainty positions marked in step S1, determine the physical position of the goods on the pallet according to their grid number, control the lifting and lowering of the loading platform to the center height of the corresponding goods, and simultaneously adjust the angle of the motorized pan-tilt head of camera No. 2: when acquiring the top image, the pan-tilt head angle is set to 0° (vertically downward), when acquiring the front image, the pan-tilt head angle is set to 90° (horizontally forward), and when acquiring the side image, the pan-tilt head angle is set to 45° (obliquely to the side), triggering camera No. 2 to acquire unobstructed local area images of the goods; for each high uncertainty position, acquire 3 frames of local images at different lifting heights, covering the top, front, and side of the goods, ensuring that the printed patterns, textures, outlines, and other features of the goods packaging are completely acquired, and automatic focusing is performed during the acquisition process.

[0034] The acquired local images are then input into a pre-trained Mask2Former instance segmentation model. This model uses ResNet50 as the backbone network, configured with 100 query vectors, a 6-layer Transformer encoder, and a 6-layer Transformer decoder. It uses the official pre-trained weights from the COCO dataset as initial weights and fine-tunes them for warehouse goods scenarios. The fine-tuning dataset contains 20,000 local images of warehouse goods with different packaging types, lighting conditions, and shooting angles. The annotations include the segmentation mask, contour, and category information of the goods. During fine-tuning, the weights of the first three layers of the backbone network are frozen, the initial learning rate is set to 1e-5, the batch size is 8, and the training is iterated for 30 epochs using the AdamW optimizer. After fine-tuning, the model achieves a mask intersection-over-union (mIoU) ≥ 95% on the warehouse goods test set, with inference time per image ≤ 50ms. The model outputs the segmentation mask, contour pixel coordinates, and category information for each target item, removing interference from irrelevant areas such as the background and pallets. Based on the instance segmentation results, local real-world data for the corresponding goods is extracted as a high-confidence verification benchmark for subsequent belief map correction. The local real-world data specifically includes: Actual cargo outline dimensions: The actual pixel dimensions of the cargo are calculated by using the outer boundary of the segmentation mask. Combined with the intrinsic and extrinsic parameters of the camera calibration, the dimensions are converted into physical dimensions to verify whether they match the standard single box dimensions. Cargo surface features: Extract texture features, printed patterns, barcode or QR code features from cargo packaging for subsequent generation of temporal visual fingerprints; Cargo integrity information: By analyzing the integrity and outline continuity of the segmentation mask, we can determine whether there are any abnormalities such as damage, missing parts, deformation, or empty boxes in the cargo.

[0035] In this step, the local real-world data is real data obtained in an unobstructed scene, and its confidence weight is set to 1.0. It is the highest priority data in the entire process and is used first for error correction of the initial belief map.

[0036] S3. Acquire images of the stack of goods after placement from a second-person perspective and perform target detection to obtain the second-person perspective detection results; input the initial belief map, local real-time data, and second-person perspective detection results into the pre-trained neural network fusion module; update the initial belief map through the neural network fusion module and complete the state information of the position of the occluded goods.

[0037] In practice, after completing the local real-time data collection, the stacker crane forks are extended to smoothly place the entire pallet of goods back to the target location. After the forks are fully retracted, the pallet is left to stand still for 2 seconds until the goods are completely stable and without shaking. Then, the loading platform is raised and lowered to the center height of the pallet, triggering camera 3 to collect a side view image of the pallet after loading. The acquisition parameters and preprocessing procedures are consistent with the front view image in step S1 to ensure the uniformity of image quality from different perspectives.

[0038] The same preprocessing operation as S1 is then performed on the image after the goods are placed. The same YOLOv8s object detection model is input, and the output is the goods detection box, confidence score, and center coordinates from the side view. Based on the pre-calibrated extrinsic matrix of the camera and the hand-eye calibration results, the detection results from the side view are transformed into the pixel coordinate system of the front view and mapped onto the two-dimensional grid constructed in S1. This completes the binding of the second view detection results with the goods position grid, achieving spatial alignment of the dual-view detection results.

[0039] The initial belief map generated in step S1, the local real-world data obtained in step S2, and the second-view detection results obtained in this step are all input into the pre-trained neural network fusion module; if a temporal visual fingerprint is generated in step S2, the fingerprint data and matching results are simultaneously input into this module.

[0040] In this embodiment, the neural network fusion module is a graph neural network (GAT) based on the attention mechanism. Its specific network structure, pre-training process, and inference execution flow are as follows: Network structure definition: The network consists of 3 graph attention layers, each with 8 attention heads. The activation function is LeakyReLU, with a negative slope of 0.2 and a dropout rate of 0.3 to prevent overfitting. The output layer uses the Sigmoid activation function, outputting the "available" belief probability, "unavailable" belief probability, and updated uncertainty label value for each grid cell. The core computational logic of the graph attention layer is as follows: For the Layered graph attention layer, nodes The input features are ,in The input feature dimension is , and the output feature is . First, through linear transformation matrix Perform a unified mapping on all node features, and then use attention coefficients. compute nodes For nodes The influence weight is calculated using the following formula: LeakyReLU is a linear rectified activation function with leakage. middle This is a vector transpose operation. For the first The single-layer attention weight vector of the layered graph attention layer. For feature splicing operations, For nodes For nodes The original attention coefficient; Only on the node With nodes When graph edges exist, the calculation is performed, and the attention coefficients are finally normalized using the softmax function to obtain the final weights. : in, It is a natural exponential function; For nodes The set of all adjacent nodes; For normalized nodes For nodes The final attention weights; the final node The output feature is the weighted sum of the features of all neighboring nodes, which is then processed by the activation function. The Sigmoid activation function is used for non-linear mapping of the output features. Under the multi-head attention mechanism, the output features of the eight attention heads are concatenated, and the final output layer adopts an average aggregation method to ensure that the dimension of the output features matches the dimension of the node state vector.

[0041] Graph structure construction: The initial belief graph consists of 30 grid cells as 30 graph nodes, and the input feature of each node is the 5-dimensional state vector of the corresponding grid cell. The spatial adjacency relationship (adjacent to upper and lower layers, adjacent to left and right layers in the same layer) and occlusion relationship between cargo locations are used as graph edges to construct an undirected graph structure. The initial weight of the edge between spatially adjacent nodes is 0.5, and the initial weight of the edge between occlusion-related nodes is 0.8. The specific process for determining the initial values ​​of the aforementioned edge weights is as follows: First, based on the physical characteristics of cargo stacking, the influence of occlusion association on cargo state inference is greater than that of spatial adjacency. Therefore, the initial weight of occlusion association edges needs to be higher than that of spatial adjacency edges. Based on this, a grid search is performed on both edge weights within the range of 0.1-0.9 with a step size of 0.1. The optimization objective is to achieve the cargo state prediction accuracy on the validation set. Ultimately, when the initial weight of spatial adjacency edges is determined to be 0.5 and the initial weight of occlusion association edges to be 0.8, the model achieves the highest prediction accuracy and fastest convergence speed. Furthermore, this initial value exhibits optimal generalization performance under different stacking types and occlusion scenarios. During model training, the edge weights are adaptively updated with network iterations. The initial values ​​only provide a physical prior for network training. The adaptive update rule for edge weights is: During the model's forward inference, the edge weights are updated synchronously with the attention coefficient and node feature similarity. The update formula is: in, For the first Node at the next iteration With nodes Edge weights between them For the first Edge weights in the next iteration For nodes With nodes Cosine similarity of features; during the update process, upper and lower bound constraints are set for edge weights, and the weight range of spatially adjacent edges is [value missing]. The weight range of the occlusion-related edges is This avoids graph structure distortion caused by excessive weight updates.

[0042] Model pre-training process: The training dataset contains 10,000 sets of labeled stack data, covering four standard stack types: 6 layers, 5 columns, 30 items; 6 layers, 4 columns, 24 items; 8 layers, 4 columns, 32 items; and 6 layers, 6 columns, 36 items. It also includes 15% non-standard stacking scenarios (such as missing items, misaligned stacks, and tilted stacks) and 30% complex occlusion scenarios (such as foreground goods occlusion of 30%-100%, shelf column occlusion, and adjacent stack occlusion). The dataset images are under full illumination (0-1000 lux) in actual warehouse operations. Data was collected from multiple camera angles (front, side, and top views) under the slight vibration of a stacker crane starting and stopping. The dataset includes samples of goods with different packaging materials (corrugated cardboard boxes, plastic crates), different printed patterns, and different sizes to ensure scene representativeness and feature diversity. The training and validation sets are split in a 9:1 ratio to guarantee the generalization ability of the model. Each dataset includes a front view image, a side view image, an unoccluded partial image, manually labeled real cargo location and quantity, occlusion relationship annotations, and stack type information. The loss function... The weighted sum of cross-entropy loss and mean squared error loss is used for calculation, and the formula is as follows: in, For cross-entropy loss, The loss is the mean squared error; the optimizer uses AdamW, and the initial learning rate is [value missing]. The batch size is 32, the training is iterated for 100 epochs, and the model accuracy is verified every 10 epochs.

[0043] The input method for the second-view detection results to the graph neural network is as follows: First, the second-view detection results are mapped to the corresponding grid cells. For each node, three extended features are generated: the probability of goods being in stock, the probability of goods not being in stock, and the second-view detection confidence. These are then concatenated with the five-dimensional node features of the initial belief map to form an eight-dimensional input feature vector, which is then input into the graph neural network. The specific calculation formula for feature fusion is as follows: in, For nodes The final fusion of input features, These are the node features of the initial belief graph. These are the node features corrected from local real-time data. Extended features for second-view detection results; , , Let be the weight coefficients for the three types of input data, and let the weight coefficients satisfy . The default setting is , , The weighting coefficients can be adjusted according to the confidence level of different data sources.

[0044] When the difference in the probability of having goods between the frontal view detection result and the second-view detection result is ≥0.5, it is determined to be a data conflict. First, it is checked whether there is local real-time data at this location. If there is, the correction result of the local real-time data is used as the sole benchmark to cover the conflicting data between the two views. If there is no local real-time data, the uncertainty label value of the node is forcibly set to 0.95, and it is marked as a high uncertainty node. In the final decision-making stage, the "requires manual review" label is output. At the same time, in the graph neural network inference process, the edge weight between the node and the adjacent nodes is reduced by 0.2 coefficients to avoid the conflicting data from interfering with the state inference of adjacent nodes.

[0045] During inference execution, the local real-world data is used as the highest priority benchmark to correct the state vector of the corresponding grid cell in the initial belief graph. The default correction rule for the standard scenario is as follows: When there is a high uncertainty location, for the local real-world data of the corresponding location, if it is verified that there is a complete standard cargo at that location, the initial probability value of the cargo-present node is corrected to 0.98, the initial probability value of the cargo-absent node is simultaneously corrected to 0.02, the uncertainty label value is corrected to 0.02, and the occlusion degree estimate is cleared to zero. If it is verified that there is no cargo at that location, or that the cargo is missing / empty, the initial probability value of the cargo-absent node is corrected to 0.98, the initial probability value of the cargo-present node is simultaneously corrected to 0.02, and the uncertainty label value is corrected to 0.02. When there is no high uncertainty location, the probability values ​​of all nodes in the initial belief graph are globally consistent and fine-tuned based on the global benchmark real-world data.

[0046] The corrected initial belief map and the second-view detection results are input into the graph neural network. The model infers based on the stacking occlusion pattern learned in pre-training and the prior physical structure. The graph attention layer adaptively weights the input data from different sources and the features of neighboring nodes to achieve deep fusion of multi-source data. The model outputs the updated "available" belief probability, "out of stock" belief probability, and updated uncertainty label value for each grid cell. At the same time, based on the stacking occlusion pattern and physical structure priors, it completes the state information of the completely occluded positions to form the final updated belief map.

[0047] S4. Based on the updated belief map, count the total number of goods.

[0048] In practice, based on the updated belief map, and through a pre-defined hierarchical decision-making logic, a definite state, an inference result with a confidence score, or a marker requiring manual verification is output for each cargo location. The total number of cargoes is also counted, meaning that a corresponding state result is output for each grid cell (cargo location). The specific decision-making rules are as follows: like (Right now Output a confirmed status of "In Stock"; like (Right now Output a confirmed status of "out of stock"; like (Right now The output includes this The value is used as the confidence score to indicate "indication of availability"; like (Right now The output includes this The value is used as a confidence score to indicate "out of stock" results; like (Right now If multiple data sources meet the preset information conflict conditions, a "manual review required" mark will be output.

[0049] in, The updated probability of the "available" belief for this location. For the updated "out of stock" belief probability at this location, satisfying... ,Right now ∈ , ∈ ; The first high threshold (the threshold for confirming goods are in stock) is 0.95, the second high threshold (the threshold for confirming goods are out of stock) is 0.95, the low threshold is 0.6, and the fuzzy range is 0.4-0.6. The preset information conflict condition is: the difference in the probability of goods being in stock between the frontal and side-view detection results is ≥0.5, and there is no local real-time data for correction. Based on the status results of all goods locations, a total goods count is performed: locations marked "confirmed to be in stock" and "inferred to be in stock" are included in the total goods count, locations marked "confirmed to be out of stock" are not included in the total count, and locations marked "requiring manual verification" are marked separately and not included in the automatic total count.

[0050] The final standardized inventory report includes: location number, standard stack type, rated total number of goods, automatically calculated total number of goods, a list of locations "definitely present / absent", a list of locations "inferred present / absent" with corresponding confidence levels, and a list of locations "requiring manual verification" with corresponding uncertainty values. The report is synchronously pushed to the warehouse WMS system via industrial Ethernet, completing closed-loop management of inventory data. For locations "requiring manual verification", the WMS system automatically generates a manual verification task and pushes it to the warehouse administrator's PDA terminal. The warehouse staff then completes the on-site verification and updates the inventory data to ensure that the inventory records match the actual inventory.

[0051] In some embodiments, in step S2, a temporal visual fingerprint is extracted and generated for each instance of segmented goods; in step S3, the temporal visual fingerprint corresponding to each goods is input into the neural network fusion module. During the inference process, the temporal visual fingerprint is used to verify the identity association of goods detection results from different stages and spatially adjacent locations to determine whether they belong to the same physical goods and correct the goods identity association information. The temporal visual fingerprint is generated based on local key points and descriptors extracted from the outer surface image of the goods, or packaging texture and printing pattern features encoded by a lightweight convolutional neural network.

[0052] In practical implementation, this embodiment provides two parallel generation methods, which can be flexibly selected according to the packaging type of the warehouse goods: First, fingerprint generation is based on SIFT features. The SIFT scale-invariant feature transform algorithm is used to extract local keypoints from the segmented cargo ROI region, removing invalid keypoints at edges and with low contrast, retaining stable 128-dimensional feature descriptors. After weighted averaging and L2 normalization of all valid descriptors, a unique 128-dimensional temporal visual fingerprint for the cargo is generated. In the weighted averaging process, the weight of each valid descriptor is determined by the response intensity and scale of that keypoint. The weight calculation formula is as follows: in For the first The weights of keypoint descriptors, This is the scale factor for the key point. The DoG response intensity of the key point is used; the weight is positively correlated with the scale and response intensity of the key point, ensuring that key points with significant texture features and high stability dominate the fingerprint generation, thereby improving the uniqueness and anti-interference ability of the fingerprint; after weighting, L2 normalization is performed on the 128-dimensional vector obtained by weighted averaging to finally generate a temporal visual fingerprint.

[0053] Second, fingerprint generation is based on a lightweight CNN. A pre-trained MobileNetV3_small network is used, with the weights of the feature extraction layers frozen. The segmented goods ROI images are scaled to 224×224 resolution and input into the network. The 512-dimensional feature vector output from the global average pooling layer is extracted and, after L2 normalization, used as the temporal visual fingerprint of the goods. This network is pre-trained on 100,000 images of goods packaging. The pre-training dataset covers over 1200 SKUs across four categories—food, daily chemicals, electronic components, and industrial parts—in the target warehouse, including printed cartons and blank turnover boxes. The system supports various packaging types, including laminated packaging and barcode label packaging. Image acquisition covers a full range of conditions, including ambient lighting of 0-1000 lux, multiple shooting angles (front, side, and top views), and shooting distances of 0.5m-2m. The pre-training process employs a contrastive learning framework, using images of the same product from different perspectives as positive samples and images of different products as negative samples to optimize the network's feature discrimination capabilities. This ensures that the trained model generates feature vectors with high similarity for the same product, high discrimination for feature vectors generated from different products, and a cosine similarity ≥0.85 for feature vectors of the same product from different perspectives. The generated temporal visual fingerprint is bound to the product's grid number and physical location number and stored in a temporary cache on the edge computing terminal for subsequent cross-stage authentication.

[0054] During the neural network fusion inference process in step S3, the local stable key points and feature descriptors of the unobstructed areas of the goods detected in the second-view image after delivery are extracted simultaneously to generate a local feature fingerprint to be matched. Through the built-in fingerprint matching submodule, the feature cosine similarity between the local feature fingerprint and the corresponding area of ​​the temporal visual fingerprint generated in step S2 is calculated. When the proportion of effective matching key points is ≥60% and the average feature similarity is ≥0.85, it is determined that the two detection results correspond to the same physical goods, thus completing the cargo identity association in the three stages of pre-picking, transportation, and post-delivery. This corrects the position mapping errors caused by slight displacement of goods, changes in viewpoint, and changes in occlusion, avoids the same goods being counted repeatedly or missed, and ensures the consistency and uniqueness of inventory counting. The aforementioned feature similarity threshold of 0.85 can be adjusted within the range of 0.75-0.90 based on the complexity of the cargo packaging features. The threshold is determined based on the matching test results of 2000 sets of multi-view images of the same cargo. At the threshold of 0.85, the matching success rate of the same cargo reaches 99.2%, and the mismatch rate of different cargoes is less than 0.5%, balancing matching accuracy and fault tolerance.

[0055] To address feature loss due to partial soiling or occlusion of cargo packaging, a keypoint block matching strategy is adopted. The cargo ROI region is divided into four quadrant blocks. A valid match is deemed complete if the similarity of two or more quadrant blocks is ≥0.85 and the total number of valid matching keypoints is ≥10. A similarity gradient tolerance range is also set. When the average similarity is between 0.80 and 0.85, a secondary verification is performed using the cargo's outline dimensions and physical location information. Matches with an outline dimension error ≤5% and a matching physical location are further deemed valid. The alignment logic for multi-view fingerprints of the same cargo is as follows: First, based on the scale-invariant property of SIFT features, the complete temporal visual fingerprint generated in step S2 and the local feature fingerprint generated in step S3 are scale-normalized and aligned in the main direction to eliminate scale and rotation deviations caused by shooting angle and distance. Then, the RANSAC algorithm is used to remove mismatched keypoints. Rigid transformation alignment of the feature space is performed based on the matched inlier pairs. After alignment, the feature cosine similarity is calculated to ensure the matching accuracy of multi-view features.

[0056] In some embodiments, the specific process of constructing the initial belief map of cargo location with uncertainty annotation in step S1 is as follows: The image before pickup is classified into stack types, and the spatial grid division rules for cargo location corresponding to the stack are determined based on the classified standard stack types; then, visible cargo targets are detected in the image before pickup, and based on the target detection results and the spatial grid division rules, an initial belief map of cargo location with uncertainty annotation is constructed. The spatial grid division rules are as follows: Based on the number of cargo stacking layers and the number of goods in a single column of the classified standard stack types, a two-dimensional grid of equal size corresponding to the physical storage space of the stack is generated; each grid cell uniquely corresponds to an independent standard cargo storage location in the stack, and the row and column arrangement of the grid and the total number of cells match the cargo stacking arrangement and the rated total number of cargo storage for that standard stack type.

[0057] In practice, for non-full stacks, the stack type classification network will output the closest standard stack type. The grid division will still be completed according to the rated parameters of the standard stack type to ensure that each empty position has a corresponding grid unit in the non-full stack scenario, so as to achieve state coverage of the entire stack and avoid the problem of empty positions having no corresponding state representation. For example, if the classification result is a standard stack type of 6 layers, 5 columns and 30 pieces, but the actual stack is not full, it will still be divided into 6 rows and 5 columns and 30 grid units. The grid unit of each empty position will still generate a corresponding state vector to ensure that there is no omission in the full position count.

[0058] In some embodiments, in step S1, the initial belief map of cargo location represents the cargo stack space in the form of a grid. Each grid cell is associated with a multidimensional state vector, which includes at least: an initial probability value of the presence of cargo at the location generated based on the target detection result, a list of other cargo identifiers that may obscure the location, an estimated value representing the degree of obstruction of the location, and an uncertainty label value used to quantify the reliability of the location state judgment.

[0059] In practice, the dimensions of the state vector can be expanded according to the actual management needs of the warehouse. For example, dimensions such as cargo category matching degree, location compliance labeling, and cargo anomaly marking can be added to adapt to different warehouse management needs. The calculation of the initial probability value can also be weighted by combining the category confidence of the target detection. The higher the detection confidence, the higher the weight of the initial probability value, further improving the accuracy of the initial belief map.

[0060] In some embodiments, in step S2, local real-world data is used as a verification benchmark for state correction of the initial belief map in step S3. The local real-world data includes at least the actual outline size, surface features, and integrity information of the corresponding goods.

[0061] In practice, during the correction process in step S3, a custom correction threshold can be set according to the accuracy requirements of the warehouse inventory count. If local real-time data verifies that there are complete goods at the corresponding location, then the goods at that location are directly... The value is corrected to above 0.95, and the uncertainty label value is reduced to below 0.1; if the local real-world data verification shows that there are no goods at the corresponding location, or that the goods are damaged or missing, then the value at that location is directly adjusted. The value is corrected to above 0.95 to ensure that high-confidence data is used to correct the error of the initial belief map first, and to avoid the interference of low-confidence test results on the final inventory results.

[0062] In some embodiments, in step S3, during the inference process, the neural network fusion module first uses local real-world data as a benchmark to correct the probability value and uncertainty label value of the corresponding position in the initial belief map, and then performs fusion calculation by combining the detection results from the second perspective. The neural network fusion module is a graph neural network based on the attention mechanism. The fusion calculation process of the neural network fusion module is based on the stacking occlusion pattern and physical structure prior learned during pre-training. The stacking occlusion pattern includes the occlusion rules of foreground goods on background goods, and the physical structure priors include the lower layer support rules of goods stacking and the uniform arrangement rules of the same layer.

[0063] In practical implementation, the attention-based graph neural network adaptively adjusts the weights of adjacent nodes during the fusion calculation process, assigning higher weights to nodes in the lower support positions to align with the physical rules of goods stacking. The stacking occlusion pattern is learned from massive amounts of labeled data, covering the occlusion area and proportion of foreground goods on background goods under different stacking layers and goods sizes, adaptable to different specifications of goods and stack types. The physical structure prior is embedded in the graph neural network's loss function, forcibly satisfying the lower support rule of "if there is goods on the upper layer, there must be goods in the corresponding position on the lower layer" and the same-layer arrangement rule of "goods in the same layer are evenly and equidistantly arranged" during model training and inference. This ensures that the inference results conform to the physical logic of goods stacking, avoiding unrealistic state inferences and further improving the accuracy of state completion for occluded positions.

[0064] A warehouse item counting system based on image recognition, used to implement the aforementioned warehouse item counting method based on image recognition, includes: an image acquisition unit, an initial perception and reasoning unit, a local perception unit, a neural fusion reasoning unit, and a decision output unit.

[0065] The image acquisition unit, configured on the stacker crane, is used to acquire images of the stack or goods from corresponding perspectives in three stages: before picking up the stack, during transport, and after placing the goods. The main hardware consists of the aforementioned No. 1, No. 2, and No. 3 fixed-focus industrial cameras, a matching ring light source, and an image acquisition card, all installed on the stacker crane's loading platform and electrically connected to the stacker crane's PLC controller. It receives trigger commands from the PLC to complete image acquisition. The core function is to acquire images of the stack or goods from corresponding perspectives in the three stages: before picking up the stack, during transport, and after placing the goods, and to transmit the pre-processed images to the corresponding algorithm unit of the edge computing terminal in real time.

[0066] The image acquisition unit is triggered in three time-series nodes: The first node, upon receiving a signal from the WCS system indicating that the stacker crane has reached the target location and come to a complete stop, triggers camera 1 to acquire a frontal view image before the goods are picked up; the second node, upon receiving dual confirmation signals from the WCS system and photoelectric sensors, as well as the acquisition instruction list from the initial sensing and inference unit, triggers camera 2 to perform targeted local image acquisition; the third node, upon receiving a signal from the WCS system indicating that the goods have been returned to their location and the forks have retracted, triggers camera 3 to acquire a side view image after the goods have been placed. Each image acquired at each node is accompanied by a unique location number and timestamp to ensure the data stream corresponds to the location and inventory stages.

[0067] The initial perception and inference unit is used to classify the stacks of goods to be inventoried, divide the spatial grid of goods locations, and detect visible goods targets, generating an initial belief map of goods locations with uncertainty annotations. After completing the construction of the initial belief map, this unit will simultaneously perform two actions: first, cache the initial belief map data in the shared memory of the edge computing terminal for direct reading by the neural fusion inference unit; second, simultaneously send the list of acquisition instructions and ROI region parameters for high uncertainty locations to the image acquisition unit and the local perception unit as the basis for subsequent local acquisition and instance segmentation.

[0068] The Local Perception Unit (LPU) is used to perform instance segmentation of local areas of cargo images and generate local real-time data. The LPU has a built-in pre-trained instance segmentation model and an optional visual feature extraction submodule. Its core functions are: receiving unobstructed local area images transmitted from the image acquisition unit, performing instance segmentation of the local cargo image, and generating high-confidence local real-time data; generating a temporal visual fingerprint corresponding to the cargo through the visual feature extraction submodule, binding and storing the fingerprint with the cargo location, and transmitting the local real-time data and temporal visual fingerprint to the neural fusion inference unit. This unit only performs instance segmentation on local images corresponding to high-uncertainty locations marked by the initial perception and inference units. After segmentation, the local real-time data is bound to the corresponding grid number and written to shared memory in real time. If the temporal visual fingerprint function is enabled, the generated fingerprint is simultaneously bound and stored with the cargo's physical location number for use by the neural fusion inference unit.

[0069] The neural fusion inference unit has a built-in pre-trained neural network fusion module for performing fusion inference of multi-source data, updating the cargo location belief map and supplementing the state information of occluded locations. The neural fusion inference unit has a built-in pre-trained neural network fusion module and can be optionally configured with a fingerprint matching submodule. Its core functions are: receiving the initial belief map, local real-time data, and second-view detection results; performing fusion inference of multi-source data; updating the cargo location belief map and supplementing the state information of occluded locations; and using the fingerprint matching submodule to perform temporal visual fingerprint similarity calculation and cross-stage cargo identity association verification, correcting cargo identity association errors, and transmitting the updated belief map to the decision output unit.

[0070] The decision output unit generates status results for each cargo location based on the updated belief map, and outputs total cargo count and inventory results. The unit features built-in hierarchical decision logic with visual configuration, allowing users to customize decision thresholds via a web interface. Its core functions include: generating status results for each cargo location based on the updated belief map using hierarchical decision logic; generating total cargo count and inventory report reports; and integrating with the warehouse WMS / WCS system to synchronize inventory results to the warehouse management system, thus completing a fully automated inventory process.

[0071] The system's workflow is as follows: After receiving the inventory task from the WMS system, the system sends a stacker crane control command to the WCS system, driving the stacker crane to the target location. The system completes full-process image acquisition through the image acquisition unit, and then processes and infers the data through the initial perception and reasoning unit, the local perception unit, and the neural fusion reasoning unit. Finally, the system generates the inventory result through the decision output unit and sends it back to the WMS system to complete the inventory of a single location.

[0072] Those skilled in the art will understand that all or part of the steps in the above embodiments can be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the steps of the methods described above. The storage medium may include media such as a disk, optical disk, ROM, RAM, etc.

[0073] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A warehouse item counting method based on image recognition, characterized in that, Includes the following steps: S1. Acquire images of the stack of goods to be inventoried from a first-person perspective before picking up the goods, and construct an initial belief map of the goods location with uncertainty annotations based on the images before picking up the goods. S2. When the goods in the stack are moved out of the storage location and are in a transport state, for the goods positions marked as high uncertainty in the initial belief map, local area images of the corresponding goods are collected and instance segmented to obtain local real-time data. S3. Acquire images of the stack of goods after placement from a second perspective and perform target detection to obtain the second perspective detection results; input the initial belief map, the local real-time data, and the second perspective detection results into a pre-trained neural network fusion module; update the initial belief map through the neural network fusion module and complete the state information of the occluded goods position; S4. Based on the updated belief map, count the total number of goods.

2. The warehouse item counting method based on image recognition according to claim 1, characterized in that: In step S2, the goods segmented for each instance are extracted and a temporal visual fingerprint is generated. In step S3, the temporal visual fingerprints corresponding to each cargo are also input into the neural network fusion module. During the inference process, the temporal visual fingerprints are used to determine whether each cargo belongs to the same physical cargo and to correct the cargo identity association information.

3. The warehouse item counting method based on image recognition according to claim 2, characterized in that: The temporal visual fingerprint is generated based on local key points and descriptors extracted from images of the outer surface of the goods, or packaging texture and printing pattern features encoded by a lightweight convolutional neural network.

4. The warehouse item counting method based on image recognition according to claim 1, characterized in that, In step S1, the specific process of constructing the initial belief map of cargo location with uncertainty annotations is as follows: The image before pickup is classified into stack types, and the spatial grid division rules for the cargo location corresponding to the stack are determined based on the standard stack types obtained from the classification. Then, visible cargo targets are detected in the image before pickup, and an initial belief map of cargo location with uncertainty annotation is constructed based on the target detection results and the spatial grid division rules.

5. The warehouse item counting method based on image recognition according to claim 4, characterized in that, The spatial grid division rule is as follows: Based on the number of stacking layers and the number of goods per column of the standard stack type obtained from the classification, a two-dimensional grid of the same size is generated that corresponds one-to-one with the physical storage space of the stack. Each grid cell uniquely corresponds to an independent standard goods storage location in the stack. The row and column arrangement of the grid and the total number of cells match the stacking arrangement of the goods and the rated total number of goods for that standard stack type.

6. The warehouse item counting method based on image recognition according to claim 1, characterized in that: In step S1, the initial belief map represents the stack space in the form of a grid. Each grid cell is associated with a multi-dimensional state vector. The state vector includes at least: an initial probability value of the presence of goods at the location generated based on the target detection results, a list of other goods identifiers that may obscure the location, an estimated value representing the degree of obstruction at the location, and an uncertainty label value used to quantify the reliability of the location state judgment.

7. The warehouse item counting method based on image recognition according to claim 1, characterized in that: In step S2, the local real-time data serves as a verification benchmark for correcting the state of the initial belief map in step S3. The local real-time data includes at least the actual outline dimensions, surface features, and integrity information of the corresponding goods.

8. The warehouse item counting method based on image recognition according to claim 1, characterized in that: In step S3, during the inference process, the neural network fusion module first uses the local real-time data as a benchmark to correct the probability value and uncertainty label value of the corresponding position in the initial belief map, and then performs fusion calculation by combining the detection results from the second perspective.

9. A warehouse item counting method based on image recognition according to claim 8, characterized in that: In step S3, the neural network fusion module is a graph neural network based on the attention mechanism. The fusion calculation process of the neural network fusion module is performed based on the stacking occlusion pattern and physical structure prior learned during pre-training.

10. A warehouse item counting system based on image recognition, used to implement the warehouse item counting method based on image recognition according to any one of claims 1-9, characterized in that, include: The image acquisition unit is used to acquire images of the stack or goods from corresponding perspectives in three stages: before picking up the stack, during the transport of goods, and after the goods are placed. The initial perception and reasoning unit is used to construct an initial belief map of the cargo location with uncertainty annotations based on the pre-pickup image; The local sensing unit is used to perform instance segmentation of local areas of cargo images and generate local real-time data; The neural fusion reasoning unit has a built-in pre-trained neural network fusion module, which is used to perform fusion reasoning of multi-source data, update the cargo position belief map and complete the state information of the occluded position. The decision output unit is used to complete the total number of goods statistics and inventory results output based on the updated belief map.

Images