A box material intelligent inventory method

Abstract

Embodiments of the present application provide a kind of box material intelligent inventory method.The method comprises the following steps: step 1, inventory path generation;Step 2, feature recognition detection;Step 3, obtain the feature center coordinates of camera coordinate system;Step 4, box stacking layer calculation;Step 5, obtain the global coordinate system box center coordinates;Step 6, inventory result fusion and save.The method installs binocular camera on the suspension rail type travelling crane, traverses and collects the upper surface image of the warehouse stacking box material from the bird's-eye view, based on the target detection algorithm and stereovision ranging method in machine vision, combined with travelling crane running position feedback, realizes the label-free, automatic, high-precision inventory operation of the type, quantity and position information of the warehouse stacking box material.

Description

Technical Field

[0001] This application relates to the fields of warehouse automation and machine vision technology, and in particular to a method for intelligent inventory counting of boxed materials. Background Technology

[0002] Containerized storage is widely used for storing industrial raw materials, electromechanical components, food, pharmaceuticals, and military equipment. To fully utilize limited space and increase storage density, these materials are typically stored in indoor warehouses using a stacking method. Currently, inventory counting of containerized materials mainly relies on manual labor, labels, or inspection robots. Inventory counting based on barcodes, QR codes, or RFID tags involves attaching labels to each container, using handheld or fixed readers to read unique identifiers, and then matching the label to the corresponding item type and recording the location through the warehousing system. The accuracy of this method in terms of inventory quantity and location depends on manual registration and updating during initial warehousing. Inventory counting based on ground inspection robots utilizes visual recognition to identify the type of low-lying containers and uses methods such as odometers and SLAM to obtain its own location, thereby estimating the container's position. Manual inventory counting relies on workers visually recording the type and quantity of goods, supplemented in some scenarios by handheld terminals recording location information.

[0003] For the intelligent inventory management of high-rise stacked containerized materials, existing solutions struggle to achieve high-precision, comprehensive, and automated type identification, quantity counting, and spatial positioning of these materials. The drawbacks of three common technical solutions are as follows: 1) The label inventory method requires the establishment of a complete label system. Manual personnel use handheld or fixed terminals to read information such as type and location of each item. If the labels are worn, detached or damaged, the inventory data will be missing or incorrect. In addition, the accuracy of this method in counting the type, quantity and location of the boxed materials depends heavily on manual registration at the initial entry and updates after the items are released, and it lacks real-time update capabilities.

[0004] 2) Ground inspection robots are limited by their field of view height, making it difficult to obtain information on materials stacked on high levels; in addition, in order to improve space utilization under warehousing conditions, the passages between storage locations are narrow and irregular, which is not suitable for ground inspection robots to shuttle between storage locations; furthermore, when the storage location changes, it is necessary to rebuild the driving map and plan the driving trajectory of the inspection robot, which is difficult to adapt to application scenarios where the storage location is not fixed.

[0005] 3) Manual inventory counting relies on workers scanning or visually inspecting the types and quantities of goods in containers one by one, supplemented by handheld terminals to record their location information. This method has problems such as low inventory counting efficiency, high labor intensity for workers, and high data inconsistency. Especially in high-rise stacking and large-area storage environments, it is difficult to achieve comprehensive coverage and high-frequency updates of inventory counts.

[0006] In summary, existing solutions cannot fully achieve high-precision, comprehensive, and automated type identification, quantity counting, and spatial positioning of containerized materials. They exhibit significant shortcomings, particularly in complex real-world scenarios such as high-rise stacking, mixed batch storage, and changing storage locations, failing to meet the demands of modern intelligent warehousing for rapid, accurate, and automated inventory counting of containerized materials. Therefore, there is an urgent need for an intelligent inventory counting method that integrates identification and positioning to achieve intelligent inventory counting of containerized material type, quantity, and location in high-density storage environments. This would reduce the labor intensity of inventory counting personnel and improve inventory counting efficiency and accuracy. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes an intelligent inventory method for boxed materials. A binocular camera is mounted on a suspended track trolley to collect images of the upper surface of stacked boxed materials from a bird's-eye view. Based on target detection algorithms and stereo vision ranging methods in machine vision, combined with trolley position feedback, the method enables label-free, automated, and high-precision inventory of the type, quantity, and location information of stacked boxed materials in the warehouse.

[0008] This invention provides a method for intelligent inventory management of containerized materials, the method comprising: Step 1: Inventory path generation; Step 2, Feature recognition and detection; Step 3: Obtain the coordinates of the feature center in the camera coordinate system; Step 4: Calculate the number of stacking layers for the containers; Step 5: Obtain the coordinates of the center of the box in the global coordinate system; Step 6: Integrate and save the inventory results.

[0009] In some embodiments, step 1, inventory path generation, includes: The warehouse area requiring inventory checks is divided into nodes, and the crane's operating path is planned. The crane's motion control unit controls the crane to move along a serpentine straight path. When the crane reaches a preset node, the binocular camera simultaneously captures the left image. And the right picture Record the position of the vehicle in the global coordinate system within the parking garage during the shooting process. .

[0010] In some embodiments, the warehouse area requiring inventory checks is divided into nodes, the vehicle's operating path is planned, and the vehicle motion control unit controls the vehicle to move along a serpentine straight path. When the vehicle reaches a preset node, the binocular camera simultaneously captures the left image. And the right picture Record the position of the vehicle in the global coordinate system within the parking garage during the shooting process. ,include: Assume the maximum stacking height of the containerized materials is The camera installation height is The horizontal and vertical field of view are respectively Its effective field of view is:

[0011]

[0012] Divide the pre-inventory area into grids and set the overlap ratio to [value missing]. The grid side length is:

[0013] If the pre-inventory area size is The number of grid divisions is: ,

[0014] The total number of shooting points is:

[0015] The vehicle's operating path is designed as a serpentine traversal, recording the global position coordinates within the vehicle depot at each shooting point. Cover all grid node locations sequentially to ensure no blind spots.

[0016] In some embodiments, step 2, feature recognition and detection, includes: identifying the type of container material based on the YOLO v8 target detection model, and obtaining the coordinates of the center of the container feature detection box in the pixel coordinate system.

[0017] In some embodiments, the step of identifying the type of container material based on the YOLO v8 object detection model and obtaining the coordinates of the center of the container feature detection box in the pixel coordinate system includes: Pre-planning of each type of containerized material The dimensions are measured a priori, and the box features to be identified are selected and labeled. An object detection model is trained based on YOLO v8 and deployed in the control terminal software. During vehicle movement, the system identifies box images taken from above at various nodes in real time, and the output is as follows:

[0018] in, This is the current image recognition result; Let be the coordinates of the center point of the i-th target detection box in the image plane; The width and height of the detection frame; The type of goods in the container to be identified; To test the confidence level; This represents the total number of boxes detected.

[0019] In some embodiments, step 3, obtaining the feature center coordinates in the camera coordinate system, includes: transforming the detection box center and boundary dimensions from the pixel coordinate system to the camera coordinate system by combining the binocular camera model; determining the completeness of the current detection target based on the prior box and the actual size model of the features; and when the target is incompletely detected, inferring the position of the complete feature center coordinates in the camera coordinate system based on the prior feature size.

[0020] In some embodiments, obtaining the feature center coordinates in the camera coordinate system includes: transforming the detection box center and boundary dimensions from the pixel coordinate system to the camera coordinate system using a stereo camera model; determining the completeness of the current detection target based on the prior box and actual feature size model; and when the target is incompletely detected, inferring the position of the complete feature center coordinates in the camera coordinate system based on the prior feature size, including: For the center point of the detection frame Images from the left and right cameras , Disparity calculation to obtain its depth value The coordinates of the center point of the feature detection box are:

[0021] In the formula, The three-dimensional coordinates of the center point of the feature detection box in the camera coordinate system; The focal length of the camera; The binocular baseline length; , Principal point coordinates; The disparity values ​​are those between the left and right images. The width and height pixel values ​​of the detection box Convert to actual space dimensions:

[0022] For each type of box Let the prior features be the actual physical dimensions. The feature completeness index is defined as follows:

[0023] when When the target detection box is deemed to have good integrity, no compensation for the center coordinates is required. or If the target detection box is deemed to have poor integrity, the detection result is deleted. or When the integrity of the target detection box is deemed to be average, compensation is needed for the center coordinates of the feature detection box in the camera coordinate system. like and The feature in the upper left corner of the detection box is complete; if and The feature in the upper right corner of the detection box is complete; if and The feature in the lower right corner of the detection box is complete; if and The lower left corner of the detection box has complete features. Then, based on the coordinates of the corner points with complete features and the width and height of the complete detection box, the 3D coordinates of the center point of the compensated complete detection box in the camera coordinate system can be obtained. The output of the material identification result for each container is as follows:

[0024] in, and The i-th The prior characteristics of the type of box are its actual width and height.

[0025] In some embodiments, step 4, calculating the number of stacking layers of the boxes, includes: calculating the number of stacking layers of the boxes based on the prior height dimensions and depth measurement information of the boxes.

[0026] Based on the identification results, the prior height dimension of the corresponding box type can be obtained. The camera mounting height H is obtained from the feature center point height in step 3) via binocular stereo measurement. The number of stacking layers of the boxes can be expressed as:

[0027] Due to measurement errors, the number of stacking layers... The value may be a non-integer value, therefore the judgment rule is designed as follows:

[0028] In the formula, To determine the threshold, if the residual is small, it can be considered to be caused by measurement error or gap; if the residual exceeds the threshold, it is considered that a new layer of box actually exists.

[0029] In some embodiments, step 5, obtaining the coordinates of the box center in the global coordinate system, includes: calculating the position coordinates of the box center in the global coordinate system using the position coordinates of the feature center point in the camera coordinate system and the prior position relationship between the box features and the box center.

[0030] The three-dimensional coordinates in the camera coordinate system are transformed to the warehouse global coordinate system using the following formula:

[0031] In the formula, , , For the first in the global coordinate system Coordinates of the center point of each target feature; and These are the rotation and translation matrices obtained from the camera extrinsic calibration, respectively. These are the vehicle's position coordinates as the camera captured the image.

[0032] Finally, based on the prior positional relationship between the feature center and the box center, the position coordinates of the box center in the global coordinate system are obtained:

[0033] In the formula, and These are the prior offsets of the feature centers in the x and y directions of the global coordinate system relative to the center of the box, respectively.

[0034] In some embodiments, step 6, fusion and storage of inventory results, includes: fusing and deduplicating the multi-frame image recognition calculation results based on spatial Euclidean distance, uploading information such as the type, quantity, and location of the container materials to the data storage and management unit, and completing intelligent inventory counting.

[0035] Let the current frame detection result set be... Each target contains information including three-dimensional location coordinates, category label, and confidence score, represented as follows: The target set for historical location fusion Each target For each detected target in the current frame traverse the historical target set Calculation and Objective Euclidean distance:

[0036] Let the distance threshold be ,like This further verifies the categories of the two targets. and If the categories are consistent, the two targets are considered to belong to the same object, and a weighted average fusion of the target positions is performed:

[0037] Simultaneously update the confidence level of historical targets:

[0038] If no matching target is found, the target is considered a new crate of supplies. Add to historical target set .

[0039] Finally, the target types, quantities, and locations of all identified targets are uploaded to the data storage and management unit.

[0040] This invention first employs a top-mounted crane combined with top-view binocular vision to replace traditional ground inspection robots or RFID tags, offering advantages such as simple deployment, large scanning range, and high flexibility. Secondly, it proposes an algorithm that combines detection box set information with image edge distance for complete feature center point position compensation, effectively improving recognition accuracy in cases of target occlusion or incomplete feature capture, particularly suitable for complex, dynamic, and variable stacking scenarios. Furthermore, this invention integrates 3D measurement data with a priori structural model in stacking layer recognition, exhibiting strong versatility and scalability. Finally, by constructing a transformation chain from image coordinates to spatial coordinates, a closed loop from 2D visual perception to 3D spatial positioning is achieved, providing key technical support for the automated inventory management and efficient operation of intelligent warehousing systems.

[0041] This invention enables contactless, high-precision, and automated high-density inventory counting of stacked container materials, tracking their type, quantity, and location. The beneficial effects of this invention include: 1) The top crane has full warehouse coverage capability, a large scanning range, and high flexibility. It is compatible with storage scenarios with dynamic changes in stack positions. The deployment of the intelligent inventory system can be completed by adding binocular cameras to the existing cranes in the warehouse without large-scale modification and with low hardware costs. 2) Deep learning models can be trained on the appearance features of various types of boxes, have high scalability, and have the ability to recognize and calculate under the condition of target occlusion or incomplete feature capture. Combined with three-dimensional visual depth measurement algorithms, the recognition accuracy and spatial resolution are higher, which can effectively improve the accuracy of inventory counting. 3) No need to attach labels to the container surface or manually conduct inventory checks, reducing deployment and maintenance costs. The camera can automatically take pictures and conduct inventory without manual movement, reducing the labor intensity of operators. Attached Figure Description

[0042] The accompanying drawings illustrate, by way of example and not limitation, the various embodiments discussed herein.

[0043] Figure 1 A schematic diagram of the layout of the overhead crane and binocular cameras; Figure 2 This is a schematic diagram of the process of generating paths and acquiring images for the inventory area.

[0044] Figure 3 A flowchart of a method for intelligent inventory management of containerized materials; Figure 4 Flowchart for calculating the coordinates of the complete feature center.

[0045] Symbol explanation: 1—Tractor trolley, 2—Tractor trolley, 3—Binocular camera module, 4—Control terminal. Detailed Implementation

[0046] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0047] In the embodiments described in this application, it should be noted that, unless otherwise stated and limited, the term "connection" should be interpreted broadly. For example, it can be an electrical connection, or a connection between two internal components. It can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above term according to the specific circumstances.

[0048] It should be noted that the terms "first," "second," and "third" used in the embodiments of this application are merely used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first," "second," and "third" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first," "second," and "third" can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in an order other than those illustrated or described herein.

[0049] This invention proposes an intelligent inventory method for boxed materials, which can be used to identify the type of stacked materials, count the quantity of materials, and estimate the location of materials, solving the problem of blind-spot-free inventory counting in high-rise stacking scenarios and improving the efficiency and accuracy of inventory management. The method involves mounting a binocular camera on a suspended track-type trolley to collect images of the upper surface of stacked boxed materials from a bird's-eye view. Based on target detection algorithms and stereo vision ranging methods in machine vision, combined with trolley position feedback, it achieves label-free, automated, and high-precision inventory counting of the type, quantity, and location information of stacked boxed materials in the warehouse.

[0050] The application scenario of this method is described as follows: The warehouse top is equipped with a rail-mounted inspection trolley or a crane that can run automatically according to a predetermined route. The boxed materials are stored in high-density stacks, and the storage position can be dynamically changed. The same type of boxed materials are stacked on top of each storage location. The upper surfaces of different types of boxes have unique structural or textural features, and the box size and features are prior data.

[0051] The hardware system used in the intelligent inventory method for containerized materials in this embodiment of the invention includes: a binocular camera module 3 mounted on the trolley 1 (viewed from above), as shown in the schematic diagram below. Figure 1As shown, the control terminal 4 includes a vehicle motion control unit, a back-end recognition and processing unit, and a data storage and management unit. The binocular camera module follows the movement of the trolley and the main vehicle, and can acquire images without stopping. The vehicle motion control unit can control the vehicle to run automatically along a predetermined route. The back-end recognition and processing unit can complete the identification of the container material type, location calculation, and quantity inventory based on the binocular camera images and vehicle position feedback. The data storage and management unit is used to store the prior container size model and inventory results.

[0052] The workflow diagram for the intelligent inventory method of containerized materials is as follows: Figure 3 As shown. The working steps are as follows: Step 1, Inventory Path Generation: Divide the warehouse area requiring inventory counting into nodes, plan the vehicle's operating path, and the vehicle motion control unit controls the vehicle to follow the path. Figure 2 The image shows a serpentine back-and-forth linear path movement. Upon reaching a preset node, the binocular cameras simultaneously capture the left image. And the right picture Record the position of the vehicle in the global coordinate system within the parking garage during the shooting process. ; Step 2, Feature Recognition and Detection: Identify the type of materials in the container based on the YOLO v8 target detection model, and obtain the coordinates of the center of the container feature detection box in the pixel coordinate system; Step 3, obtain the feature center coordinates in the camera coordinate system: combine the binocular camera model to transform the detection box center and boundary size from the pixel coordinate system to the camera coordinate system; based on the prior box and the actual size model of the feature, determine the integrity of the current detection target. When the target is incomplete, infer the position of the complete feature center coordinates in the camera coordinate system based on the prior feature size. Step 4, Calculation of the number of stacking layers of the boxes: The number of stacking layers of the boxes is calculated based on the prior height and depth measurement information of the boxes; Step 5, obtain the coordinates of the box center in the global coordinate system: calculate the position coordinates of the box center in the global coordinate system by using the position coordinates of the feature center point in the camera coordinate system and the prior position relationship between the box features and the box center; Step 6, Inventory Result Fusion and Saving: Based on the spatial Euclidean distance, the multi-frame image recognition calculation results are fused and deduplicated, and the information such as the type, quantity, and location of the container materials is uploaded to the data storage and management unit to complete the intelligent inventory count.

[0053] Further, step 1, inventory path generation, includes: Due to the limitations of the camera's field of view and the stacking height of the containerized goods, when using a top-down binocular camera to capture images of a designated area, it is necessary to reasonably calculate the camera's shooting step size based on the actual site dimensions, camera installation height, and the highest stacking height of the goods. The area to be photographed should be divided into a suitable grid as the basis for vehicle route planning, ensuring that the pre-inventory area is completely covered. Let the highest stacking height of the containerized goods be... The camera installation height is The horizontal and vertical field of view are respectively Its effective field of view is:

[0054]

[0055] Divide the pre-inventory area into grids and set the overlap ratio to [value missing]. The grid side length is:

[0056] If the pre-inventory area size is The number of grid divisions is: ,

[0057] The total number of shooting points is:

[0058] The train's operating path is designed to traverse in a serpentine manner, such as... Figure 2 As shown, the global position coordinates of each shooting point within the parking garage are recorded. Cover all grid node locations sequentially to ensure no blind spots.

[0059] Further, step 2, feature recognition and detection, includes: To enhance the ability to process inspection targets, each type of containerized material is pre-processed. The dimensions are measured a priori. Box-shaped annotations are completed for the features of the boxes to be identified. An object detection model is trained based on YOLO v8 and deployed in the control terminal software. During the vehicle's movement, top-down images of the boxes taken at various nodes are identified in real time, and the output results are as follows:

[0060] in, This is the current image recognition result; Let be the coordinates (in pixels) of the center point of the i-th target detection box in the image plane. The width and height of the detection frame; The type of goods in the container to be identified; To test the confidence level; This represents the total number of boxes detected.

[0061] Further, step 3, obtaining the coordinates of the feature center in the camera coordinate system, including: For the center point of the detection frame Images from the left and right cameras , Disparity calculation to obtain its depth value The coordinates of the center point of the feature detection box are:

[0062] In the formula, The three-dimensional coordinates of the center point of the feature detection box in the camera coordinate system; The focal length of the camera; The binocular baseline length; , Principal point coordinates; This represents the disparity value between the left and right images.

[0063] The same method is used to measure the pixel values ​​of the detection box width and height. Convert to actual space dimensions:

[0064] For each type of box Let the prior features be the actual physical dimensions. The feature completeness index is defined as follows:

[0065] when When the target detection box is deemed to have good integrity, no compensation for the center coordinates is required. or If the target detection box is deemed to have poor integrity, the detection result is deleted. or When the integrity of the target detection box is deemed average, compensation is needed for the center coordinates of the feature detection box in the camera coordinate system. The calculation process for the complete feature center coordinates is as follows: Figure 4 As shown. If the longer side of the feature detection box is parallel to the camera's X-axis, then the width of the complete detection box is equal to the actual height of the prior feature. If the longer side of the feature detection box is parallel to the camera's Y-axis, then the width of the complete detection box is equal to the actual width of the prior features. ; by calculating the distances from the four sides (left, right, top, and bottom) of the detection box to the image boundary. Determine the compensation direction. Let the image width and height be... The distances from the image boundary are as follows:

[0066]

[0067] like and The feature in the upper left corner of the detection box is complete; if and The feature in the upper right corner of the detection box is complete; if and The feature in the lower right corner of the detection box is complete; if and The lower left corner of the detection box has complete features. Then, based on the coordinates of the corner points with complete features and the width and height of the complete detection box, the 3D coordinates of the center point of the compensated complete detection box in the camera coordinate system can be obtained. The output of the material identification result for each container is as follows:

[0068] in, and The i-th The prior characteristics of the type of box are its actual width and height.

[0069] Further, step 4, calculating the number of stacking layers of the containers, includes: Based on the identification results, the prior height dimension of the corresponding box type can be obtained. The camera mounting height H is obtained from the feature center point height in step 3) via binocular stereo measurement. The number of stacking layers of the boxes can be expressed as:

[0070] Due to measurement errors, the number of stacking layers... The value may be a non-integer value, therefore the judgment rule is designed as follows:

[0071] In the formula, To determine the threshold, if the residual is small, it can be considered to be caused by measurement error or gap; if the residual exceeds the threshold, it is considered that a new layer of box actually exists.

[0072] Further, step 5 involves obtaining the coordinates of the center of the global coordinate system box, including: After completing the identification of the containerized material target and the estimation of the 3D coordinates of the feature center in the camera coordinate system, in order to achieve global positioning of the container and information fusion with the warehouse management system, it is necessary to transform the coordinates to the global warehouse coordinate system. The transformation formula is as follows:

[0073] In the formula, , , For the first in the global coordinate system Coordinates of the center point of each target feature; and These are the rotation and translation matrices obtained from the camera extrinsic calibration, respectively. These are the vehicle's position coordinates as the camera captured the image.

[0074] Finally, based on the prior positional relationship between the feature center and the box center, the position coordinates of the box center in the global coordinate system are obtained:

[0075] In the formula, and These are the prior offsets of the feature centers in the x and y directions of the global coordinate system relative to the center of the box, respectively.

[0076] Further, step 6, merging and saving the inventory results, includes: To avoid duplicate detection of the same object due to multiple viewpoints or multiple frames during inventory checks, a deduplication method based on 3D spatial location is designed. Let the set of detection results for the current frame be... Each target contains information including three-dimensional location coordinates, category label, and confidence score, represented as follows: The target set for historical location fusion Each target For each detected target in the current frame traverse the historical target set Calculation and Objective Euclidean distance:

[0077] Let the distance threshold be ,like This further verifies the categories of the two targets. and If the categories are consistent, the two targets are considered to belong to the same object, and a weighted average fusion of the target positions is performed:

[0078] Simultaneously update the confidence level of historical targets:

[0079] If no matching target is found, the target is considered a new crate of supplies. Add to historical target set .

[0080] Finally, the target types, quantities, and locations of all identified targets are uploaded to the data storage and management unit.

[0081] This invention first employs a top-mounted crane combined with top-view binocular vision to replace traditional ground inspection robots or RFID tags, offering advantages such as simple deployment, large scanning range, and high flexibility. Secondly, it proposes an algorithm that combines detection box set information with image edge distance for complete feature center point position compensation, effectively improving recognition accuracy in cases of target occlusion or incomplete feature capture, particularly suitable for complex, dynamic, and variable stacking scenarios. Furthermore, this invention integrates 3D measurement data with a priori structural model in stacking layer recognition, exhibiting strong versatility and scalability. Finally, by constructing a transformation chain from image coordinates to spatial coordinates, a closed loop from 2D visual perception to 3D spatial positioning is achieved, providing key technical support for the automated inventory management and efficient operation of intelligent warehousing systems.

[0082] This invention can automatically generate inspection paths based on the required inventory range without increasing the complexity of ground equipment, and complete the type identification of large-scale, high-precision, and high-density stacked box materials based on deep learning; This invention proposes a method for judging the integrity of a detection box based on the ratio of the detection box to the prior size. It supports further inference of the complete feature region by combining the image edge distance when the identified features are incomplete, and executes a center point coordinate compensation strategy. This method does not require additional labels or hardware, and only uses binocular vision information to complete the reliable estimation of the feature center point position, thereby realizing the solution of the three-dimensional position of the box. This invention uses binocular stereo vision to measure the three-dimensional position of the center point of the box, and calculates the actual number of stacking layers by using height measurement information and standard box thickness, which can realize automatic inventory of materials.

[0083] The technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0084] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for intelligent inventory counting of boxed materials, characterized in that, The method includes: Step 1: Inventory path generation; Step 2, Feature recognition and detection; Step 3: Obtain the coordinates of the feature center in the camera coordinate system; Step 4: Calculate the number of stacking layers for the containers; Step 5: Obtain the coordinates of the center of the box in the global coordinate system; Step 6: Integrate and save the inventory results.

2. The intelligent inventory method for containerized materials according to claim 1, characterized in that, Step 1, inventory path generation, includes: The warehouse area requiring inventory checks is divided into nodes, and the crane's operating path is planned. The crane's motion control unit controls the crane to move along a serpentine straight path. When the crane reaches a preset node, the binocular camera simultaneously captures the left image. And the right picture Record the position of the vehicle in the global coordinate system within the parking garage during the shooting process. .

3. The intelligent inventory method for containerized materials according to claim 2, characterized in that, The process involves dividing the warehouse area requiring inventory checks into nodes, planning the vehicle's operating path, and controlling the vehicle's motion control unit to move the vehicle along a serpentine back-and-forth straight path. Upon reaching a preset node, the binocular cameras simultaneously capture the left image. And the right picture Record the position of the vehicle in the global coordinate system within the parking garage during the shooting process. ,include: Assume the maximum stacking height of the containerized materials is The camera installation height is The horizontal and vertical field of view are respectively Its effective field of view is: Divide the pre-inventory area into grids and set the overlap ratio to [value missing]. The grid side length is: If the pre-inventory area size is The number of grid divisions is: , The total number of shooting points is: The vehicle's operating path is designed as a serpentine traversal, recording the global position coordinates within the vehicle depot at each shooting point. Cover all grid node locations sequentially to ensure no blind spots.

4. The intelligent inventory method for containerized materials according to claim 3, characterized in that, Step 2, feature recognition and detection, includes: identifying the type of container material based on the YOLO v8 target detection model, and obtaining the coordinates of the center of the container feature detection box in the pixel coordinate system.

5. The intelligent inventory method for containerized materials according to claim 4, characterized in that, The method for identifying the type of materials in a container based on the YOLO v8 target detection model and obtaining the coordinates of the center of the container feature detection box in the pixel coordinate system includes: Pre-planning of each type of containerized material The dimensions are measured a priori, and the box features to be identified are selected and labeled. An object detection model is trained based on YOLO v8 and deployed in the control terminal software. During vehicle movement, the system identifies box images taken from above at various nodes in real time, and the output is as follows: in, This is the current image recognition result; Let be the coordinates of the center point of the i-th target detection box in the image plane; The width and height of the detection frame; The type of goods in the container to be identified; To test the confidence level; This represents the total number of boxes detected.

6. The intelligent inventory method for containerized materials according to claim 5, characterized in that, Step 3, obtaining the feature center coordinates in the camera coordinate system, includes: transforming the detection box center and boundary dimensions from the pixel coordinate system to the camera coordinate system by combining the binocular camera model; judging the completeness of the current detection target based on the prior box and the actual size model of the features; when the target is incomplete, inferring the position of the complete feature center coordinates in the camera coordinate system based on the prior feature size.

7. The intelligent inventory method for containerized materials according to claim 6, characterized in that, The acquisition of the feature center coordinates in the camera coordinate system includes: transforming the detection box center and boundary dimensions from the pixel coordinate system to the camera coordinate system using a stereo camera model; determining the completeness of the current detection target based on the prior box and actual feature size model; and when the target is incompletely detected, inferring the position of the complete feature center coordinates in the camera coordinate system based on the prior feature size, including: For the center point of the detection frame Images from the left and right cameras , Disparity calculation to obtain its depth value The coordinates of the center point of the feature detection box are: In the formula, The three-dimensional coordinates of the center point of the feature detection box in the camera coordinate system; The focal length of the camera; The binocular baseline length; , Principal point coordinates; The disparity values ​​are those between the left and right images. The width and height pixel values ​​of the detection box Convert to actual space dimensions: For each type of box Let the prior features be the actual physical dimensions. The feature completeness index is defined as follows: when When the target detection box is deemed to have good integrity, no compensation for the center coordinates is required. or If the target detection box is deemed to have poor integrity, the detection result is deleted. or When the integrity of the target detection box is deemed to be average, compensation is needed for the center coordinates of the feature detection box in the camera coordinate system. like and The feature in the upper left corner of the detection box is complete; if and The feature in the upper right corner of the detection box is complete; if and The feature in the lower right corner of the detection box is complete; if and The lower left corner of the detection box has complete features. Then, based on the coordinates of the corner points with complete features and the width and height of the complete detection box, the 3D coordinates of the center point of the compensated complete detection box in the camera coordinate system can be obtained. The output of the material identification result for each container is as follows: in, and The i-th The prior characteristics of the type of box are its actual width and height.

8. The intelligent inventory method for containerized materials according to claim 1, characterized in that, Step 4, the calculation of the number of stacking layers of the boxes, includes: calculating the number of stacking layers of the boxes based on the prior height and depth measurement information of the boxes.

9. The intelligent inventory method for containerized materials according to claim 1, characterized in that, Step 5, obtaining the coordinates of the box center in the global coordinate system, includes: calculating the position coordinates of the box center in the global coordinate system using the position coordinates of the feature center point in the camera coordinate system and the prior position relationship between the box features and the box center.

10. The intelligent inventory method for containerized materials according to claim 1, characterized in that, Step 6, the fusion and storage of inventory results, includes: fusing and deduplicating the multi-frame image recognition calculation results based on spatial Euclidean distance, uploading information such as the type, quantity, and location of the container materials to the data storage and management unit, and completing the intelligent inventory count.

Images