Warehouse article intelligent identification system and method based on laser radar scanning

Abstract

The invention belongs to the technical field of article identification image data processing, and particularly relates to a warehouse article intelligent identification system and method based on laser radar scanning, and the method comprises the following steps: scanning the form of an article through laser radar equipment, obtaining point cloud data and image information, and uploading the scanning data to a cloud end or an edge calculation center in real time; based on a preset article model atlas library, performing feature matching on the scanning data and the atlas, and dividing a multi-level classification atlas according to basic article units, splicing relations or storage areas of the articles; generating a unique identification code for each type of article unit according to the hierarchical structure of the multi-level classification image set, and associating the three-dimensional model features of the unique identification code; and analyzing the point cloud data by using a convolutional neural network, constructing an article three-dimensional model obtained by scanning, and estimating the volume and number of articles in combination with storage height and position information. According to the invention, the method can achieve the recognition of the warehouse articles in a complex environment, and can achieve the calculation of the number of the warehouse articles.

Description

Technical Field

[0001] The present invention belongs to the technical field of object recognition image data processing, and in particular relates to a warehouse object intelligent recognition system and method based on laser radar scanning. Background Art

[0002] Intelligent identification of warehouse items is achieved by integrating AI, Internet of Things, image recognition, lidar and other technologies to realize the automated identification, positioning, sorting and management of items in the storage environment.

[0003] Problems with existing technologies: The identification of warehouse items is usually only performed on independent single items and orderly stacks of items. However, it is impossible to perform high-precision identification and quantity calculation on stacks of items in complex environments, making the actual warehouse item identification ineffective. Summary of the Invention

[0004] The purpose of the present invention is to provide a warehouse item intelligent identification system and method based on laser radar scanning, which can realize the identification of warehouse items in complex environments and calculate the quantity of warehouse items.

[0005] The technical solutions adopted by the present invention are as follows: A method for intelligently identifying warehouse items based on laser radar scanning includes the following steps: Scan the object shape through the LiDAR device, obtain point cloud data and image information, and upload the scan data to the cloud or edge computing center in real time; Based on the preset object model atlas library, the scanned data is matched with the atlas, and multi-level classification atlases are divided according to the basic item units, splicing relationships or storage areas; According to the hierarchical structure of the multi-level classification atlas, a unique identification code is generated for each type of item unit and associated with its three-dimensional model features; Use convolutional neural networks to analyze point cloud data, build a 3D model of the scanned objects, and estimate the volume and quantity of the objects by combining storage height and location information; Obtain photos of the objects, segment them into independent units through point cloud data processing, compare them with multi-angle views of the object's 3D model features, and verify the consistency of the recognition results; Based on the item association algorithm, the splicing or storage relationship of adjacent item units is judged, the second-level and above atlases are quickly matched, and the recognition results are compared with the inventory data to trigger difference warnings or precise measurement instructions.

[0006] When the recognition result does not match the inventory data verification, or when different types of item units exist in the same type of item units, one or more item units in the same type of item units are identified and compared with the item three-dimensional model or multi-level classification atlas to distinguish miscellaneous items in the same type of item units.

[0007] During the miscellaneous recognition process, for the item units that are blocked by the identified item units, the identification of the blocked item units is achieved by identifying and extracting the local features of the item units. The specific steps include: Image processing algorithms are used to extract the surface texture, geometric shape or identification symbol of the target object unit as local features; Compare the local features with the corresponding features of the top covering item unit. If the similarity is greater than or equal to the preset threshold, it is determined that the target item unit and the top item unit belong to the same category; If the similarity is less than the preset threshold, it is determined that the target item unit and the top item unit belong to different categories; Compare the obtained local features of the miscellaneous items with the distinguishing categories, combine the number of items and inventory data, and generate the proportion of the miscellaneous items in the total number of item units. Determine the category and position of the item unit based on whether the number of miscellaneous items occupies a dominant proportion and the inventory data. The preset threshold is dynamically adjusted based on historical verification data.

[0008] The method for generating a full image of the occluding object unit based on the identified local features of the occluding object unit specifically includes the following steps: Obtain the point cloud data of the occluded object unit and extract the local feature vector data; The extracted features are matched with the pre-stored object unit library for similarity, and the generative model is used to compensate for the unexposed areas to generate a complete three-dimensional model; Based on the dimensional parameters of the reconstructed complete 3D model and the bearing surface characteristics of the storage container bottom, the available stacking space is calculated and physical constraints are established; Simulate the stacking process of object units in a virtual environment, correct morphological deformation based on material mechanics parameters, and optimize the placement plan through heuristic algorithms; Based on the stable stacking simulation results, the maximum capacity is output, and the shape of the obstructing item unit is determined.

[0009] The method for estimating the volume and quantity of items by combining the storage height and location information of the item unit includes the following steps: Extracting the dimensional data and spatial coordinate position information of the 3D model based on the acquired 3D model of the object; Obtain the data of the item unit at the top, and determine the size information of the top item unit in combination with the item model atlas library; Obtain the physical characteristics of item units and the physical constraints of item storage locations based on the item model atlas library, and eliminate over-limit item units; Simulate the stacking of item units in a regular or irregular manner, implement spatial constraints based on item storage locations, and correct abnormal item units; Combining point cloud data with image information, and comparing multiple angles, verifying whether the simulated stacking state is consistent with the object's 3D model can correct errors caused by occlusion. Combined with historical item unit storage record data, cross-validate the current simulation stacking method and dynamically adjust the threshold and estimation results; Get the number of items based on the verification results and simulated stacking method.

[0010] The rule-based stacking simulation method includes the following steps: According to the acquired top item unit data, determining whether the top item unit is in a regular stacking state; If it is a regular stacking state, the regular method is used to simulate the stacking state; According to the unit size parameters of the top items, a theoretical stacking model is generated according to the same-layer and inter-layer arrangement rules; By dividing the total stacking height by the unit height of a single item and combining it with the physical constraints of the storage location, we can eliminate any abnormal stacking scenarios that exceed the limit. The irregular stacking simulation method includes the following steps: If the top item unit is in an irregular stacking state, use an irregular method to simulate the stacking state; Set physical contact relationship reasoning, mechanical simulation and shape correction, model collision volume, and simulate various stacking methods of item units based on the physics engine; After obtaining simulation assumptions of multiple stacking methods, the morphological similarity is compared with the obtained irregular storage state of the top item unit. If the similarity exceeds the threshold, it is determined to be consistent. If it is lower than the similarity threshold, this stacking assumption is eliminated.

[0011] A warehouse item intelligent identification system based on laser radar scanning, including: Data acquisition module: Deployed on a mobile carrier, it uses lidar to scan and generate high-density point cloud data, simultaneously recording spatial coordinates and intensity information. Combined with a visual camera, it performs point cloud and image registration and color mapping to generate enhanced point cloud data with texture attributes. Multi-level classification atlas database: stores a multi-level classification model divided by basic item units, splicing relationships and storage areas. Each level and each item unit is associated with a unique identification code and a three-dimensional feature vector library; Intelligent recognition engine: Integrates convolutional neural networks and graph neural networks to perform object unit recognition and image data processing; Dynamic verification compensation module: Perform the following steps for the occluded object unit: Extract local feature vectors and compare them with the top cover, dynamically adjust the similarity threshold, and distinguish similar miscellaneous items; A generative adversarial network is used to reconstruct a complete 3D model of the unexposed area, and the stacking state is simulated in combination with physical constraints. Collaboration module: Outputs recognition data and proofreading results to the warehouse scheduling system, dynamically plans routes, and triggers exception handling instructions.

[0012] According to another aspect of an embodiment of the present invention, an electronic device is provided. The electronic device includes a memory and a processor; the memory is used to store programs; and the processor executes the programs to implement any one of the aforementioned methods.

[0013] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The storage medium stores a computer program. When the computer program is executed by a processor, any one of the aforementioned methods is implemented.

[0014] According to yet another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, which implements any of the aforementioned methods when executed by a processor.

[0015] The technical effects achieved by the present invention are: In addition to identifying the number of items in the warehouse, the present invention can also identify incorrectly placed items and reposition and place them through robots or logistics vehicles, thereby enabling unmanned management of warehouse items. By comparing the three-dimensional model features in the database with the matching item information, the application of computing power can be reduced, the recognition speed can be improved, and the identification of items can be quickly realized according to the warehouse placement rules, thereby improving efficiency and reducing the application of redundant computing power.

[0016] The present invention can improve the accuracy of miscellaneous item recognition by adjusting thresholds driven by historical data, and combines a multi-level classification atlas with the FPFH algorithm to solve the confusion problem between combined items and miscellaneous items, reduce the misjudgment rate, and realize an automated closed loop from difference detection to inventory correction, thereby improving warehouse management efficiency.

[0017] The present invention, by combining point clouds, images and historical data, can realize the identification of objects and the identification of the number of objects using a single sensor, and reduce the identification error based on historical data. By introducing physical engine simulation and simulated stacking, it can realize the estimation of the number of object units of different materials using a static threshold method, and further realize the identification of object units in disordered stacking scenarios through multi-angle view comparison. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the intelligent identification method in the present invention; Figure 2 This is a flow chart of the method for identifying an obstructing object unit in the present invention; Figure 3 is a flow chart of a method for generating a full image of an obstructing object unit based on recognized local features of the obstructing object unit in the present invention; Figure 4 This is a flow chart of a method for estimating the volume and quantity of items by combining the storage height and location information of item units in the present invention; Figure 5 It is a structural diagram of the intelligent recognition system in the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is described in detail below with reference to the following examples. It should be understood that the following text is only used to describe one or more specific embodiments of the present invention and does not strictly limit the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] According to an embodiment of the present invention, a method embodiment of a method for intelligent identification of warehouse items based on lidar scanning is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0022] like Figure 1 As shown, a method for intelligent identification of warehouse items based on laser radar scanning includes the following steps: S1. Data collection and transmission: Scan the object shape through the lidar device, obtain point cloud data and image information, and upload the scan data to the cloud or edge computing center in real time; S2. Construction of multi-level classification atlas: Based on the preset object model atlas library, the scanned data is matched with the atlas, and the multi-level classification atlas is divided according to the basic item units, splicing relationships or storage areas; S3. Identification code generation and association: Based on the hierarchical structure of the multi-level classification atlas, a unique identification code is generated for each type of item unit and associated with its 3D model features; S4. 3D model analysis and quantity estimation: Use convolutional neural networks to analyze point cloud data, construct 3D models of scanned objects, and estimate the volume and quantity of objects by combining storage height and location information. S5. Segmentation and verification of disordered objects: Obtain photos of the objects, segment them into independent units through point cloud data processing, compare them with multi-angle views of the object's 3D model features, and verify the consistency of the recognition results; S6. Association matching and inventory proofreading: Based on the item association algorithm, the splicing or storage relationship of adjacent item units is determined, and the second-level and above atlases are quickly matched. The recognition results are then proofread with the inventory data to trigger difference warnings or precise measurement instructions.

[0023] According to step S1, the laser radar device includes at least one group of laser radars, and image acquisition devices such as cameras can also be set up. The acquired images are used to realize system calculations with the data acquired by the laser radar, and the above data information is uploaded to the cloud or edge computing center. The transmission methods include but are not limited to intra-domain network transmission, bus transmission, wireless data transmission, etc. The data is quickly processed by a device with high computing power, and a computing power center can also be set up on the carrier where the data acquisition device is installed.

[0024] In addition, the installation method of the data acquisition equipment includes but is not limited to installation on carriers such as robots, drones, and tracks to realize the automation of data acquisition, and the installation method has at least three axial movement methods. The above specific installation methods are existing technologies, so they will not be elaborated here.

[0025] Furthermore, data acquisition methods include static collection and dynamic collection. The static collection method is suitable for complex item placement areas, and the dynamic collection method is suitable for regular and small placement areas. By identifying different placement areas, the data collection efficiency and recognition efficiency can be improved.

[0026] For example, by deploying a multi-beam lidar device on a drone, the scanning frequency and angular resolution are dynamically adjusted to scan the shape of warehouse items, generating point cloud data containing three-dimensional coordinates, laser reflection intensity and timestamps, and synchronously triggering a high-resolution camera to take photos of the item placement. The spatiotemporal alignment of the point cloud and image data is achieved through a time synchronization protocol, and the collected data is uploaded to the cloud or edge computing center in real time via 5G, Wi-Fi modules, Star Flash or local area network, using a compressed transmission protocol to reduce bandwidth usage.

[0027] In step S2, data modeling is performed on the items entering the warehouse to build a library of model atlases of different items, and a secondary or multi-level classification atlas is built for the combination of multiple items. By comparing with the acquired scanned item image information, it is used to quickly distinguish the types of items and reduce the application of computing power. The above comparison method can be stored on a carrier to achieve rapid data processing and data transmission, generate corresponding logs for data storage, or generate corresponding warning information for reporting.

[0028] Furthermore, the execution operation includes the following steps: S201. Perform principal component analysis and dimensionality reduction on the point cloud data to extract geometric features (such as curvature and normal vectors of the object) or texture features (such as grooves and protrusions). S202. Use the improved Fast Point Feature Histogram (FPFH) algorithm to calculate the similarity between the scan data and the atlas, and divide the levels according to the matching threshold: Level 1 atlas: single basic item unit (such as a standard pallet, a single model part, a single component); Secondary atlas: two or more basic item units spliced together (e.g. mechanical parts connected by fixed connections, item units connected by plug-in connections in non-fixed ways); Level 3 atlas: groups divided by storage area (e.g., groups of items within a shelf partition); S203. Regularly optimize the atlas library through incremental learning algorithms to adapt to feature changes of newly stored items.

[0029] In step S3, a structured identification code is generated for each category of items in the multi-level classification atlas in step S2, including a hierarchical identifier, a feature hash, and a timestamp. The identification code is similar to the management batch number of warehouse management and is used for rapid machine identification in a specific numerical arrangement. Alternatively, the above-mentioned structured identification code is generated into a corresponding QR code, barcode, etc. At the same time, the identification code is bound to the three-dimensional model feature of the item and stored in a database. The three-dimensional model and inventory location information and related information are quickly retrieved through the identification code. Through the location information and related information, the corresponding three-dimensional model feature and the matching item information can be found from the database during the machine identification process, which can reduce the application of computing power and improve the recognition speed.

[0030] Furthermore, by setting features, the point cloud data and image information in step S1 can be identified through the main features, which can further reduce the difficulty of identification, match features in a targeted manner, and improve the information data matching between the three-dimensional model and the object at a faster speed.

[0031] In step S4, the improved PointNet++ is used to implement deep learning of the 3D point cloud data. The point cloud data is parsed through the network and the following steps are performed: S401, separating independent object units using a point cloud clustering algorithm; S402: Apply the minimum bounding box algorithm to estimate the volume of each unit, and calculate the number of stacking layers based on the pre-stored standard size library of items; S403. Introduce the material compression coefficient library to correct the estimation results: ; S404. Filter outliers that exceed the limit using statistical methods.

[0032] In step S5, for disorderly placed items, if they cannot be identified in step S4, the number of items placed is estimated in step S4 to generate reference parameters, and further identification is performed in step S5, which specifically includes the following steps: S501, identifying and segmenting the acquired item placement photos, extracting point cloud void features using the Alpha Shape algorithm, and separating the boundaries of adjacent items using density clustering; S502: Project the segmented point cloud into a 2D depth map (e.g., a top view, a side view, an oblique view, or any other view), and perform a structural similarity (SSIM) calculation with the pre-stored multi-angle views of the 3D model, obtaining a threshold of accuracy of more than 95% through training; S503, extracting a local feature vector from the occluded area and comparing it with the feature of the top cover. If the similarity is lower than a threshold, it is determined that one of the items is a miscellaneous item. S504. Automatically adjust the similarity threshold based on historical verification data: new threshold = original threshold × (1 + α × false recognition rate), where α is the learning rate; According to step S6, based on the item association analysis, an item relationship graph is constructed through a graph neural network, where the nodes are item units and the edge weights are splicing probabilities. For example, through the contact surface normal vector consistency calculation, the Hungarian algorithm is used to quickly match the secondary and above atlases, and the matching score is calculated to determine the matching degree. The recognition result is compared with the inventory data in the inventory management system. If the difference rate exceeds the threshold, a difference warning is triggered.

[0033] After triggering the difference warning, the target area is scanned at high density by controlling the robotic arm, stirring device, etc., and the point cloud data is corrected using the ICP registration algorithm to achieve accurate measurement.

[0034] Furthermore, through simulation optimization, genetic algorithms are applied in a virtual environment to optimize the item placement plan, and the maximum capacity and optimal path planning are output to the scheduling system. Through devices such as robots and logistics vehicles, items that trigger difference warnings will be rearranged and their locations planned.

[0035] According to the above steps, in addition to being able to identify the number of items in the warehouse, it is also possible to identify incorrectly placed items and reposition and place them through robots or logistics vehicles, thereby enabling unmanned management of warehouse items. By comparing the three-dimensional model features in the database with the matching item information, the application of computing power can be reduced, the recognition speed can be improved, and the identification of items can be quickly realized according to the warehouse placement rules, thereby improving efficiency and reducing the application of redundant computing power.

[0036] Furthermore, rapid response can be achieved through data collection and inventory verification, which can improve the overall management efficiency of the warehouse. Through multi-level classification atlas and neural network association analysis, the limitations of single item recognition and recognition time can be realized, and the problem of slow combination matching can be solved. Combined with limited hardware costs, the cost of warehouse association construction can be reduced by combining the collaborative computing of edge computing centers and the cloud. The similarity threshold can be dynamically adjusted through historical data to improve the robustness of miscellaneous item recognition.

[0037] It should be further explained that in addition to using lidar to scan objects, any one or a combination of methods including but not limited to ultrasonic radar, 4D millimeter wave radar, and matrix camera are also used.

[0038] As an optional embodiment, when the recognition result does not match the inventory data verification, or when different types of item units exist in the same type of item units, by identifying one or more item units in the same type of item units, the three-dimensional model of the item or the multi-level classification atlas is compared to distinguish the miscellaneous items in the same type of item units.

[0039] As an optional embodiment, refer to Figure 2 In the process of identifying miscellaneous items, for the item unit blocked by the identified item unit, the identification of the blocked item unit is achieved through local feature recognition and extraction of the item unit, which specifically includes the following steps: S1001. Using an image processing algorithm to extract the surface texture, geometric shape, or identification symbol of a target object unit as a local feature; S1002: Compare the local features with the corresponding features of the top covering item unit. If the similarity is greater than or equal to a preset threshold, it is determined that the target item unit and the top item unit belong to the same category. S1003. If the similarity is less than a preset threshold, it is determined that the target item unit and the top item unit belong to different categories; S1004. Compare the obtained local features of the miscellaneous items with the distinguishing categories, and combine the item quantity and inventory data to generate the proportion of the miscellaneous items in the total number of item units. Determine the category and position of the item unit based on whether the miscellaneous items have a dominant proportion and the inventory data. S1005. The preset threshold is dynamically adjusted according to historical verification data.

[0040] Furthermore, the three-dimensional model features of the object generated by the lidar scan (such as volume, shape, and surface texture) are compared with the recorded data in the inventory management system. If the difference rate exceeds the preset threshold, the miscellaneous item identification process is triggered, and the spatial distribution of similar item units is analyzed through a clustering algorithm. If the point cloud features of the local area (such as reflection intensity, geometric curvature, and convexity) are detected to deviate from the similar benchmark model by more than a dynamic threshold (adaptively adjusted according to historical data), it is marked as a potential miscellaneous area.

[0041] Furthermore, the point cloud data of the suspected miscellaneous area is segmented, and its local feature vectors are extracted. The extracted local features are compared with the pre-stored three-dimensional model library, and the similarity score is calculated using a fast point feature histogram algorithm. If the score is lower than the threshold, it is determined to be a non-similar item, and a multi-level classification atlas is called for hierarchical matching. If the combined features of the miscellaneous items and a certain secondary atlas have a high degree of match, it is determined to be a combined item rather than a misidentification.

[0042] Furthermore, based on historical verification records, such as the false recognition rate and the distribution pattern of miscellaneous items, the similarity threshold is dynamically adjusted through a machine learning model. The learning model used is a random forest. Multi-angle 2D depth maps are generated for the segmented miscellaneous units, such as top view, side view, oblique view or any angle view. SSIM similarity calculation is performed with the multi-view views of the three-dimensional model library, and a threshold is set to further verify the recognition consistency.

[0043] Furthermore, when mismatching occurs due to occlusion or deformation, the GAN generative adversarial network is used to reconstruct the complete three-dimensional model and correct the recognition results. After confirming that the object is a foreign object, a difference report is generated and its location, quantity and possible category are marked.

[0044] It should be further explained that miscellaneous items include other item units that do not conform to the placement position, or another item unit appears among multiple similar item units, or the placement position of the entire item unit is incorrect, etc.

[0045] Furthermore, the subsequent identification and processing method is to first determine whether the identified item unit is in the correct placement position when identifying one or more item units in the same type of item unit. After ensuring that it is correct, or after removing the incorrectly placed item unit, for cross-stored items such as drone brushless motors, drone blades, or a combination of the two, on the one hand, through feature recognition, the position between the connection port (connection end) and the connection port (connection end) of another item unit is identified. If it is in a connected state, it is determined to be a second-class item unit, or a multi-level item unit, and the above-mentioned items that do not match the placement position of the item unit are removed and rearranged.

[0046] Based on the above, the accuracy of miscellaneous item recognition can be improved by adjusting the threshold driven by historical data. In combination with the multi-level classification atlas and the FPFH algorithm, the confusion problem between combined items and miscellaneous items can be solved, the misjudgment rate can be reduced, and an automated closed loop can be achieved from difference detection to inventory correction to improve warehouse management efficiency.

[0047] As an optional embodiment, refer to Figure 3 The occluding object unit generates a full image of the occluding object unit based on the identified local features. The method specifically includes the following steps: S2001, obtaining point cloud data of the occluding object unit and extracting local feature vector data; S2002: performing similarity matching between the extracted features and the pre-stored object unit library, performing morphological compensation on the unexposed areas using a generative model, and generating a complete three-dimensional model; S2003: Calculate available stacking space and establish physical constraints based on the dimensional parameters of the reconstructed complete three-dimensional model and the bearing surface characteristics of the bottom of the storage container; S2004. Simulate the stacking process of object units in a virtual environment, correct the morphological deformation by combining material mechanical parameters, and optimize the placement plan through a heuristic algorithm; S2005: Output the maximum capacity based on the stable stacking simulation result, and determine the shape of the obstructing object unit.

[0048] Furthermore, double verification can be performed by exposing the bottom features through mechanical disturbance.

[0049] Furthermore, as an optional embodiment, the item unit matching and compensation module further includes: A hierarchical item unit library is constructed, classified according to connection end type and surface topological features, and a conditional generative adversarial network is used for occluded areas. The compensation form with the optimal probability distribution is generated based on the exposed features. The dimensional measurement error caused by tilted placement is eliminated through an elastic deformation algorithm, and the deformation parameters are dynamically adjusted according to the material elastic modulus.

[0050] Furthermore, as an optional embodiment, further calculations can be performed through a space calculation module, including: The alpha shape algorithm is used to extract the irregular support boundary at the bottom of the storage container. The stress distribution of the load-bearing surface is calculated through finite element analysis, and high-risk collapse areas are marked. Combined with the projection of the center of gravity of the item unit and the friction coefficient, a two-dimensional thermal map of the allowable stacking area is generated.

[0051] Furthermore, as an optional embodiment, a bending stiffness coefficient matrix is introduced into the flexible object unit to simulate the shape collapse under different stacking angles. A reinforcement learning algorithm is used to train the stacking strategy. The reward function includes three dimensions: space utilization, structural stability, and shape retention. During the simulation process, the normal pressure of the contact surface between the object units is calculated in real time, and position adjustment is triggered when the threshold is exceeded.

[0052] As an optional embodiment, refer to Figure 4 The method for estimating the volume and quantity of items by combining the storage height and location information of the item unit includes the following steps: S3001. Extracting dimensional data and spatial coordinate position information of the three-dimensional model based on the acquired three-dimensional model of the object; S3002: Acquire the data of the item unit at the top, and determine the size information of the top item unit in combination with the item model atlas library; S3003. Obtain the physical characteristics of the item units and the physical constraints of the item storage location based on the item model atlas library, and eliminate over-limit item units. S3004: Simulate stacking of item units in a regular or irregular manner, implement spatial constraints based on item storage locations, and correct abnormal item units. S3005. Combining the point cloud data with the image information, and comparing the multi-angle views to verify whether the simulated stacking state is consistent with the three-dimensional model of the object, and correcting errors caused by occlusion; S3006. Combine historical item unit storage record data to cross-validate the current simulation stacking method and dynamically adjust the threshold and estimation results; S3007. Obtain the number of items based on the verification result and the simulated stacking method.

[0053] According to the above steps, by combining point clouds, images and historical data, it is possible to use a single sensor to identify objects and the number of objects, and reduce the recognition error based on historical data. By introducing physical engine simulation and simulated stacking, it is possible to use a static threshold method to estimate the number of object units of different materials. Furthermore, through multi-angle view comparison, it is possible to identify object units in disordered stacking scenarios.

[0054] As an optional embodiment, the regular simulation stacking method includes the following steps: S4001. Determine whether the top item unit is in a regular stacking state based on the acquired top item unit data; S4002: If the stacking state is regular, simulate the stacking state using a regular method; S4003. Generate a theoretical stacking model based on the unit size parameters of the top items and the same-layer and inter-layer arrangement rules; S4004. Eliminate excessive and abnormal stacking by dividing the total stacking height by the unit height of each item, taking into account the physical constraints of the storage location. The irregular stacking simulation method includes the following steps: S5001. If the top item unit is in an irregular stacking state, simulate the stacking state in an irregular manner; S5002: Set physical contact relationship reasoning, mechanical simulation and shape correction, and model collision volume, and implement simulation ideas for various stacking methods of item units based on the physics engine; S5003. After obtaining simulation assumptions of multiple stacking methods, the morphological similarity is compared with the obtained irregular storage state of the top item unit. If the similarity exceeds a threshold, it is determined to be consistent. If it is lower than the similarity threshold, this stacking assumption is eliminated.

[0055] According to the above steps, when constructing the point cloud adjacency graph, independent units are divided by point cloud density clustering, the minimum distance between each unit is calculated (if the closest point distance is less than the safety threshold, it is determined to be in contact), and a stacking topology graph is constructed based on the contact relationship to identify implicit support relationships (such as the bottom unit supporting the upper unit).

[0056] Furthermore, by introducing a physical engine (such as Bullet, PhysX, etc.) to simulate the natural stacking form under the action of gravity, input parameters such as the mass distribution of a single piece and the friction coefficient, a virtual model after stable stacking is generated, and by comparing the matching degree between the simulated form and the actual point cloud form, the contact relationship and stacking order are reversely corrected.

[0057] Furthermore, by extracting the exposed local geometric features (such as the shape of the connection end, identification symbols, texture, bumps, etc.) from the occluded area (such as the bottom features covered by the upper units), hash matching is performed with the pre-stored model library, and combined with the texture continuity in the multi-view image (such as the degree of texture fracture at the occlusion boundary), the rationality of the stacking logic is verified.

[0058] Please follow Figure 5 , a warehouse item intelligent identification system based on laser radar scanning, including: Data acquisition module: Deployed on a mobile carrier, it uses lidar to scan and generate high-density point cloud data, simultaneously recording spatial coordinates and intensity information. Combined with a visual camera, it performs point cloud and image registration and color mapping to generate enhanced point cloud data with texture attributes. Multi-level classification atlas database: stores a multi-level classification model divided by basic item units, splicing relationships and storage areas. Each level and each item unit is associated with a unique identification code and a three-dimensional feature vector library; Intelligent recognition engine: Integrates convolutional neural networks and graph neural networks to perform object unit recognition and image data processing; Dynamic verification compensation module: Perform the following steps for the occluded object unit: Extract local feature vectors and compare them with the top cover, dynamically adjust the similarity threshold, and distinguish similar miscellaneous items; A generative adversarial network is used to reconstruct a complete 3D model of the unexposed area, and the stacking state is simulated in combination with physical constraints. Collaboration module: Outputs recognition data and proofreading results to the warehouse scheduling system, dynamically plans routes, and triggers exception handling instructions.

[0059] Furthermore, the enhanced point cloud data and item feature vectors are transmitted to the multi-level classification atlas database through the data acquisition module for matching the pre-stored multi-level classification model (basic unit, combination unit, area division), and the three-dimensional feature vector library and classification rules are provided through the intelligent recognition engine, supporting the neural network to extract surface texture and other features, and the inference splicing relationship is realized through the neural network. The preliminary recognition results are output through the dynamic verification and compensation module. The results are preliminarily judged according to the category, position, and quantity, triggering local feature comparison and occlusion area reconstruction, and then the reconstructed complete three-dimensional model and the corrected feature data are fed back through the multi-level classification atlas database to dynamically update the classification atlas. The corrected recognition results are transmitted to the collaborative module through the dynamic verification and compensation module, and combined with the inventory data to trigger difference warnings or precise measurement instructions. Finally, the path planning instructions and exception handling signals are output through the data system between the collaborative module and the warehouse scheduling system, and the inventory records are updated synchronously and recorded in the log.

[0060] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0061] According to another aspect of an embodiment of the present invention, an electronic device is provided. The electronic device includes a memory and a processor; the memory is used to store programs; and the processor executes the programs to implement any one of the aforementioned methods.

[0062] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The storage medium stores a computer program. When the computer program is executed by a processor, any one of the aforementioned methods is implemented.

[0063] According to yet another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, which implements any of the aforementioned methods when executed by a processor.

[0064] The foregoing is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained herein shall, unless otherwise specified or limited, be implemented in accordance with conventional means in the art.

Claims

1. A method for intelligent identification of warehouse items based on laser radar scanning, characterized in that: The steps include: Scan the object shape through the LiDAR device, obtain point cloud data and image information, and upload the scan data to the cloud or edge computing center in real time; Based on the preset object model atlas library, the scanned data is matched with the atlas, and multi-level classification atlases are divided according to the basic item units, splicing relationships or storage areas; According to the hierarchical structure of the multi-level classification atlas, a unique identification code is generated for each type of item unit and associated with its three-dimensional model features; Use convolutional neural networks to analyze point cloud data, build a 3D model of the scanned objects, and estimate the volume and quantity of the objects by combining storage height and location information; Obtain photos of the objects, segment them into independent units through point cloud data processing, compare them with multi-angle views of the object's 3D model features, and verify the consistency of the recognition results; Based on the item association algorithm, the splicing or storage relationship of adjacent item units is judged, the second-level and above atlases are quickly matched, and the recognition results are compared with the inventory data to trigger difference warnings or precise measurement instructions.

2. The method for intelligent warehouse item identification based on laser radar scanning according to claim 1, characterized in that: When the identification result does not match the inventory data, or when different types of item units exist in the same type of item units, one or more item units in the same type of item units are identified and compared with the item three-dimensional model or multi-level classification atlas to distinguish miscellaneous items in the same type of item units.

3. The method for intelligent identification of warehouse items based on laser radar scanning according to claim 2 is characterized in that: During the process of identifying the miscellaneous items, for the item units blocked by the identified item units, the identification of the blocked item units is achieved by identifying and extracting the local features of the item units, which specifically includes the following steps: Image processing algorithms are used to extract the surface texture, geometric shape or identification symbol of the target object unit as local features; Comparing the local features with corresponding features of the top covering item unit, and if the similarity is greater than or equal to a preset threshold, determining that the target item unit and the top item unit belong to the same category; If the similarity is less than the preset threshold, it is determined that the target item unit and the top item unit belong to different categories; Compare the obtained local features of the miscellaneous items with the distinguishing categories, combine the number of items and inventory data, and generate the proportion of the miscellaneous items in the total number of item units. Determine the category and position of the item unit based on whether the number of miscellaneous items occupies a dominant proportion and the inventory data. The preset threshold is dynamically adjusted according to historical verification data.

4. The method for intelligent identification of warehouse items based on laser radar scanning according to claim 3 is characterized in that: The obstructing object unit generates a full image of the obstructing object unit based on the identified local features, and the method specifically includes the following steps: Obtain the point cloud data of the occluded object unit and extract the local feature vector data; The extracted features are matched with the pre-stored object unit library for similarity, and the generative model is used to compensate for the unexposed areas to generate a complete three-dimensional model; Based on the dimensional parameters of the reconstructed complete 3D model and the bearing surface characteristics of the storage container bottom, the available stacking space is calculated and physical constraints are established; Simulate the stacking process of object units in a virtual environment, correct morphological deformation based on material mechanics parameters, and optimize the placement plan through heuristic algorithms; Based on the stable stacking simulation results, the maximum capacity is output, and the shape of the obstructing item unit is determined.

5. The method for intelligent identification of warehouse items based on laser radar scanning according to claim 1 is characterized in that: The method for estimating the volume and quantity of items by combining the storage height and location information of the item unit includes the following steps: Extracting the dimensional data and spatial coordinate position information of the 3D model based on the acquired 3D model of the object; Obtain the data of the item unit at the top, and determine the size information of the top item unit in combination with the item model atlas library; Obtain the physical characteristics of item units and the physical constraints of item storage locations based on the item model atlas library, and eliminate over-limit item units; Simulate the stacking of item units in a regular or irregular manner, implement spatial constraints based on item storage locations, and correct abnormal item units; Combining point cloud data with image information, and comparing multiple angles, verifying whether the simulated stacking state is consistent with the object's 3D model can correct errors caused by occlusion. Combined with historical item unit storage record data, cross-validate the current simulation stacking method and dynamically adjust the threshold and estimation results; Get the number of items based on the verification results and simulated stacking method.

6. The method for intelligent warehouse item identification based on laser radar scanning according to claim 5, characterized in that: The rule-based stacking method includes the following steps: According to the acquired top item unit data, determining whether the top item unit is in a regular stacking state; If it is a regular stacking state, the regular method is used to simulate the stacking state; According to the unit size parameters of the top items, a theoretical stacking model is generated according to the same-layer and inter-layer arrangement rules; By dividing the total stacking height by the unit height of a single item and combining it with the physical constraints of the storage location, we can eliminate any abnormal stacking scenarios that exceed the limit. The irregular stacking simulation method comprises the following steps: If the top item unit is in an irregular stacking state, use an irregular method to simulate the stacking state; Set physical contact relationship reasoning, mechanical simulation and shape correction, model collision volume, and simulate various stacking methods of item units based on the physics engine; After obtaining simulation assumptions of multiple stacking methods, the morphological similarity is compared with the obtained irregular storage state of the top item unit. If the similarity exceeds the threshold, it is determined to be consistent. If it is lower than the similarity threshold, this stacking assumption is eliminated.

7. A warehouse item intelligent identification system based on laser radar scanning, using the method according to any one of claims 1 to 6, characterized in that: include: Data acquisition module: Deployed on a mobile carrier, it uses lidar to scan and generate high-density point cloud data, simultaneously recording spatial coordinates and intensity information. Combined with a visual camera, it performs point cloud and image registration and color mapping to generate enhanced point cloud data with texture attributes. Multi-level classification atlas database: stores a multi-level classification model divided by basic item units, splicing relationships and storage areas. Each level and each item unit is associated with a unique identification code and a three-dimensional feature vector library; Intelligent recognition engine: Integrates convolutional neural networks and graph neural networks to perform object unit recognition and image data processing; Dynamic verification compensation module: Perform the following steps for the occluded object unit: Extract local feature vectors and compare them with the top cover, dynamically adjust the similarity threshold, and distinguish similar miscellaneous items; A generative adversarial network is used to reconstruct a complete 3D model of the unexposed area, and the stacking state is simulated in combination with physical constraints. Collaboration module: Outputs recognition data and proofreading results to the warehouse scheduling system, dynamically plans routes, and triggers exception handling instructions.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images