An automatic vertical storage system and vertical storage method based on three-dimensional point cloud technology

Abstract

The present invention discloses an automatic vertical warehouse system and a vertical warehouse method based on three-dimensional point cloud technology, comprising: a warehouse management system, and an image acquisition and data preprocessing module, a warehouse storage model construction module, a cargo identification and classification module, a position determination and strategy formulation module, an automatic handling planning and execution module, and an offset detection and position adjustment module installed inside the warehouse management system; the present invention determines the predefined grasping force F0 of a manipulator on cargo and the actual grasping force F1 of the cargo, as well as the three-dimensional vectors P0 and P1 of the contact point with the cargo by constructing a warehouse storage model, obtains the offset information of the cargo, controls the rotation angle and the extension distance of the manipulator, so that the cargo can be placed in the predefined position, improves the clarity of the cargo edge in the three-dimensional point cloud data, ensures the clear boundaries between the cargoes, reduces the difficulty of the point cloud segmentation algorithm, and thus improves the accuracy of subsequent cargo identification and positioning.

Description

Technical Field

[0001] The present invention relates to the technical field of model construction, and specifically to a method for constructing a warehouse storage model simulating an automatic vertical warehouse using three-dimensional point cloud technology, and in particular to an automatic vertical warehouse system and a vertical warehouse method thereof based on three-dimensional point cloud technology. Background Art

[0002] 3D point cloud technology is a data structure and method for representing objects or scenes in 3D space. It is a collection of a series of 3D coordinate points, each of which contains location information in space. These points are obtained by discrete sampling of the actual object or scene surface. The collection of points can be regarded as a discrete representation of the scene surface in a given coordinate system. In addition to basic spatial coordinate information, 3D point cloud data may also contain other attribute information, such as color, classification value, intensity value, and time. In order to obtain point cloud data, sensors such as lidar, depth camera, or structured light scanner can be used to scan and collect the acquired object (such as a warehouse) and convert it into point cloud data. After obtaining the point cloud data, it needs to be processed and organized for subsequent analysis and application. The processing of point cloud data includes filtering, registration, segmentation, and feature extraction. Filtering is used to remove noise and outliers. The registration criterion is to align multiple point cloud data to the same coordinate system to achieve multi-perspective integration. Segmentation can divide the point cloud into subsets with different attributes or structures, such as separating objects in the scene from the background.

[0003] However, the accuracy of 3D point cloud data will be affected by environmental factors. For example, the way goods are placed may affect the collection and quality of 3D point cloud data, and thus affect the performance of the system. When goods are stacked in a warehouse, due to some external factors and the storage environment, the goods may be tilted or moved. In the subsequent storage and handling strategy of the warehouse goods, the goods need to be moved to other locations in the warehouse. The placement of the goods cannot be adjusted according to the offset of the goods, and the goods will still be unevenly placed, which may cause the edges of the goods in the point cloud data to be more blurred and the boundaries between different goods to be unclear. This will increase the difficulty of the point cloud segmentation algorithm and may lead to inaccurate segmentation, which in turn affects the subsequent identification and positioning of goods.

[0004] Therefore, it is necessary to design an automatic vertical storage system and a vertical storage method based on three-dimensional point cloud technology to solve the above problems. Summary of the invention

[0005] The present invention overcomes the shortcomings of the prior art and provides an automatic vertical storage system and a vertical storage method based on three-dimensional point cloud technology.

[0006] To achieve the above object, the technical solution adopted by the present invention is: an automatic vertical library method based on three-dimensional point cloud technology, comprising the following steps:

[0007] S1. Use several cameras to ensure that images of the interior of the warehouse are captured at multiple different angles and positions, accurately capture the internal layout of the warehouse and the three-dimensional shape and position of the goods inside the warehouse, extract pixel depth information in the image, obtain three-dimensional point cloud data, and pre-process the obtained three-dimensional point cloud data and upload it to the warehouse management system;

[0008] S2. Based on the three-dimensional point cloud data of goods and warehouses in the warehouse management system, extract the relevant features of the warehouse and goods, use the machine learning algorithm to build a warehouse storage model, and define the grasping force F0 of the goods according to the relevant features of the warehouse and goods, and the three-dimensional vector of the contact point is recorded as P0;

[0009] S3. Then, according to the real-time status of the warehouse, the point cloud segmentation algorithm is used to segment the 3D point cloud data of the warehouse and the goods, and the warehouse is divided into different goods areas. The relevant features of the goods and the key features of the goods area are extracted for each goods area, and compared with the goods model in the warehouse management system to identify different types of goods;

[0010] S4. Based on the identified cargo types and key features of the cargo area, the warehouse storage model is aligned with the point cloud data using 3D registration technology to obtain the precise location of the cargo in the warehouse. Based on the cargo location information and related features, as well as the real-time status of the warehouse, the storage and handling strategy of the warehouse cargo is re-formulated to ensure that the cargo is transferred to the designated cargo area.

[0011] S5. According to the storage and handling strategy of the warehouse goods, the relevant characteristics of the goods, the relevant characteristics of the warehouse and the key characteristics of the goods area, the path and grasping action of the automated handling robot are planned, and the grasping force F1 of the manipulator is recorded by the pressure sensor on the manipulator, and the three-dimensional vector of the contact point is recorded as P1;

[0012] S6. By comparing the two grasping forces F0 and F1 of the manipulator and the two three-dimensional vector records P0 and P1 of the contact point, it is determined whether the goods are offset, the offset information of the goods is calculated, and based on the offset information, the position where the manipulator places the goods is controlled to ensure the accuracy of the placement of the goods.

[0013] In a preferred embodiment of the present invention, in S1, three-dimensional point cloud data is obtained, specifically, the depth information of each pixel is determined according to the parallax between pixels in the image, and the depth information of each pixel is combined to generate a three-dimensional point cloud data;

[0014] Preprocessing includes filtering and denoising, simplification and compression, coordinate transformation and alignment.

[0015] In a preferred embodiment of the present invention, the relevant characteristics of the goods include: the size, weight, shape, surface characteristics of the goods, and the height, width and length of the goods stacking; the relevant characteristics of the warehouse include: the size, shape and arrangement of the shelves in the warehouse; the key characteristics of the cargo area include: the size, shape and arrangement of the shelves in the warehouse, and the relative position of the goods in the shelves.

[0016] In a preferred embodiment of the present invention, in said S2, the method for constructing a cargo model comprises the following steps:

[0017] S21. Based on extracting relevant features of the warehouse and goods from the three-dimensional point cloud data of the goods and warehouse, a warehouse storage model is constructed through a machine learning model;

[0018] S22. Use the three-dimensional point cloud data of the goods as a training set to optimize the warehouse storage model parameters, and compare it with the goods model in the warehouse storage model to distinguish different types of goods;

[0019] S23. Create a model library for each type of goods in the warehouse management system, store the corresponding goods model in the library, assign a unique label to it, and integrate the constructed goods model into the model library of the warehouse management system;

[0020] S24. Based on the relevant characteristics of the goods, the grasping force is determined to be F0 through experimental simulation, and the three-dimensional vector of the contact point is recorded as P0, where P0 is the center of gravity of the goods.

[0021] In a preferred embodiment of the present invention, in said S3, the real-time status of the warehouse includes: the real-time quantity, type and location of the goods, the warehousing, outbound, movement and storage status of the goods, and the idle status of the warehouse cargo area.

[0022] In a preferred embodiment of the present invention, in S3, the point cloud segmentation algorithm is specifically an edge-based point cloud segmentation algorithm.

[0023] In a preferred embodiment of the present invention, in the S4, the precise position of the goods in the warehouse is obtained, specifically, the features of the warehouse storage model are extracted, and the features in the three-dimensional point cloud data of the goods and the warehouse are extracted, and the two sets of features are compared; then the rotation matrix and translation vector transformation relationship between the three-dimensional point cloud data of the goods and the warehouse, and the three-dimensional point cloud data of the warehouse storage model are calculated according to the matched features, and the transformation relationship is refined through an iterative optimization algorithm to minimize the alignment error between the two three-dimensional point cloud data. The optimized transformation relationship is used to map the cargo location information in the warehouse storage model to the real-time three-dimensional point cloud data to obtain the precise position of the goods in the warehouse.

[0024] In a preferred embodiment of the present invention, in said S4, the storage and handling strategy of the warehouse goods is to reclassify and prioritize the goods according to the importance, urgency and operational requirements of the goods.

[0025] In a preferred embodiment of the present invention, in S6, the offset information specifically includes an offset angle. The method for calculating the offset information of the goods is specifically to calculate the difference between the two contact force vectors by the formula ΔF=F1-F0. The offset direction can be determined by analyzing the change in the contact point position. If P1 moves relative to P0, then the offset direction of the goods is the direction of P1-P0. This direction vector is normalized to obtain a unit vector U in the offset direction, and U=(P1-P0)÷|P1-P0|, wherein |P1-P0| is the Euclidean distance between P1 and P0. The offset angle can be obtained by calculating the angle between the contact force change vector ΔF and the unit vector U in the offset direction, and the formula is θ=acos[(ΔF*U)÷(|ΔF|*|U|)], wherein acos is the inverse cosine function.

[0026] An automatic vertical warehouse system based on three-dimensional point cloud technology includes: a warehouse management system, and an image acquisition and data preprocessing module, a warehouse storage model building module, a cargo identification and classification module, a position determination and strategy formulation module, an automatic handling planning and execution module, and an offset detection and position adjustment module installed inside the warehouse management system.

[0027] The image acquisition and data preprocessing module is responsible for taking comprehensive photos of the goods and the warehouse, acquiring three-dimensional point cloud data of the goods and the warehouse, and preprocessing the acquired three-dimensional point cloud data;

[0028] The warehouse storage model building module is used to extract the relevant features of the warehouse and the goods, use the machine learning algorithm to build the warehouse storage model according to the relevant features of the warehouse and the goods, and at the same time, define the grasping force F0 of the goods and the three-dimensional vector P0 of the contact point according to the relevant features of the warehouse and the goods;

[0029] The cargo identification and classification module uses a point cloud segmentation algorithm to segment the three-dimensional point cloud data of the warehouse and cargo according to the real-time status of the warehouse, and divides the warehouse into different cargo areas, extracts cargo-related features and key features of each cargo area, and compares them with the cargo model in the warehouse management system, thereby identifying different types of cargo;

[0030] The location determination and strategy formulation module is used to obtain the precise location of the goods in the warehouse and re-formulate the storage and handling strategy of the warehouse goods based on the location information and related characteristics of the goods and the real-time status of the warehouse;

[0031] The automated handling planning and execution module is used to plan the path and grasping action of the automated handling robot, and record the grasping force F1 of the manipulator and the three-dimensional vector P1 of the contact point;

[0032] The offset detection and position adjustment module is used to compare the two grasping forces F0 and F1 of the manipulator and the two three-dimensional vectors P0 and P1 of the contact point, calculate the offset information of the goods, and control the manipulator to adjust the position of the placed goods according to the offset information.

[0033] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0034] (1) The present invention provides an automatic vertical warehouse system and a vertical warehouse method based on three-dimensional point cloud technology. By constructing a warehouse storage model, the predefined grasping force F0 of the manipulator on the goods and the actual grasping force F1 of the goods, as well as the three-dimensional vectors P0 and P1 of the contact points with the goods are determined, and the offset information of the goods is calculated. The manipulator is controlled to adjust the position of the goods according to the offset information, so that the goods are placed accurately, the clarity of the edges of the goods in the three-dimensional point cloud data is improved, the boundaries between the goods are ensured to be clear, and the difficulty of the point cloud segmentation algorithm is reduced, thereby improving the accuracy of subsequent goods identification and positioning, and improving the accuracy and efficiency of automatic vertical warehouse of goods.

[0035] (2) The present invention constructs a warehouse storage model and combines it with three-dimensional registration technology to align the warehouse storage model with real-time three-dimensional point cloud data, thereby accurately determining the location of goods in the warehouse and providing accurate target location information for automated transport robots, thereby helping to reduce errors and delays in the transport process and improving transport efficiency.

[0036] (3) The present invention formulates storage and handling strategies for warehouse goods in real time according to the real-time status of the warehouse, thereby realizing automatic storage of goods, thereby greatly improving the response speed to dynamic changes in the warehouse, ensuring the smoothness and efficiency of warehouse operations, helping to reduce resource waste, and improving the overall operating efficiency of the warehouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 The present invention is a flowchart of an automatic vertical library method based on three-dimensional point cloud technology. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0041] like Figure 1 As shown, an automatic library method based on three-dimensional point cloud technology includes the following steps:

[0042] S1. Use several cameras to ensure that images of the interior of the warehouse are captured at multiple different angles and positions, accurately capture the internal layout of the warehouse and the three-dimensional shape and position of the goods inside the warehouse, extract pixel depth information in the image, obtain three-dimensional point cloud data, and perform pre-processing of filtering and denoising, simplification and compression, coordinate conversion and alignment on the obtained three-dimensional point cloud data. Among them, coordinate conversion and alignment is to convert the point cloud data into a unified coordinate system and align it to ensure data consistency, and then upload the pre-processed three-dimensional point cloud data to the warehouse management system;

[0043] Specifically, a suitable location is selected in the warehouse to install the stereo camera system. The stereo camera system includes two or more cameras with fixed relative positions and angles to capture images from different perspectives. The installation of the stereo camera system is ensured to be stable, and the relative positions and angles between the cameras are calibrated to capture images. The images are processed using 3D reconstruction software to compare the position differences (i.e., parallax) of the same feature points in images from different perspectives. The depth information of each pixel is determined based on the parallax between pixels. The depth information of each pixel is combined to generate a 3D point cloud data. This data contains the 3D shape and position information of objects in the warehouse, providing a detailed 3D model of the goods and warehouse for subsequent processing and analysis.

[0044] Among them, pixel depth information refers to the physical distance from the surface point of an object in the scene represented by each pixel in the image to the optical center of the camera lens (or a fixed reference plane), and the pixel depth information specifically includes the depth value and the two-dimensional coordinates of the pixel, so that the three-dimensional coordinates of the object surface can be obtained.

[0045] S2. Based on the three-dimensional point cloud data of the goods and the warehouse in the warehouse management system, extract the relevant features of the warehouse and the goods, wherein the relevant features of the goods include: the size, weight, shape, surface characteristics of the goods, and the height, width and length of the goods stacking, and the relevant features of the warehouse include: the size, shape and arrangement of the shelves in the warehouse. By obtaining the relevant features of the warehouse and the goods, the adaptability of the goods and the shelves in the warehouse can be obtained, ensuring that the goods can be accurately placed at the corresponding shelf positions;

[0046] Among them, a warehouse storage model is constructed using a machine learning algorithm, and the grasping force F0 of the goods and the three-dimensional vector of the contact point are defined as P0 according to the relevant characteristics of the warehouse and the goods:

[0047] S21. Based on extracting relevant features of warehouses and goods from three-dimensional point cloud data of goods and warehouses, a warehouse storage model is constructed through a machine learning model. This warehouse storage model specifically expresses the placement of goods in the warehouse. Through this warehouse storage model, the location information of goods can be obtained quickly and accurately without manual search, which greatly improves the efficiency of goods management and provides a basis for navigation and positioning for automated handling robots, enabling them to independently complete the task of handling goods;

[0048] S22. Use the three-dimensional point cloud data of the goods as a training set to optimize the warehouse storage model parameters, and compare it with the goods model in the warehouse storage model to distinguish different types of goods;

[0049] S23. Create a model library for each type of goods in the warehouse management system, store the corresponding goods model in the library, and conveniently update, modify and delete the model to ensure the accuracy and timeliness of the model. Assign a unique label to each type of goods, and integrate the constructed goods model into the model library of the warehouse management system. The corresponding goods model can be quickly queried through the unique label, which facilitates subsequent operations such as goods identification and positioning.

[0050] S24. Based on the relevant characteristics of the goods, the grasping force is determined to be F0 through experimental simulation, and the three-dimensional vector of the contact point is recorded as P0, where P0 is the center of gravity of the goods or the most stable grasping point, to ensure that the manipulator can maintain balance when grasping the goods.

[0051] Among them, through the constructed warehouse storage model, the specific situation of the placement of goods in the warehouse can be understood. According to the specific situation of the placement of goods, the designated position of the automated handling robot when handling goods and the grasping action of the manipulator can be predefined, thereby controlling the grasping strength of the manipulator and the extension distance of the manipulator, so that the subsequent automated handling robot can grasp the goods on the shelf according to the predefined position and grasping action.

[0052] S3. Then, according to the real-time status of the warehouse, the real-time status of the warehouse includes: the real-time quantity, type and location of the goods, the warehousing, out-of-warehouse, movement and storage of the goods, and the idle state of the warehouse goods area, the three-dimensional point cloud data of the warehouse is segmented into different goods areas using an edge-based point cloud segmentation algorithm. The core of the edge-based point cloud segmentation algorithm lies in edge detection. The edge is the basic feature that describes the shape of the point cloud object, which usually corresponds to the area where the intensity, color or normal vector changes significantly, that is, as the adjacent area of ​​different types of goods, and the adjacent area between goods. The purpose of edge detection is to find these points or areas with significant changes as the boundaries of different goods areas. Each area corresponds to a goods or a group of similar goods.

[0053] For each cargo area, the relevant features of the cargo and the key features of the cargo area are extracted, wherein the key features of the cargo area include: the size, shape and arrangement of the shelves in the warehouse, and the relative position of the cargo in the shelves. The relevant features of the cargo and the key features of the cargo area are combined and compared with the cargo model in the warehouse management system to identify different types of cargo.

[0054] S4. Based on the identified key features of the cargo type and cargo area, the warehouse storage model is aligned with the point cloud data using 3D registration technology. The two sets of features are compared by extracting the features of the warehouse storage model and the features in the 3D point cloud data of the cargo and the warehouse. The rotation matrix and translation vector transformation relationship between the 3D point cloud data of the cargo and the warehouse and the 3D point cloud data of the warehouse storage model are calculated based on the matched features. The transformation relationship is refined through an iterative optimization algorithm to minimize the alignment error between the two 3D point cloud data. The cargo location information in the warehouse storage model is mapped to the real-time 3D point cloud data using the optimized transformation relationship to obtain the precise location of the cargo in the warehouse, providing accurate target location information for the automated handling robot, helping to reduce errors and delays in the handling process and improving handling efficiency.

[0055] Based on the location information and related characteristics of the goods, as well as the real-time status of the warehouse, the storage and handling strategy of the warehouse goods is re-formulated to ensure that the goods are transferred to the designated cargo area. The storage and handling strategy of the warehouse goods is to reclassify and prioritize the goods according to their importance, urgency and warehouse operation needs.

[0056] In the event of changes in the priority and urgency of goods, such as high-priority and urgent goods, the warehouse management system will give priority to storage locations close to the exit or with shorter transportation paths in order to quickly respond to customer needs; for important but non-urgent goods, the system will look for a stable and secure storage location to ensure the safety and quality of the goods; for low-priority goods, the system will consider using the remaining space in the warehouse for storage to optimize space utilization.

[0057] S5. Plan the path and grasping action of the automated handling robot according to the storage and handling strategy of the warehouse goods, the relevant characteristics of the goods, the relevant characteristics of the warehouse and the key characteristics of the goods area. Use the pressure sensor on the manipulator to sense the size of the clamping force in real time and set an initial clamping force target value, which is F0. When the sensor detects that the clamping force is less than the target value, the control system will automatically adjust the clamping mechanism to increase the clamping force until the goods are completely grasped. Record the gripping force F1 of the manipulator and the three-dimensional vector of the contact point as P1.

[0058] Specifically, after obtaining the precise location of the goods in the warehouse, the warehouse management system will plan the path of the manipulator from the current position to the target position according to the real-time status of the warehouse (such as shelf occupancy, whether the aisle is unobstructed, etc.) and the target position of the goods. The goal of path planning is to ensure that the manipulator can move the goods safely and efficiently while avoiding collisions with other equipment or obstacles in the warehouse.

[0059] After the path planning is completed, the automated handling robot will move to the location of the target goods according to the planned path. Then, according to the type, size, shape and other information of the goods, the manipulator will adjust its grasping mechanism (such as clamps, suction cups) to ensure that the goods can be grasped stably and safely. During the grasping process, the automated handling robot will move to the predetermined position according to the preset warehouse storage model and maintain a predetermined distance from the shelf. The manipulator is always parallel to the ground. The manipulator will control the grasping force and the extension distance of the manipulator according to the predefined grasping action in the warehouse storage model. Once the goods are successfully grasped, the manipulator will move the goods to the designated location along the planned path.

[0060] By predefining the moving position and grasping action of the automated handling robot through the warehouse storage model and comparing them with the actual grasping action of the automated handling robot, the difference between the two grasping positions and grasping forces can be directly compared, and the predefined grasping force F0 of the manipulator on the goods and the actual grasping force F1 of the goods, as well as the three-dimensional vectors P0 and P1 of the contact point with the goods can be determined to determine whether the goods are offset.

[0061] S6, by comparing the two grasping forces F0 and F1 of the manipulator, and comparing the two three-dimensional vector records P0 and P1 of the contact point, it is determined whether the goods are offset, the offset information of the goods is calculated, and based on the offset information, the position where the manipulator places the goods is controlled to ensure the accuracy of the placement of the goods;

[0062] The offset information specifically includes an offset angle. The method for calculating the offset information of the goods is specifically to calculate the difference between the two contact force vectors by the formula ΔF=F1-F0. The offset direction can be determined by analyzing the change in the contact point position. If P1 moves relative to P0, then the offset direction of the goods is the direction of P1-P0. This direction vector is normalized to obtain the unit vector U of the offset direction, and U=(P1-P0)÷|P1-P0|, where |P1-P0| is the Euclidean distance between P1 and P0. The offset angle can be obtained by calculating the angle between the contact force change vector ΔF and the unit vector U of the offset direction, and the formula is θ=acos[(ΔF*U)÷(|ΔF|*|U|)], where acos is the arccosine function;

[0063] According to the calculated cargo offset angle, the rotation angle and extension distance of the robot are controlled so that the cargo can be placed in a predefined position, ensuring the clarity of the cargo edge in the subsequent three-dimensional point cloud data segmented by the point cloud segmentation algorithm, making the boundaries between the cargo clear, and reducing the difficulty of the point cloud segmentation algorithm, thereby improving the accuracy of subsequent cargo identification and positioning, and improving the accuracy and efficiency of automatic cargo storage.

[0064] An automatic vertical warehouse system based on three-dimensional point cloud technology includes: a warehouse management system, and an image acquisition and data preprocessing module, a warehouse storage model building module, a cargo identification and classification module, a position determination and strategy formulation module, an automatic handling planning and execution module, and an offset detection and position adjustment module installed inside the warehouse management system.

[0065] The image acquisition and data preprocessing module is responsible for taking comprehensive photos of the goods and the warehouse, acquiring three-dimensional point cloud data of the goods and the warehouse, and preprocessing the acquired three-dimensional point cloud data;

[0066] The warehouse storage model building module is used to extract the relevant features of the warehouse and the goods, use the machine learning algorithm to build the warehouse storage model according to the relevant features of the warehouse and the goods, and at the same time, define the grasping force F0 of the goods and the three-dimensional vector P0 of the contact point according to the relevant features of the warehouse and the goods;

[0067] The cargo identification and classification module uses a point cloud segmentation algorithm to segment the three-dimensional point cloud data of the warehouse and cargo according to the real-time status of the warehouse, and divides the warehouse into different cargo areas, extracts cargo-related features and key features of each cargo area, and compares them with the cargo model in the warehouse management system, thereby identifying different types of cargo;

[0068] The location determination and strategy formulation module is used to obtain the precise location of the goods in the warehouse and re-formulate the storage and handling strategy of the warehouse goods based on the location information and related characteristics of the goods and the real-time status of the warehouse;

[0069] The automated handling planning and execution module is used to plan the path and grasping action of the automated handling robot, and record the grasping force F1 of the manipulator and the three-dimensional vector P1 of the contact point;

[0070] The offset detection and position adjustment module is used to compare the two grasping forces F0 and F1 of the manipulator and the two three-dimensional vectors P0 and P1 of the contact point, calculate the offset information of the goods, and control the manipulator to adjust the position of the placed goods according to the offset information.

[0071] The above is based on the ideal embodiment of the present invention. Through the above description, relevant personnel can make various changes and modifications without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the content in the specification, and the technical scope must be determined according to the scope of the claims.

Claims

1. An automatic library method based on three-dimensional point cloud technology, characterized in that: The following steps are involved: S1. Use several cameras to ensure that images of the interior of the warehouse are captured at multiple different angles and positions, accurately capture the internal layout of the warehouse and the three-dimensional shape and position of the goods inside the warehouse, extract pixel depth information in the image, obtain three-dimensional point cloud data, and pre-process the obtained three-dimensional point cloud data and upload it to the warehouse management system; S2. Based on the three-dimensional point cloud data of goods and warehouses in the warehouse management system, extract the relevant features of the warehouse and goods, use the machine learning algorithm to build a warehouse storage model, and define the grasping force F0 of the goods according to the relevant features of the warehouse and goods, and the three-dimensional vector of the contact point is recorded as P0; S3. Then, according to the real-time status of the warehouse, the point cloud segmentation algorithm is used to segment the 3D point cloud data of the warehouse and the goods, and the warehouse is divided into different goods areas. The relevant features of the goods and the key features of the goods area are extracted for each goods area, and compared with the goods model in the warehouse management system to identify different types of goods; S4. Based on the identified cargo types and key features of the cargo area, the warehouse storage model is aligned with the point cloud data using 3D registration technology to obtain the precise location of the cargo in the warehouse. Based on the cargo location information and related features, as well as the real-time status of the warehouse, the storage and handling strategy of the warehouse cargo is re-formulated to ensure that the cargo is transferred to the designated cargo area. S5. According to the storage and handling strategy of the warehouse goods, the relevant characteristics of the goods, the relevant characteristics of the warehouse and the key characteristics of the goods area, the path and grasping action of the automated handling robot are planned, and the grasping force F1 of the manipulator is recorded by the pressure sensor on the manipulator, and the three-dimensional vector of the contact point is recorded as P1; S6, by comparing the two grasping forces F0 and F1 of the manipulator, and comparing the two three-dimensional vector records P0 and P1 of the contact point, it is determined whether the goods are offset, the offset information of the goods is calculated, and based on the offset information, the position where the manipulator places the goods is controlled to ensure the accuracy of the placement of the goods; In S2, the method for constructing a warehouse storage model includes the following steps: S21. Based on extracting relevant features of the warehouse and goods from the three-dimensional point cloud data of the goods and warehouse, a warehouse storage model is constructed through a machine learning model; S22. Use the three-dimensional point cloud data of the goods as a training set to optimize the warehouse storage model parameters, and compare it with the goods model in the warehouse storage model to distinguish different types of goods; S23. Create a model library for each type of goods in the warehouse management system, store the corresponding goods model in the library, assign a unique label to it, and integrate the constructed goods model into the model library of the warehouse management system; S24, based on the relevant characteristics of the goods, determine through experimental simulation that the grasping force is F0, and the three-dimensional vector of the contact point is recorded as P0, where P0 is the center of gravity of the goods; In the S4, the precise position of the goods in the warehouse is obtained, specifically, the features of the warehouse storage model are extracted, and the features in the three-dimensional point cloud data of the goods and the warehouse are extracted, and the two sets of features are compared; then, the rotation matrix and translation vector transformation relationship between the three-dimensional point cloud data of the goods and the warehouse, and the three-dimensional point cloud data of the warehouse storage model are calculated according to the matched features, and the transformation relationship is refined by an iterative optimization algorithm to minimize the alignment error between the two three-dimensional point cloud data. The optimized transformation relationship is used to map the cargo position information in the warehouse storage model to the real-time three-dimensional point cloud data to obtain the precise position of the goods in the warehouse; In S6, the offset information specifically includes an offset angle. The method for calculating the offset information of the goods is specifically to calculate the difference between the two contact force vectors by the formula ΔF=F1-F0. The offset direction can be determined by analyzing the change in the contact point position. If P1 moves relative to P0, then the offset direction of the goods is the direction of P1-P0. This direction vector is normalized to obtain the unit vector U of the offset direction, and U=(P1-P0)÷|P1-P0|, where |P1-P0| is the Euclidean distance between P1 and P0. The offset angle can be obtained by calculating the angle between the contact force change vector ΔF and the unit vector U of the offset direction, and the formula is θ=acos[(ΔF*U)÷(|ΔF|*|U|)], where acos is the inverse cosine function.

2. The automatic vertical storage method based on three-dimensional point cloud technology according to claim 1 is characterized by: In the step S1, three-dimensional point cloud data is obtained, specifically, the depth information of each pixel is determined according to the parallax between pixels in the image, and the depth information of each pixel is combined to generate a three-dimensional point cloud data; Preprocessing includes filtering and denoising, simplification and compression, coordinate transformation and alignment.

3. The automatic vertical storage method based on three-dimensional point cloud technology according to claim 1 is characterized in that: The relevant characteristics of the goods include: the size, weight, shape, surface characteristics of the goods, and the height, width and length of the goods stacking; the relevant characteristics of the warehouse include: the size, shape and arrangement of the shelves in the warehouse; the key characteristics of the cargo area include: the size, shape and arrangement of the shelves in the warehouse, and the relative position of the goods in the shelves.

4. The automatic vertical storage method based on three-dimensional point cloud technology according to claim 1 is characterized by: In the S3, the real-time status of the warehouse includes: the real-time quantity, type and location of the goods, the warehousing, outbound, movement and storage of the goods, and the idle status of the warehouse cargo area.

5. The automatic vertical storage method based on three-dimensional point cloud technology according to claim 1 is characterized by: In S3, the point cloud segmentation algorithm is specifically an edge-based point cloud segmentation algorithm.

6. The automatic vertical storage method based on three-dimensional point cloud technology according to claim 1 is characterized by: In S4, the storage and handling strategy of the warehouse goods is to reclassify and prioritize the goods according to the importance, urgency and operational requirements of the goods.

7. An automatic vertical storage system based on three-dimensional point cloud technology, using an automatic vertical storage method based on three-dimensional point cloud technology according to any one of claims 1 to 6, comprising: A warehouse management system, and an image acquisition and data preprocessing module, a warehouse storage model building module, a cargo identification and classification module, a location determination and strategy formulation module, an automated handling planning and execution module, and an offset detection and position adjustment module installed in the warehouse management system, characterized in that: The image acquisition and data preprocessing module is responsible for taking comprehensive photos of the goods and the warehouse, acquiring three-dimensional point cloud data of the goods and the warehouse, and preprocessing the acquired three-dimensional point cloud data; The warehouse storage model building module is used to extract the relevant features of the warehouse and the goods, use the machine learning algorithm to build the warehouse storage model according to the relevant features of the warehouse and the goods, and at the same time, define the grasping force F0 of the goods and the three-dimensional vector P0 of the contact point according to the relevant features of the warehouse and the goods; The cargo identification and classification module uses a point cloud segmentation algorithm to segment the three-dimensional point cloud data of the warehouse and cargo according to the real-time status of the warehouse, and divides the warehouse into different cargo areas, extracts cargo-related features and key features of each cargo area, and compares them with the cargo model in the warehouse management system, thereby identifying different types of cargo; The location determination and strategy formulation module is used to obtain the precise location of the goods in the warehouse and re-formulate the storage and handling strategy of the warehouse goods based on the location information and related characteristics of the goods and the real-time status of the warehouse; The automated handling planning and execution module is used to plan the path and grasping action of the automated handling robot, and record the grasping force F1 of the manipulator and the three-dimensional vector P1 of the contact point; The offset detection and position adjustment module is used to compare the actual gripping force F1 of the manipulator on the goods with the predefined gripping force F0, as well as the three-dimensional vectors P0 and P1 of the contact points with the goods, calculate the offset information of the goods, and control the manipulator to adjust the position of the placed goods according to the offset information.

Images