Luggage pose adjustment method and device, storage medium and electronic equipment

Abstract

The invention provides a luggage pose adjustment method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring depth image data and RBG image data in a preset recognition area; the depth image data and / or the RBG image data are / is used as input data, and classification information of a to-be-adjusted target placed in the target area is determined in combination with the trained classification neural network model; based on the depth image data, performing three-dimensional reconstruction on the to-be-adjusted target to obtain outer contour point cloud data of the to-be-adjusted target, the outer contour point cloud data being used for representing three-dimensional coordinate information of the to-be-adjusted target in a preset space coordinate; and on the basis of the classification information and the outer contour point cloud data of the to-be-adjusted target, determining a posture adjustment mode of the to-be-adjusted target so as to control the mechanical arm to perform posture adjustment on the to-be-adjusted target. The self-adaptive determination of the pose adjustment strategy of the target saves manpower, improves the security check efficiency of the luggage and the loading efficiency of the luggage, and is convenient for packaging and consignment.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a luggage posture adjustment method, device, storage medium and electronic device. Background Art

[0002] With the frequent flow of people, the security inspection of luggage related to security is becoming more and more important. At present, when identifying passengers' luggage, the security inspection in the industry, such as airports, still needs to rely on manual operations to load passengers' air luggage onto the conveyor belt, and manually adjust the luggage placement posture according to personal experience, so that the luggage maintains a predetermined regular posture, in order to ensure that the luggage identification and packing process has good efficiency. However, this traditional luggage posture adjustment method relies too much on manual experience, has low loading efficiency and high cost, and because the shapes of different luggage are also quite different, some luggage is not suitable for stacking, and it is easy to cause the luggage to change its original regular placement posture during the conveyor belt transmission process, further affecting the efficiency of the security inspection of the luggage and the packaging and consignment. For this reason, how to accurately identify the luggage posture so that it can be properly adjusted has become a problem that needs to be solved urgently in the industry. Summary of the invention

[0003] In view of this, embodiments of the present application provide a luggage posture adjustment method, device, storage medium and electronic device to at least or partially solve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a luggage posture adjustment method, comprising:

[0005] Acquire depth image data and RBG image data within a preset recognition area;

[0006] Using the depth image data and / or RBG image data as input data, combined with the trained [HZTC-HXJKJD241017][HS2411267CCN] classification neural network model, determining classification information of the target to be adjusted placed in the target area;

[0007] Based on the depth image data, three-dimensionally reconstruct the object to be adjusted to obtain outer contour point cloud data of the object to be adjusted, wherein the outer contour point cloud data is used to represent three-dimensional coordinate information of the object to be adjusted in a preset spatial coordinate;

[0008] Based on the classification information of the target to be adjusted and the outer contour point cloud data, a method for adjusting the position and posture of the target to be adjusted is determined to control the robot arm to adjust the position and posture of the target to be adjusted.

[0009] Optionally, in an embodiment of the present application, the depth image data and the RBG image data are used as input data, and combined with a trained classification neural network model to determine the classification information of the target to be adjusted placed in the target area, including:

[0010] Using the trained YOLO classification model, the input image is divided into a grid image including a plurality of grid areas, and the object whose center point falls in the grid area is subjected to target recognition and category prediction, so as to obtain the target information recognized in each grid and the corresponding category prediction information;

[0011] Based on the identified target information and category prediction information, overlapping grid borders are removed by a non-maximum suppression method to determine the target information to be adjusted and the corresponding category information.

[0012] Optionally, in an embodiment of the present application, the three-dimensional reconstruction of the target to be adjusted based on the depth image data to obtain the outer contour point cloud data of the target to be adjusted includes:

[0013] Performing data correction on the depth image by determining the internal and external parameters of the 3D camera;

[0014] Filter and remove interference data on the corrected 3D point cloud data to obtain a pre-processed depth image with other interferences removed;

[0015] A random sampling consistency algorithm is used to perform straight line fitting on the preprocessed depth image to determine the size information of the object to be adjusted;

[0016] In combination with the size information, the object to be adjusted is three-dimensionally reconstructed to obtain the outer contour point cloud data of the object to be adjusted, which is used to reflect the size of the object to be adjusted and its three-dimensional space coordinate information in a preset space;

[0017] Optionally, in an embodiment of the present application, the method of adjusting the posture of the target to be adjusted is determined based on the classification information of the target to be adjusted and the outer contour point cloud data.

[0018] [HZTC-HXJKJD241017][HS2411267CCN] includes:

[0019] Generating text input data conforming to the large language model processing format, wherein the text input data is text data that uses text to describe the classification information and the outer contour point cloud data of the target to be adjusted in a natural language;

[0020] A text answer is given to the text input data using a trained large language model, wherein the text answer is used to provide a corresponding description of the manner in which the posture of the target to be adjusted is adjusted through natural language or computer language.

[0021] Optionally, in an embodiment of the present application, before using the trained large language model to provide a text answer to the text input data, the method further includes:

[0022] Using a short description instruction, feature alignment is performed on point features of the sample point cloud data and response text tags of the basic large language model to convert the sample point cloud data into input data containing embedded adjustment tags;

[0023] The feature mapper and the multilayer perceptron of the basic large language model are trained by using the input data after feature alignment, and the parameters of the feature mapper and the multilayer perceptron of the basic large language model are adjusted;

[0024] Freeze the parameters of the point cloud encoder, use complex instructions, and jointly with the feature mapper after parameter adjustment, train the semantic understanding ability of the prompt word of the large language model after parameter adjustment of the multilayer perceptron to obtain the trained large language model.

[0025] Optionally, in an embodiment of the present application, using the trained large language model to provide a text answer to the text input data includes:

[0026] Use the point cloud encoder to extract features from the point cloud data to obtain point cloud feature sequence data;

[0027] Using a feature mapper, the point cloud feature sequence data is mapped into a latent space of a backbone network of the large language model, so as to map the point cloud features into point cloud label data, so that the data dimension of the point cloud label data is the same as the data dimension of the prompt word semantic text of the large language model;

[0028] Constructing mixed label sequence data corresponding to the point cloud label;

[0029] Using a self-attention mechanism, generating predicted response tag sequence data corresponding to the mixed tag sequence data;

[0030] [HZTC-HXJKJD241017][HS2411267CCN]

[0031] According to a preset semantic generation algorithm, combined with the predicted response tag sequence data, and using a parser to comprehensively analyze the image classification results, the luggage posture control instructions and text answer data are generated.

[0032] Optionally, in an embodiment of the present application, before using the trained classification neural network model to identify and classify the RBG image data to determine the classification information of the target to be adjusted contained in the target area, the method further includes:

[0033] The acquired depth image data and RBG image data are preprocessed to obtain depth image data or RBG image data that meets the input requirements of the trained classification neural network model.

[0034] Optionally, in an embodiment of the present application, obtaining depth image data and RBG image data within a preset recognition area includes:

[0035] Using an object detection sensor disposed in a preset area, detecting whether there is luggage in the preset identification area;

[0036] When it is determined that there is luggage in the preset identification area, the 3D camera and RBG camera installed above the preset area to be identified are used to shoot the area to be identified to obtain depth image data and RBG image data in the preset identification area.

[0037] In a second aspect, the present application also provides a luggage posture adjustment device, including:

[0038] An image acquisition module, used to acquire depth image data and RBG image data within a preset recognition area;

[0039] A classification module, used to use the depth image data and / or RBG image data as input data, and determine the classification information of the target to be adjusted placed in the target area in combination with a trained classification neural network model;

[0040] A detection module, configured to perform three-dimensional reconstruction of the target to be adjusted based on the depth image data to obtain outer contour point cloud data of the target to be adjusted, wherein the outer contour point cloud data is used to represent three-dimensional coordinate information of the target to be adjusted in a preset spatial coordinate;

[0041] The analysis module is used to determine a method for adjusting the posture of the target to be adjusted based on the classification information of the target to be adjusted and the outer contour point cloud data, so as to control the robot arm to adjust the posture of the target to be adjusted.

[0042] In a third aspect, an embodiment of the present application further provides a computer storage medium having computer executable instructions stored thereon, and when the computer executable instructions are executed, the computer executable instructions are executed as described in the present invention.

[0043] [HZTC-HXJKJD241017][HS2411267CCN] Any one of the luggage posture adjustment methods described in the first aspect of the application embodiment.

[0044] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0045] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute any one of the luggage posture adjustment methods described in the first aspect of the embodiments of the present application.

[0046] The present application provides a luggage posture adjustment method, device, storage medium and electronic device, which obtains depth image data and RBG image data in a preset recognition area; uses the depth image data and / or RBG image data as input data, and combines the trained classification neural network model to determine the classification information of the target to be adjusted placed in the target area; based on the depth image data, the target to be adjusted is three-dimensionally reconstructed to obtain the outer contour point cloud data of the target to be adjusted, and the outer contour point cloud data is used to represent the three-dimensional coordinate information of the target to be adjusted in the preset spatial coordinates; based on the classification information of the target to be adjusted and the outer contour point cloud data, the method of adjusting the posture of the target to be adjusted is determined to control the mechanical arm to adjust the posture of the target to be adjusted. The posture adjustment method disclosed in the present application obtains the depth image and RBG image of the target, analyzes the posture and type of the target, and adaptively determines the posture adjustment strategy for the target, which greatly saves manpower, improves the security inspection efficiency of luggage, has high loading efficiency, and is convenient for packaging and consignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0048] Figure 1 A schematic diagram of a workflow of a luggage posture adjustment method provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of a working scenario for adjusting the posture of a target to be adjusted provided in an embodiment of the present application;

[0050] Figure 3A schematic diagram of a workflow of a method for obtaining luggage posture adjustment provided in an embodiment of the present application

[0051] [HZTC-HXJKJD241017][HS2411267CCN]Figure;

[0052] Figure 4 A schematic structural diagram of a luggage posture adjustment device provided in an embodiment of the present application.

[0053] Figure 5 A schematic structural diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0055] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0056] Embodiment 1

[0057] The present application embodiment provides a luggage posture adjustment method, such as Figure 1 As shown, Figure 1 A schematic diagram of a workflow of a luggage posture adjustment method provided in an embodiment of the present application includes:

[0058] Step S101, obtaining depth image data and RBG image data within a preset recognition area. The embodiment of the present application defines that these two images of the target are used as the basis for determining the adjustment of the target posture, thereby ensuring the accuracy of the generated posture adjustment method while reducing the amount of data to be processed, thereby effectively ensuring that the efficiency of the method for adjusting the target posture can be obtained more quickly.

[0059] Specifically, in an optional implementation of the embodiment of the present application, the depth image data and RBG image data in the preset recognition area are obtained, including: using an object detection sensor set in the preset area to detect whether there is a luggage item in the preset recognition area; when it is determined that there is a luggage item in the preset recognition area, using a 3D camera and an RBG camera set up above the preset area to be recognized to shoot the area to be recognized, so as to obtain the depth image data and RBG image data in the preset recognition area. At this stage, the embodiment of the present application monitors the preset area without the need for manual control or real-time shooting of the target area, thereby providing

[0060] [HZTC-HXJKJD241017][HS2411267CCN] The system can obtain the effective working rate of depth image data and RBG image data, saving system processing resources. Setting the 3D camera and RBG camera above the preset recognition area can effectively avoid the visual error caused by the different positions of the target luggage or the different shooting angles when shooting the preset recognition area, or the situation of being blocked by other objects and causing the omission of recognition, thereby improving the comprehensiveness of the target to be adjusted contained in the image of the area to be recognized.

[0061] Step S102, using the depth image data and / or RBG image data as input data, combined with the trained classification neural network model, to determine the classification information of the target to be adjusted placed in the target area. Specifically, in this stage of the embodiment of the present application, the target to be adjusted contained in the acquired depth image data and / or RBG image data is identified by a trained classification neural network model to determine whether the target to be identified is a suitcase or other object, whether it is a target luggage with a regular outer contour or other irregular objects, or to determine the classification information of whether the target to be identified is a soft or hard object. By determining the classification information including these contents, as the basis for generating a control robot arm to adjust the posture of the target to be adjusted, the accuracy and applicability of the generated posture adjustment method can be improved, and the possibility of damaging the target to be adjusted during the posture adjustment can be reduced.

[0062] Specifically, in an optional implementation of the embodiment of the present application, the depth image data and RBG image data are used as input data, combined with a trained classification neural network model, to determine the classification information of the target to be adjusted placed in the target area, including: using a trained YOLO classification model, dividing the input image into a grid image containing multiple grid areas, performing target recognition and category prediction on the objects whose center points are contained in the input image and fall in the grid area, and obtaining the target information recognized in each grid and the corresponding category prediction information; based on the recognized target information and category prediction information, removing the overlapping grid borders by non-maximum suppression method to determine the target information to be adjusted and the corresponding classification information, and comparing them according to a preset suspicious baggage photo library, if luggage with similar images is detected, the luggage is distinguished according to subsequent control instructions, such as being removed from the conveyor belt or transferred to the suspicious luggage conveyor belt. In this stage of the implementation of the present application, the classification information of the target to be adjusted is obtained by using a trained YOLO classification model, and the process is simple and easy to implement. Specifically, the model divides the input image into multiple grid areas. If the grid contains other object images that are not background images, it detects whether the center point of the imaged part of the object falls in the grid. If it falls in the grid, the image of the object corresponding to the grid area is obtained in advance, and the image is analyzed based on the image.

[0063] [HZTC-HXJKJD241017][HS2411267CCN] recognizes target information and predicts the corresponding category, and then removes the overlapping grid borders when fusing images of different grids containing the same object through non-maximum suppression, so as to fuse the imaging images of the same object in different grids, thereby more completely determining the imaging image of the object contained in the input image, determining the final target to be adjusted according to the acquired imaging image, and determining the classification information of the fused imaging object, i.e. the target to be adjusted, based on the predicted category of each grid. This process has high recognition accuracy and the determined classification information is more reliable.

[0064] Furthermore, after determining the target information to be adjusted and the corresponding classification information, the method further includes: based on the target information to be adjusted and the corresponding classification information, comparing the input image of the target to be adjusted with the image in the preset suspicious baggage image database to obtain the similarity between the images; if the similarity is higher than or equal to the preset similarity threshold, the target to be adjusted is determined to be the target baggage for which a control instruction needs to be generated in the future to adjust its posture; if the similarity is lower than the preset threshold, the target to be adjusted is removed from the targets that need to be adjusted in posture, or transferred to the suspicious baggage conveyor belt for manual processing, so as to further improve the accuracy and applicability of the method described in the embodiment of the present application.

[0065] Step S103: Based on the depth image data, the object to be adjusted is reconstructed in three dimensions to obtain the outer contour point cloud data of the object to be adjusted, and the outer contour point cloud data is used to represent the three-dimensional coordinate information of the object to be adjusted in the preset spatial coordinates. The embodiment of the present application determines the outer contour point cloud data of the object to be adjusted by the three-dimensional reconstruction method, and the process is accurate, reliable and easy to implement.

[0066] Specifically, in an optional implementation method of an embodiment of the present application, the target to be adjusted is three-dimensionally reconstructed based on the depth image data to obtain the outer contour point cloud data of the target to be adjusted, including: performing data correction on the depth image by determining the internal and external parameters of the 3D camera; filtering and removing interference data on the corrected 3D point cloud data to obtain a preprocessed depth image with other interferences removed; using a random sampling consistency algorithm to perform straight line fitting on the preprocessed depth image to determine the size information of the object to be adjusted; combining the size information, the object to be adjusted is three-dimensionally reconstructed to obtain the outer contour point cloud data of the object to be adjusted, which is used to reflect the size of the object to be adjusted and its three-dimensional space coordinate information in a preset space. At this stage of the embodiment of the present application, data correction can be used to eliminate data errors caused by lens distortion when the 3D camera takes pictures of the target. Specifically, this process can be performed by standardizing the parameters of the set up 3D camera.

[0067] [HZTC-HXJKJD241017][HS2411267CCN] After calibration, the internal and external parameters of the camera are obtained, wherein the internal parameters mainly include: focal length, pixel size and distortion coefficient, etc.; the external parameters mainly include parameters such as the position and rotation direction of the 3D camera in a preset coordinate system such as the world coordinate system. By calibrating the values ​​of these parameters of the 3D camera, the calibration process solves the mapping relationship through the preset coordinate system coordinates and pixel coordinates of the determined calibration control points. According to the mapping relationship, the world coordinates are inferred from the pixel coordinates, thereby achieving the accuracy and convenience of three-dimensional reconstruction of the target to be adjusted. In the embodiment of the present application, the point cloud data of the target to be adjusted obtained here is also filtered and de-interference processed. Specifically, for example, the point cloud data of the target to be adjusted can be cut in the X and Y directions of the preset coordinate system by performing a straight-through filtering process on the obtained point cloud data, and the image of the part or other interference objects such as the baggage conveyor belt image at the bottom of the luggage in the actual application scenario of the embodiment of the present application can be removed. At the same time, there may be some outlier noise data at the edge of the point cloud data, which can also be removed by statistical filtering, thereby improving the accuracy of the point cloud data of the target to be adjusted. The bottom surface of the target to be adjusted is segmented by using a random sampling consistency algorithm (RANSAC algorithm), and the average height of the bottom surface of the target (i.e., the luggage conveyor belt in the application scenario of the present application) from the ground is calculated to eliminate this part of non-target image point cloud data. The upper surface point cloud data of the target to be adjusted is extracted by using the random sampling consistency algorithm again. According to the average height of the upper surface point cloud data from the upper surface, the plan view of the upper surface is fitted with the RANSAC algorithm four times. The intersection of the first and third fitted lines, and the intersection of the second and fourth lines are taken. These two intersection points are the diagonal points of the plane where the upper surface of the target to be adjusted is located. The length and width of the upper surface of the target to be adjusted can be calculated by subtracting the x-coordinate value and the y-coordinate value of the diagonal point in the preset coordinate system and taking the absolute value. The height of the surface to be adjusted is obtained by subtracting the determined upper surface height and bottom surface height and taking the absolute value, and the surface concave-convexity of the target to be adjusted is determined. In this way, not only can the position of the target to be adjusted be located, but also the outer contour size and outer surface convexity of the target to be adjusted can be determined more directly and quickly, so that based on these parameters as the basis for adjustment, the robot arm can perform high-precision posture adjustment on the target to be adjusted.

[0068] Step S104: Based on the classification information of the target to be adjusted and the outer contour point cloud data, determine the method for adjusting the position and posture of the target to be adjusted, so as to control the robot arm to adjust the position and posture of the target to be adjusted. That is, according to the outer contour point cloud data of the target to be adjusted, determine its position information and outer contour shape information in a preset coordinate system, and combine the classification type of the target to be adjusted.

[0069] [HZTC-HXJKJD241017][HS2411267CCN]soft luggage or hard luggage and other information, so as to accurately control how the robot arm should adjust the posture of the target to be adjusted, the force position of the robot arm and other information, and quickly and accurately adjust the posture of the target.

[0070] Optionally, in an embodiment of the present application, the method of adjusting the posture of the target to be adjusted based on the classification information of the target to be adjusted and the outer contour point cloud data includes: generating text input data that conforms to the processing format of the large language model, the text input data is text data that uses text to describe the classification information and outer contour point cloud data of the target to be adjusted in natural language; using the trained large language model to give a text answer to the text input data, the text answer is used to describe the method of adjusting the posture of the target to be adjusted in natural language or computer language. At this stage of the embodiment of the present application, the method of adjusting the posture of the target to be adjusted is determined by the trained large language model in the manner of information processing in this text. The whole process is very intuitive and convenient for operation and maintenance personnel to check the corresponding process and verify the system. Improve the controllability and identifiability of the system operation. The later operation and maintenance process has lower professional requirements for operation and maintenance personnel, and the operation and maintenance process is relatively easy to implement.

[0071] The present application hereby exemplifies a method of using a trained large language model to provide a text answer to text input data, wherein the text answer is used to provide a corresponding description of the way of adjusting the posture of the target to be adjusted by natural language or computer language, and exemplifies the description. Figure 2 As shown, Figure 2 A schematic diagram of a working scenario for adjusting the posture of a target to be adjusted provided in an embodiment of the present application:

[0072] Set up the system to prompt the large language model: You are a large model that can understand and analyze 3D point clouds. Please analyze and calculate according to the input point cloud data and the instruction information, and output the instructions for controlling the robotic arm to straighten the luggage according to the instruction format.

[0073] Obtain the classification label and outer contour point cloud data of the target A to be adjusted in the running direction of the conveyor belt, and identify whether the target A is a baggage target. If it is determined to be a baggage target, then identify the angle between the extension line of the long side of the baggage and the running direction of the conveyor belt. Perform feature extraction on the point cloud data composed of the outer contour point cloud data, classification label and angle data of the baggage A, and obtain the point cloud identification data after feature extraction;

[0074] Data input for the trained large language model:<p_start> {Point cloud representation data after feature extraction}<p_end> {Instructions: Analyze the point cloud identification data, extract and determine the type of luggage (such as hard luggage, soft luggage), determine whether there are large depressions or protrusions on the luggage surface, and determine the size of the luggage (length l,

[0075] [HZTC-HXJKJD241017][HS2411267CCN] width w, height h information), determine the coordinate positions of the 8 vertices of the luggage (x1, y1, z1) ~ (x8, y8, z8), determine the angle between the long side of the luggage and the direction of movement of the conveyor belt, here the clockwise angle of the top view plane is preset as a positive angle, and output the instruction to straighten the luggage. For example: the length of luggage A is l, the width is w, and the height is h. The position on the conveyor belt is (x1, y1, z1) ~ (x8, y8, z8), and it is swung 30 degrees counterclockwise.

[0076] The large language model answers by outputting the type, size, posture, inclination angle (angle value) on the conveyor belt, and coordinate position on the conveyor belt of the luggage that needs to be adjusted, and determines the instructions for controlling the robotic arm to straighten the luggage based on the above parameters.

[0077] In this stage of the embodiment of the present application, the posture adjustment method described above is used to intuitively and quickly demonstrate the process of posture adjustment of the target to be adjusted (baggage) by the set robotic arm through the trained large language model. If luggage with similar images is detected, the luggage is distinguished according to subsequent control instructions, such as being removed from the conveyor belt or transferred to the suspicious luggage conveyor belt. The process is efficient, accurate, safe and controllable.

[0078] Optionally, in an embodiment of the present application, before using the trained large language model to provide a text answer to the text input data, the method further includes: using a brief description instruction to align the point features of the sample point cloud data with the response text mark of the basic large language model to convert the sample point cloud data into input data containing embedded adjustment marks; freezing the parameters of the large language model and the point cloud encoder through the feature-aligned input data to train the point cloud feature mapper and. Among them, the brief description instruction for the point cloud is a short natural language instruction that the large language model can understand, such as: please describe the size of the object according to the point cloud, or please describe whether the object is a soft luggage such as a handbag, a woven bag or a hard suitcase. In order to expand the number of point cloud samples, the embodiment of the present application can use a certain number of brief descriptions here. Use a large language model to use a brief description instruction on the point cloud data set to generate a data set corresponding to the point cloud and the description result and use it in the first stage of training. The first stage is mainly the feature alignment stage, and the second stage is the instruction tuning stage. In this stage, the point cloud encoder is frozen, the feature mapper and the large language model parameters are trained, and the large language model is helped to better output the luggage posture control instruction. Freeze the parameters of the point cloud encoder, use complex instructions, and combine the feature mapper with adjusted parameters to train the semantic understanding ability of the prompt word of the large language model with adjusted parameters of the multilayer perceptron to obtain a trained large language model. The complex instructions include multiple short description instructions, such as: please describe the size, location, classification, and surface description of the luggage according to the point cloud data, and output control instructions. When preparing training data, the embodiment of the present application uses

[0079] [HZTC-HXJKJD241017][HS2411267CCN] Use a multimodal model to generate descriptions and control instructions for point clouds based on complexity, and expand the training samples. In the second stage, the expanded training samples are used to train the model, and the generated posture adjustment instructions are verified. The reinforcement learning method is used to adjust the instruction generation strategy by optimizing the reward function, thereby improving the model for generating posture adjustment instructions, so that the generated posture adjustment instructions are more accurate. Since the adjustment method is generated by a large language model at this stage of the embodiment of the present application, the model training is performed by minimizing the negative log-likelihood of the text labeling of the posture information at this stage of this implementation. The end-to-end training method defined above is used to enable the model to effectively integrate the point cloud data and the text format mode required by the large language model to perform corresponding semantic analysis and processing. Specifically, the training process of the model includes two stages. The first stage is the feature alignment stage, in which the point cloud encoder and large language model parameters are frozen, and the multi-layer perceptron and feature mapper are trained to enable the large language model obtained after training to effectively align the input point cloud data with the data dimensions of the text tag space of the text prompt information that the model can process. The second stage is the instruction adjustment stage, in which the parameters of the point cloud encoder still need to be frozen, and the large language model is trained to establish the ability to understand and respond to complex instructions (including point cloud data), so as to ensure that the trained large language model can accurately analyze and process the input point cloud data, and obtain a targeted and accurate posture adjustment method for semantic feedback.

[0080] In an optional implementation of the embodiment of the present application, as Figure 3 As shown, Figure 3A workflow diagram of a method for obtaining luggage posture adjustment provided in an embodiment of the present application, wherein the specific process of generating an output result including luggage posture description and control instruction information may include: using a trained large language model to provide a text answer to the input instruction text and point cloud input data, wherein the instruction text is a text describing the luggage posture, position, size, classification, and shape, and using a word segmenter to extract features from the input instruction text data to generate text features, i.e., tag information. A point cloud encoder is used to extract features from the point cloud data of the data to obtain point cloud feature sequence data; a feature mapper is used to map the point cloud feature sequence data into the latent space of the backbone network of the large language model to map the point cloud features into point cloud tag data, so that the data dimension of the point cloud tag data is the same as the data dimension of the prompt word semantic text of the large language model; the purpose of generating point cloud tags and text tags is to unify the data format to facilitate the large language model to recognize and analyze; construct mixed tag sequence data corresponding to the point cloud tags; a self-attention mechanism is used to generate predicted response tag sequence data corresponding to the mixed tag sequence data; the response tag sequence data and the classification information obtained from the image data through the network model are input into the parser together, and the parser parses the response tag sequence into text information according to the vocabulary, and comprehensively processes to generate output results, including luggage posture description and control instruction information.

[0081] According to the preset semantic generation algorithm, combined with the predicted response tag sequence data, and using the parser to comprehensively analyze the image classification results, the luggage posture control instruction and text answer data are generated. The trained large language model used in this stage of the embodiment of the present application is mainly composed of five parts: a word segmenter, a pre-trained point cloud encoder, a feature mapper, and a basic large language model. The word segmenter is used to extract features from the input instruction data to generate text features, i.e., tag information. The point cloud encoder converts the point cloud data P∈R n×d As input, n is the number of points contained in the point cloud data, and d is the feature dimension of each point. The output of the point cloud encoder is the point feature sequence data X = (x1, x2, ..., x m )∈R m×c , where m is the number of point features and c is the feature dimension. The feature mapper is a multi-layer perceptron neural network that maps point features X to point cloud labels Y = (y1, y2, ..., y m )∈R m×c′ , where c′ is the dimension of the point tag, which is the same as the data dimension of the target text tag to be processed by the basic large language model. The backbone of the basic large language model is a self-attention module of a pure decoder, which accepts as input a mixed tag sequence consisting of text and point cloud feature sequences. This mixed tag sequence is represented by Z = (z1, z2, ..., z k )∈Rk×c′ , where k is the total number of tags. Using the self-attention mechanism, the backbone network can understand the contextual relationship between different types of tags, so as to generate corresponding responses according to the text and point cloud input, that is, the corresponding text information feedback, which is used to digitize the adjustment method of the luggage posture. In the actual application scenario of the embodiment of the present application, the input point cloud data and the user's instruction data are analyzed, and the answer is given in the specified format. The following step is the process of converting the predicted tag Z^ into the specified format answer text. The output answer text includes the category, location, posture, size information of the luggage, and the instruction information for alignment. The output of the large language model is a string of predicted tags Z^=(^z1,^z2,...,^z k )∈R k×c′ , where k is the total number of markers. The predicted value of the i-th marker ^z i , each ^zi will pass through the last linear layer and then perform a softmax operation to map the hidden state to the probability distribution of the vocabulary. This additional layer is called the vocabulary layer: R c′ →R V , where V is the size of the vocabulary. The final prediction result of the i-th token ^z i is the word with the highest probability in the vocabulary. The following are the parameters configured for the large language model in the embodiment of the present application, which can also be replaced with other parameters. Among them, each set of point cloud data uses n = 10240 points and d = 6 dimensions. The point cloud encoder outputs m = 513 point features, each feature has c = 384 dimensions. The feature mapper includes three sampling

[0082] [HZTC-HXJKJD241017][HS2411267CCN] The linear layer activated by GeLU can map point features to labels of c′=5120 dimensions. The response label sequence data and the classification information obtained from the input image data using the trained classification neural network network model described in the aforementioned step S102 are input into the parser. The parser parses the response label sequence into text information according to a preset vocabulary. The text information includes baggage posture description and control instruction information described in natural language that can be recognized by the operation and maintenance personnel, and synthesizes them to generate output results. This ensures the accuracy of the baggage posture description information and control instruction information finally obtained, so as to obtain comprehensive, intuitive and accurate response feedback.

[0083] Optionally, in one embodiment of the present application, before using the trained classification neural network model to identify and classify the RBG image data and determine the classification information of the target to be adjusted contained in the target area, the method also includes: preprocessing the acquired depth image data and RBG image data to obtain depth image data or RBG image data that meets the input requirements of the trained classification neural network model. In this embodiment of the present application, the preprocessing includes but is not limited to preprocessing methods such as image scaling and normalization, so that the acquired image data can meet the input requirements of the trained classification model. So that the method described in the embodiment of the present application can adapt to the processing of acquired image data of different resolution sizes or different formats.

[0084] The present application provides a luggage posture adjustment method, which obtains depth image data and RBG image data in a preset recognition area; uses the depth image data and / or RBG image data as input data, and combines the trained classification neural network model to determine the classification information of the target to be adjusted placed in the target area; based on the depth image data, the target to be adjusted is three-dimensionally reconstructed to obtain the outer contour point cloud data of the target to be adjusted, and the outer contour point cloud data is used to represent the three-dimensional coordinate information of the target to be adjusted in the preset spatial coordinates; based on the classification information of the target to be adjusted and the outer contour point cloud data, the method of adjusting the posture of the target to be adjusted is determined to control the robot arm to adjust the posture of the target to be adjusted. The posture adjustment method disclosed in the present application obtains the depth image and RBG image of the target, analyzes the posture and type of the target, and adaptively determines the posture adjustment strategy for the target, which greatly saves manpower, improves the security inspection efficiency of luggage, has high loading efficiency, and is convenient for packaging and consignment.

[0085] Embodiment 2

[0086] [HZTC-HXJKJD241017][HS2411267CCN]

[0087] Based on the luggage posture adjustment method provided in the first embodiment of the present application, the present embodiment also provides a luggage posture adjustment device, such as Figure 2 As shown, Figure 2 The present invention provides a schematic diagram of a luggage posture adjustment device 20 according to an embodiment of the present invention. The luggage posture adjustment device 20 includes:

[0088] An image acquisition module 201 is used to acquire depth image data and RBG image data within a preset recognition area;

[0089] A classification module 202, configured to use the depth image data and / or RBG image data as input data and determine classification information of the target to be adjusted placed in the target area in combination with a trained classification neural network model;

[0090] A detection module 203 is used to perform three-dimensional reconstruction on the target to be adjusted based on the depth image data to obtain outer contour point cloud data of the target to be adjusted, wherein the outer contour point cloud data is used to represent three-dimensional coordinate information of the target to be adjusted in a preset spatial coordinate;

[0091] The analysis module 204 is used to determine a method for adjusting the posture of the target to be adjusted based on the classification information of the target to be adjusted and the outer contour point cloud data, so as to control the robot arm to adjust the posture of the target to be adjusted.

[0092] Optionally, in one embodiment of the present application, the classification module 202 is also used to: use a trained YOLO classification model to divide the input image into a grid image comprising multiple grid areas, perform target recognition and category prediction on objects contained in the input image whose center points fall in the grid areas, and obtain the recognized target information and corresponding category prediction information in each grid; based on the recognized target information and category prediction information, remove overlapping grid borders through a non-maximum suppression method to determine the target information to be adjusted and the corresponding classification information.

[0093] Optionally, in one embodiment of the present application, the detection module 203 is further used to: perform data correction on the depth image by determining the internal and external parameters of the 3D camera; perform filtering and interference data removal processing on the corrected 3D point cloud data to obtain a preprocessed depth image with other interferences removed; use a random sampling consistency algorithm to perform straight line fitting on the preprocessed depth image to determine the size information of the object to be adjusted; combine the size information to perform three-dimensional reconstruction on the object to be adjusted, and obtain the outer contour point cloud data of the object to be adjusted, which is used to reflect the size of the object to be adjusted and its three-dimensional space coordinate information in the preset space;

[0094] Optionally, in one embodiment of the present application, the analysis module 204 is further used to: generate

[0095] [HZTC-HXJKJD241017][HS2411267CCN] language model processing format text input data, the text input data is text data that uses text to describe the classification information and outer contour point cloud data of the target to be adjusted in natural language; use the trained large language model to give a text answer to the text input data, and the text answer is used to give a corresponding description of the way to adjust the posture of the target to be adjusted through natural language or computer language.

[0096] Optionally, in one embodiment of the present application, the device 20 further includes a training module (not shown in the drawings), which is used to: use a short description instruction to perform feature alignment on the point features of the sample point cloud data and the response text tags of the basic large language model to convert the sample point cloud data into input data containing embedded adjustment tags; train the feature mapper and the multilayer perceptron of the basic large language model through the feature-aligned input data, and adjust the parameters of the feature mapper and the multilayer perceptron of the basic large language model; freeze the parameters of the point cloud encoder, use complex instructions, and jointly with the feature mapper after parameter adjustment, train the prompt word semantic understanding ability of the large language model after the parameter adjustment of the multilayer perceptron, so as to obtain a trained large language model.

[0097] Optionally, in an implementation of the embodiment of the present application, the analysis module 204 is also used to: use the point cloud encoder to perform feature extraction on the point cloud data of the data to obtain point cloud feature sequence data; use a feature mapper to map the point cloud feature sequence data into the latent space of the backbone network of the large language model to map the point cloud features into point cloud tag data, so that the data dimension of the point cloud tag data is the same as the data dimension of the prompt word semantic text of the large language model; construct mixed tag sequence data corresponding to the point cloud tags; use a self-attention mechanism to generate predicted response tag sequence data corresponding to the mixed tag sequence data; and generate the text answer data in combination with the predicted response tag sequence data according to a preset semantic generation algorithm.

[0098] Optionally, in one embodiment of the present application, the device 20 also includes a preprocessing module (not shown in the drawings), which is used to: preprocess the acquired depth image data and RBG image data to obtain depth image data or RBG image data that meets the input requirements of a trained classification neural network model.

[0099] Optionally, in one embodiment of the present application, the image acquisition module 201 is further used to: use an object detection sensor set in a preset area to detect whether there is a luggage item in the preset identification area; when it is determined that there is a luggage item in the preset identification area, use a 3D camera and an RBG camera set up above the preset [HZTC-HXJKJD241017][HS2411267CCN] area to be identified to shoot the area to be identified, so as to obtain depth image data and RBG image data in the preset identification area.

[0100] The present application provides a luggage posture adjustment device, which sets an image acquisition module to acquire depth image data and RBG image data in a preset recognition area; sets a classification module to use the depth image data and / or RBG image data as input data, and determines the classification information of the target to be adjusted placed in the target area in combination with a trained classification neural network model; sets a detection module to perform three-dimensional reconstruction on the target to be adjusted based on the depth image data to obtain the outer contour point cloud data of the target to be adjusted, and the outer contour point cloud data is used to represent the three-dimensional coordinate information of the target to be adjusted in the preset spatial coordinates; sets an analysis module to determine the way to adjust the posture of the target to be adjusted based on the classification information of the target to be adjusted and the outer contour point cloud data, so as to control the mechanical arm to adjust the posture of the target to be adjusted. The posture adjustment device disclosed in the embodiment of the present application acquires the depth image and RBG image of the target, analyzes the posture and type of the target, and adaptively determines the posture adjustment strategy for the target, which greatly saves manpower, improves the security inspection efficiency of luggage, has high loading efficiency, and is convenient for packaging and consignment.

[0101] Embodiment 3

[0102] The embodiment of the present application further provides a storage medium on which a computer program is stored. When the program is executed by a processor, any one of the luggage posture adjustment methods described in the first embodiment of the present application is implemented.

[0103] Embodiment 4:

[0104] The present application also provides an electronic device, such as Figure 3 As shown, Figure 3 A schematic diagram of the structure of an electronic device 30 is provided for an embodiment of the present application. The electronic device 30 includes:

[0105] One or more processors 301, communication interface 302, memory 303 and communication bus 304, the processor 301, memory 303 and communication interface 302 communicate with each other via the communication bus 304;

[0106] Memory 303, used to store one or more programs;

[0107] When the one or more programs are executed by the one or more processors 301, the one or more processors 301 implement any one of the luggage posture adjustment methods described in Embodiment 1 of the present application.

[0108] So far, this application has described specific embodiments of the present subject matter. In some cases,

[0109] [HZTC-HXJKJD241017][HS2411267CCN]The actions recited in the claims may be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.

[0110] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system layer on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0111] The controller may be implemented in any suitable manner. For example, the controller may be implemented in the form of a microprocessor.

[0112] [HZTC-HXJKJD241017][HS2411267CCN] A memory controller is a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0113] The system layers, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0114] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0116] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, system layers or computer program products. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment or a combination of

[0117] [HZTC-HXJKJD241017][HS2411267CCN] The forms of embodiments of software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0118] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific transactions or implement specific abstract data types. The present application may also be practiced in distributed computing environments where transactions are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0119] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system layer embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0120] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A luggage posture adjustment method, characterized in that: include: Acquire depth image data and RBG image data within a preset recognition area; Using the depth image data and / or RBG image data as input data, combined with a trained classification neural network model, to determine classification information of the target to be adjusted placed in the target area; Based on the depth image data, three-dimensionally reconstruct the object to be adjusted to obtain outer contour point cloud data of the object to be adjusted, wherein the outer contour point cloud data is used to represent three-dimensional coordinate information of the object to be adjusted in a preset spatial coordinate; Based on the classification information of the target to be adjusted and the outer contour point cloud data, a method for adjusting the position and posture of the target to be adjusted is determined to control the robot arm to adjust the position and posture of the target to be adjusted.

2. The luggage posture adjustment method according to claim 1, characterized in that: The method of using the depth image data and the RBG image data as input data and combining the trained classification neural network model to determine the classification information of the target to be adjusted placed in the target area includes: Using the trained YOLO classification model, the input image is divided into a grid image including a plurality of grid areas, and the object whose center point falls in the grid area is subjected to target recognition and category prediction, so as to obtain the target information recognized in each grid and the corresponding category prediction information; Based on the identified target information and category prediction information, overlapping grid borders are removed by a non-maximum suppression method to determine the target information to be adjusted and the corresponding category information.

3. The luggage posture adjustment method according to claim 1, characterized in that: The three-dimensional reconstruction of the target to be adjusted based on the depth image data to obtain the outer contour point cloud data of the target to be adjusted includes: Performing data correction on the depth image by determining the internal and external parameters of the 3D camera; Filter and remove interference data on the corrected 3D point cloud data to obtain a pre-processed depth image with other interferences removed; A random sampling consistency algorithm is used to perform straight line fitting on the preprocessed depth image to determine the size information of the object to be adjusted; In combination with the size information, the object to be adjusted is three-dimensionally reconstructed to obtain the outer contour point cloud data of the object to be adjusted, which is used to reflect the size of the object to be adjusted and its three-dimensional space coordinate information in a preset space.

4. The luggage posture adjustment method according to claim 1, characterized in that: The method of determining a method for adjusting the position and posture of the target to be adjusted based on the classification information of the target to be adjusted and the outer contour point cloud data includes: Generating text input data conforming to the large language model processing format, wherein the text input data is text data that uses text to describe the classification information and the outer contour point cloud data of the target to be adjusted in a natural language; A text answer is given to the text input data using a trained large language model, wherein the text answer is used to provide a corresponding description of the manner in which the posture of the target to be adjusted is adjusted through natural language or computer language.

5. The luggage posture adjustment method according to claim 4, characterized in that: Before using the trained large language model to provide a text answer to the text input data, the method further includes: Using a short description instruction, feature alignment is performed on point features of the sample point cloud data and response text tags of the basic large language model to convert the sample point cloud data into input data containing embedded adjustment tags; The feature mapper and the multilayer perceptron of the basic large language model are trained by using the input data after feature alignment, and the parameters of the feature mapper and the multilayer perceptron of the basic large language model are adjusted; Freeze the parameters of the point cloud encoder, use complex instructions, and jointly with the feature mapper after parameter adjustment, train the semantic understanding ability of the prompt word of the large language model after parameter adjustment of the multilayer perceptron to obtain the trained large language model.

6. The luggage posture adjustment method according to claim 5, characterized in that: The using the trained large language model to provide a text answer to the text input data includes: Use the point cloud encoder to extract features from the point cloud data to obtain point cloud feature sequence data; Using a feature mapper, the point cloud feature sequence data is mapped into a latent space of a backbone network of the large language model, so as to map the point cloud features into point cloud label data, so that the data dimension of the point cloud label data is the same as the data dimension of the prompt word semantic text of the large language model; Constructing mixed label sequence data corresponding to the point cloud label; Using a self-attention mechanism, generating predicted response tag sequence data corresponding to the mixed tag sequence data; According to a preset semantic generation algorithm, combined with the predicted response tag sequence data, and using a parser to comprehensively analyze the image classification results, the luggage posture control instructions and text answer data are generated.

7. The luggage posture adjustment method according to claim 1, characterized in that Before using the trained classification neural network model to identify and classify the RBG image data and determine the classification information of the target to be adjusted contained in the target area, the method also includes: The acquired depth image data and RBG image data are preprocessed to obtain depth image data or RBG image data that meets the input requirements of the trained classification neural network model.

8. The luggage posture adjustment method according to claim 1, characterized in that: Obtaining the depth image data and RBG image data within the preset recognition area includes: Using an object detection sensor disposed in a preset area, detecting whether there is luggage in the preset identification area; When it is determined that there is luggage in the preset identification area, the 3D camera and RBG camera installed above the preset area to be identified are used to shoot the area to be identified to obtain depth image data and RBG image data in the preset identification area.

9. A luggage posture adjustment device, characterized in that: include: [HZTC-HXJKJD241017][HS2411267CCN] Image acquisition module, used to obtain depth image data and RBG image data within the preset recognition area; A classification module, used to use the depth image data and / or RBG image data as input data, and determine the classification information of the target to be adjusted placed in the target area in combination with a trained classification neural network model; A detection module, configured to perform three-dimensional reconstruction of the target to be adjusted based on the depth image data to obtain outer contour point cloud data of the target to be adjusted, wherein the outer contour point cloud data is used to represent three-dimensional coordinate information of the target to be adjusted in a preset spatial coordinate; The analysis module is used to determine a method for adjusting the posture of the target to be adjusted based on the classification information of the target to be adjusted and the outer contour point cloud data, so as to control the robot arm to adjust the posture of the target to be adjusted.

10. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and when the computer executable instructions are executed, the luggage posture adjustment method according to any one of claims 1 to 8 is executed.

Images