Material stacking method, device and equipment based on large model and medium

Abstract

The application discloses a material stacking method and device based on a large model, equipment and a medium. The method comprises the following steps: converting a voice instruction input by a user into a text instruction through a preset language interaction model; analyzing and processing the text instruction and a corresponding target acquisition image through a preset image processing model to obtain space information of a material to be stacked and state information of a stacking storage area; generating a stacking control code through a preset large model according to the text instruction, the space information of the material to be stacked and the state information of the stacking storage area; and controlling a robot to stack the material to be stacked according to the stacking control code. The method can solve the problem that an industrial robot in the prior art is difficult to use and cannot be applied to a complex stacking task.

Description

Technical Field

[0001] This invention relates to the technical field of material palletizing, and in particular to a material palletizing method, apparatus, equipment and medium based on a large model. Background Technology

[0002] With the continuous advancement of robotics technology, industrial robots are increasingly widely used in various fields, especially demonstrating great potential in palletizing operations for the efficient handling of various objects. However, existing industrial robots rely heavily on control codes manually compiled by users according to specific palletizing requirements when performing palletizing tasks. Unfortunately, once the shape, specifications, or palletizing standards of the materials change, the user must intervene again, manually adjust the control codes, and undergo multiple tedious debugging processes to ensure the robot can adapt to the new operational requirements. This frequent and complex need to adjust the control codes not only increases the difficulty of operation but also limits the ability of industrial robots to flexibly handle diverse and complex palletizing tasks, making it difficult for existing industrial robots to adapt to complex and varied palletizing tasks. Summary of the Invention

[0003] This invention provides a material palletizing method, apparatus, equipment, and medium based on a large model, aiming to solve the problem that industrial robots in the prior art are difficult to use and cannot be applied to complex palletizing tasks.

[0004] In a first aspect, embodiments of the present invention provide a material palletizing method based on a large model, comprising: converting user-inputted voice commands into text commands through a preset language interaction model; analyzing and processing the text commands and corresponding target acquisition images through a preset image processing model to obtain spatial information of the material to be palletized and state information of the palletizing storage area; generating palletizing control code by using the text commands, the spatial information of the material to be palletized, and the state information of the palletizing storage area through a preset large model; and controlling a robot to palletize the material to be palletized according to the palletizing control code.

[0005] Secondly, embodiments of the present invention also provide a material palletizing device based on a large model, comprising: a conversion unit for converting user-inputted voice commands into text commands through a preset language interaction model; an analysis unit for analyzing and processing the text commands and corresponding target acquired images through a preset image processing model to obtain spatial information of the material to be palletized and state information of the palletizing storage area; a generation unit for generating palletizing control code from the text commands, the spatial information of the material to be palletized, and the state information of the palletizing storage area through a preset large model; and a palletizing unit for controlling a robot to palletize the material to be palletized according to the palletizing control code.

[0006] Thirdly, embodiments of the present invention also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0007] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the above-described method.

[0008] This invention provides a material palletizing method, apparatus, device, and medium based on a large model. The method includes: converting user-inputted voice commands into text commands using a preset language interaction model; analyzing the text commands and corresponding target images using a preset image processing model to obtain spatial information of the material to be palletized and state information of the palletizing storage area; generating palletizing control code using the text commands, the spatial information of the material to be palletized, and the state information of the palletizing storage area through a preset large model; and controlling a robot to palletize the material according to the palletizing control code. This invention avoids the need for users to write complex code and the process of teaching the industrial robot, thereby reducing the difficulty of using the industrial robot and enabling it to handle various complex palletizing scenarios. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating the material palletizing method based on a large model provided in an embodiment of the present invention;

[0011] Figure 2 A schematic diagram of a sub-process of a material palletizing method based on a large model provided in an embodiment of the present invention;

[0012] Figure 3 A schematic diagram of a sub-process of a material palletizing method based on a large model provided in an embodiment of the present invention;

[0013] Figure 4 A schematic diagram of a sub-process of a material palletizing method based on a large model provided in an embodiment of the present invention;

[0014] Figure 5 A schematic diagram of a sub-process of a material palletizing method based on a large model provided in an embodiment of the present invention;

[0015] Figure 6 A schematic diagram of the palletizing point locations for a material palletizing method based on a large model provided in an embodiment of the present invention;

[0016] Figure 7 A schematic diagram of a sub-process of a material palletizing method based on a large model provided in an embodiment of the present invention;

[0017] Figure 8 A schematic diagram of a sub-process of a material palletizing method based on a large model provided in an embodiment of the present invention;

[0018] Figure 9 A schematic block diagram of a material palletizing device based on a large model provided for an embodiment of the present invention;

[0019] Figure 10 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] Please see Figure 1 , Figure 1 This is a flowchart illustrating the material palletizing method based on a large model provided in this embodiment of the invention. The material palletizing method based on a large model in this embodiment can be applied to the control of palletizing robots. Palletizing tasks are obtained through the palletizing robot's language interaction model, and code for controlling the robot is generated based on the obtained palletizing tasks using an image processing model and a preset large model. This eliminates the need for users to write complex code and avoids the industrial robot teaching process, thereby reducing the difficulty of using industrial robots.

[0025] Figure 1 This is a flowchart illustrating a material palletizing method based on a large model provided in an embodiment of the present invention. As shown in the figure, the method includes the following steps S110-S140.

[0026] S110. Convert user-inputted voice commands into text commands using a preset language interaction model.

[0027] In this embodiment, the preset language interaction model refers to a model used to enable interaction between humans and devices through natural language. This preset language interaction model is an artificial intelligence system, including a speech recognition model and a speech synthesis model. The speech recognition model converts the user's voice input into text, and the speech synthesis model converts the response text into natural and fluent speech output. The specific training process for the speech recognition and speech synthesis models is not limited, as long as the corresponding effect is achieved. The user issues voice commands with palletizing tasks to the palletizing robot using natural language. The speech recognition model in the preset language interaction model converts the natural language into text commands. The voice commands must describe the characteristics of the material to be palletized and the requirements for palletizing. For example, if the user inputs a voice command such as "Place the express boxes in 3 rows and 3 columns, 2 layers, on the wooden board," the speech recognition model can convert it into a text command. By converting the user's voice command into a text command, specific and understandable execution instructions are obtained, and corresponding control codes are generated based on these instructions.

[0028] S120. The text instructions and the corresponding target acquisition images are analyzed and processed by a preset image processing model to obtain the spatial information of the material to be palletized and the status information of the palletizing storage area.

[0029] In this embodiment, the target acquisition image is an image of the material to be palletized and an image of the palletizing storage area acquired according to a text command. Specifically, the text command carries the material to be palletized and the area where it will be stored. After acquiring the text command, the robot will automatically acquire image data of the material to be palletized and the palletizing storage area through a vision sensor and use it as the target acquisition image. The preset image processing model refers to a computational model used to process and analyze images. Specifically, in this embodiment, the preset image processing model can be a VLM (Vision-Language Models) multimodal large model or a neural network model. The VLM multimodal large model is an artificial intelligence model that can process image and text data simultaneously. This model integrates information from both visual and language modalities and can perform complex cross-modal tasks. Specifically, the VLM multimodal large model can be pre-trained using training images to determine information such as the location, size, and quantity of the target. The specific training process is not limited. The text commands and corresponding target images are input into the preset image processing model for analysis and processing. This allows the acquisition of spatial information about the material to be palletized, such as its position and dimensions, as well as status information about the palletizing storage area, such as its location and the quantity of palletized material. The analysis and processing by the preset image processing model provides a data foundation for generating the robot's control code.

[0030] In one embodiment, such as Figure 2 As shown, steps S1201-S1203 are included before step S120.

[0031] S1201. Search for the corresponding material to be palletized among several materials according to the feature information of the material to be palletized, and perform image acquisition on the material to be palletized to obtain an image of the material to be palletized.

[0032] S1202. Locate the corresponding palletizing storage area according to the palletizing requirement information, and acquire an image of the palletizing storage area.

[0033] S1203. Determine the target acquisition image based on the image of the material to be palletized and the image of the palletizing storage area.

[0034] In this embodiment, the text instruction includes material characteristic information to be palletized and palletizing requirement information. The material characteristic information defines the material's attributes, such as the category and quantity to be palletized; for example, the material characteristic information could be 18 express boxes. The palletizing requirement information defines the palletizing method and location, such as palletizing the materials in a 3×3×2 (3 rows, 3 columns, 2 layers) arrangement to storage area A. Based on the material characteristic information, the corresponding material to be palletized is searched among several materials. Specifically, based on the attributes of the material to be palletized, the corresponding material is searched among several stored materials, and image information of the material to be palletized is acquired using a visual sensor. The corresponding palletizing storage area is located based on the palletizing location in the palletizing requirements, and an image of the palletizing storage area is acquired. The target acquisition image is determined by combining the image of the material to be palletized and the image of the palletizing storage area. By acquiring image data according to the information corresponding to the text instruction, the specific size information of the material and the spatial information of the palletizing storage area are obtained.

[0035] In one embodiment, such as Figure 3 As shown, step S120 further includes steps S121-S122.

[0036] S121. The feature information of the material to be palletized and the image of the material to be palletized are analyzed and processed by the preset image processing model to obtain the position information and size information of the material to be palletized.

[0037] S122. The palletizing requirement information and the image of the palletizing storage area are analyzed and processed by the preset image processing model to obtain the location information of the palletizing storage area and the quantity information of the palletized materials.

[0038] In this embodiment, the feature information of the material to be palletized and the image of the material to be palletized are both inputs to the preset image processing model at the same time. After being input into the model together, the model can generate and output the position and size information of the material to be palletized. Specifically, the preset image processing model can preprocess the input image of the material to be palletized, including image enhancement, denoising, grayscale conversion, and other preprocessing operations to obtain a high-quality image. The feature extraction module then extracts features from the preprocessed image to obtain feature information related to the material to be palletized, such as the material's edges, contours, colors, and textures. Based on the feature information, it is determined whether it matches the feature information of the material to be palletized. If it matches, the position of the material in the image coordinate system is determined by the position detection algorithm in the image processing model, such as edge detection and contour tracking. Based on the camera calibration parameters and the mapping relationship between the image coordinate system and the robot's base coordinate system, the position in the image coordinate system is converted into the position in the robot's base coordinate system. Furthermore, by using a size measurement algorithm in the image processing model, the physical dimensions of the material, such as length, width, and height, are calculated based on the extracted material feature information, thereby obtaining the position and size information of the material to be palletized. The specific analysis and processing flow of the preset image processing model is not limited, as long as it can obtain accurate position and size information of the material to be palletized. The palletizing requirements information and the image of the palletizing storage area are used as the same input and jointly input to the preset image processing model for analysis and processing. The specific analysis and processing flow is the same as the flow for obtaining the position and size information of the material to be palletized, and will not be repeated here. It should be noted that the position of the palletizing storage area is only a marker point, not an entire region. The image of the palletizing storage area is acquired using a vision sensor, and after analysis and processing by the preset image processing model, coordinates are obtained in a coordinate system established with the robot's base (usually the robot's mounting base) as the origin. The robot can then directly use this coordinate system to move to that position. The image processing model is used to analyze and process the data to obtain the location and size information of the material to be palletized, as well as the location information of the palletizing storage area and the quantity of the material already palletized. This facilitates the generation of corresponding control codes to control the robot to perform palletizing.

[0039] S130. The text instructions, the spatial information of the material to be palletized, and the status information of the palletizing storage area are used to generate palletizing control code through a preset large model.

[0040] In this embodiment, the preset large-scale model is a large-scale language model based on deep learning technology, possessing strong learning and reasoning capabilities. In this embodiment, the preset large-scale model serves as the foundational large-scale model. Through deep learning technology, this model can automatically learn and process large amounts of natural language text data, generating natural and fluent language text. The text instructions, the spatial information of the material to be palletized, and the state information of the palletizing storage area are used by the preset large-scale model to generate palletizing control code. Specifically, after receiving the text instructions, the spatial information of the material to be palletized, and the state information of the palletizing storage area, the large-scale model preprocesses and extracts features from the input information to capture key features such as the material's position, size, palletizing requirements, and the state of the storage area. Then, the model applies a series of optimization algorithms, such as path planning algorithms and collision detection algorithms, to determine the optimal palletizing strategy and the robot's motion trajectory. These algorithms comprehensively consider various factors, such as material handling efficiency, palletizing stability, and robot kinematic constraints, thereby generating the palletizing control code. The control code consists of a series of basic action API functions, each corresponding to a specific action performed by the robot, such as grasping materials, moving to a designated location, or placing materials. These basic action API functions are combined in a predetermined order and with predetermined parameters to form complete palletizing control code. A pre-set large model generates control code based on the text instructions, the spatial information of the materials to be palletized, and the state information of the palletizing storage area, ensuring that the generated palletizing control code is both efficient and accurate, thus improving the efficiency and accuracy of palletizing operations.

[0041] In one embodiment, such as Figure 4 As shown, step S130 further includes steps S131-S132.

[0042] S131. A prompt instruction to construct the preset large model based on the text instruction, the spatial information of the material to be palletized, the status information of the palletizing storage area, and the preset basic function, preset output requirements, and preset samples;

[0043] S132. Input the prompt command into the preset large model to obtain the palletizing control code that meets the preset output requirements.

[0044] In this embodiment, the prompt instruction is the input text or command provided to the model to guide it in generating a specific type of response. Specifically, the large model only accepts one type of input, namely the prompt instruction. All engineering work related to the large model revolves around the prompt instruction. Therefore, it is necessary to construct the prompt instruction of the preset large model based on the text instruction, the spatial information of the material to be palletized, the state information of the palletizing storage area, and preset basic functions, preset output requirements, and preset samples. Specifically, the prompt instruction consists of three parts: task description, output format, and sample example. The task description defines the role of the task as an industrial robot palletizing code generation assistant, provides the task instructions as output functions and responses, and explains the function information related to the task. The output format describes the form of output requirements and specifies the operation keys and execution keys for the output requirements. The sample example provides standard responses to input and output, which the large model can learn and imitate. Specifically, the user's voice input is converted into text and used as a task description. Preset basic functions and preset output requirements are used as the output format, preset samples are used as example samples, and the spatial information of the material to be palletized and the state information of the palletizing storage area are used as context, along with other background information related to the task. These together constitute the prompts for the preset large model. When these prompts are input into the preset large model, the model generates palletizing control code that conforms to the preset output requirements based on the preset samples and preset basic functions. For example:

[0045] #Task Description

[0046] instruction="..."

[0047] You are an industrial robot palletizing code generation assistant. The robot has some built-in functions. Your task is to output the functions to be run and the responses in JSON format, based on my instructions.

[0048] Introduction to built-in functions:

[0049] Palletizing point generation function: point_generation()

[0050] Stacking execution function: stacking() ...

[0052] #Output Format

[0053] output format = ...

[0054] The 'function' key outputs a list of function names, representing the names and parameters of the functions to be run.

[0055] In the 'response' button, following my instructions, output your reply to me in the first person.

[0056] #Small Sample Examples

[0057] examples="..."

[0058] My instructions: Stack the gifts into the boxes in a 2x2x3 pattern.

[0059] You output in this format:

[0060] {

[0061] 'function':[point_generation(2,2,3,l 礼品 ,w 礼品 ,h 礼品 ,x 箱子 ,y 箱子 ,z 箱子 )','stacking(x 礼品 ,y 礼品 ,z 礼品 ,A,0)']

[0062] Response: Instructions received. We are stacking gifts into boxes. Please maintain a safe distance.

[0063] }

[0064] Where 2,2,3 represent the palletizing method, l 礼品 ,w 礼品 ,h 礼品 For the size information of the gift, x 箱子 ,y 箱子 ,z 箱子 For the location information of the box, x 礼品 ,y 礼品 ,z 礼品 This refers to the location information of the gifts, where A,0 represents the location and quantity of the gifts. The code above is merely an example of a prompt instruction and not a specific limitation; therefore, the parameters will not be explained in detail. Generating palletizing control code from a large model eliminates the need for users to write complex code and avoids the industrial robot teaching process, thus reducing the difficulty of using industrial robots.

[0065] In one embodiment, such as Figure 5 As shown, step S132 further includes steps S1321-S1324.

[0066] S1321. The size information of the material to be palletized, the palletizing requirement information, and the location information of the palletizing storage area are used as the receiving parameters of the palletizing point generation function.

[0067] S1322. Obtain the palletizing point generation code through the palletizing point generation function according to the received parameters, wherein the palletizing point generation code generates a palletizing point for each material to be palletized in the palletizing storage area.

[0068] S1323. Based on the location information of the material to be palletized, the palletizing point, and the quantity information of the already palletized material, generate palletizing execution code through the palletizing execution function;

[0069] S1324. Determine the palletizing control code based on the palletizing point generation code and the palletizing execution code.

[0070] In this embodiment, the preset basic functions include a palletizing point generation function and a palletizing execution function. The palletizing point generation function generates the location points (palletizing points) where the material to be palletized is stored in the palletizing storage area. The palletizing execution function controls how the robot performs palletizing. Specifically, the palletizing point generation function `point_generation(L,W,H,l)`... 物料 ,w 物料 ,h 物料 ,x 存放 ,y 存放 ,z 存放 This is a function that generates the three-dimensional coordinates of the palletizing points. It receives the palletizing method (L (rows) × W (columns) × H (layers)) and the size information of the material to be palletized (l... 物料 ,w 物料 ,h 物料 Location information of the palletizing storage area (x 存放 ,y 存放 ,z 存放 The palletizing method defines a matrix A that stores the three-dimensional coordinates of the palletizing points. Matrix A has a size of L×W×H. The palletizing points in the palletizing storage area are used as the values ​​of matrix A[0][0][0]. The size of the material to be palletized is l. 物料 ,w 物料 ,h 物料These values ​​represent the increments along the positive directions of the X, Y, and Z axes, respectively. The size of the material to be palletized is represented as the increment along the positive directions of the X, Y, and Z axes. Each palletizing point in matrix A has its own coordinates. Specifically, the coordinates of row i are: xi = x + i * l; the coordinates of column j are: yj = y + j * w; and the coordinates of layer k are: zk = z + k * h. The values ​​of i range from 0 to L-1, j from 0 to W-1, and k from 0 to H-1. For example, if the palletizing method involves placing materials in a 3x3 pattern with two layers, and the size of the material to be palletized is (250, 250, 100), and the palletizing storage area is located at (150, 150, 0), then this function generates the positions of each palletizing point, as shown below. Figure 6 The diagram shows the location of the palletizing points. The function that generates the palletizing data is a pre-written function; there are no restrictions on its implementation, as long as it can generate the corresponding code based on the received parameters and methods. The palletizing execution function stacking(x) 物料 ,y 物料 ,z 物料 ,A,n) is an encapsulated function that implements the process of grabbing materials to be palletized and placing them at various palletizing points. The parameters received by the palletizing execution function are the location information of the materials to be palletized, the palletizing point matrix A output by the palletizing point generation function, and the number of materials already in the storage area. Based on the location information of the materials to be palletized, the palletizing points, and the number of materials already palletized, the palletizing execution function generates palletizing execution code. For example, if there are 18 materials to be palletized, arranged in a 3*3 pattern on each layer, and there are 9 materials already palletized, then skip the position from A[0][0[0] to A[2][2][0] (i.e., all palletizing points on the first layer) and generate stacking(x 物料 ,y 物料 ,z 物料 The palletizing execution code is A[0][0[1] (the first palletizing point of the second layer). The palletizing control code is determined based on the palletizing point generation code and the palletizing execution code. By generating the palletizing control code, the robot can palletize the materials according to the palletizing point and the palletizing storage location.

[0071] S140. Control the robot to palletize the material to be palletized according to the palletizing control code.

[0072] In this embodiment, the robot moves according to the instructions of the generated palletizing control code to pick up the materials to be palletized, and moves to the designated palletizing point according to the trajectory planned by the palletizing control code. After reaching the palletizing point, the robot places the materials in the predetermined position according to the instructions of the palletizing control code to complete one palletizing operation. After completing one material picking, moving, and placing operation, the robot returns to the material storage area and repeats the above steps until all the materials to be palletized have been palletized. By controlling the robot to palletize the materials according to the palletizing control code, a reasonable palletizing strategy can enable the robot to perform precise palletizing.

[0073] In one embodiment, such as Figure 7 As shown, step S140 includes steps S141-S142.

[0074] S141. Control the robot to obtain the material to be palletized based on the position information of the material to be palletized;

[0075] S142. Based on the quantity information of the already stacked materials, remove the occupied stacking points and move the materials to be stacked to unoccupied stacking points.

[0076] In this embodiment, the robot moves the material to be palletized through the palletizing execution function in the palletizing control code. Specifically, the robot moves to a safety point above the material to be palletized based on the position information of the material to be palletized, and descends to the gripping point of the material to be palletized to grip the material. After moving to the safety point above the material to be palletized again, the robot moves the material to be palletized. The safety point and the gripping point can be set according to the specific material to be palletized, and there is no limitation on them. Based on the matrix A containing the three-dimensional coordinates of the palletizing points and the information on the quantity of already palletized materials, the materials are arranged. Specifically, the occupied palletizing points in matrix A are removed. If there are 18 materials to be palletized, each layer is arranged in a manner of 3 (rows) × 3 (columns) × 2 (layers). If there are already 9 palletized materials, then the first layer is full. At this time, the positions A[0][0[0], A[1][0[0], A[][1[0], A[1][1[0], up to A[2][2][0] in the first layer of matrix A (i.e., all palletizing points in the first layer) are occupied. The occupied palletizing points are removed, and the robot is controlled to place the materials to be palletized at the second layer A[0][0][1] (the first position in the second layer), and the steps of moving to the safe point above the materials to be palletized and transporting the materials are repeated. By controlling the robot to move the material to be palletized to an unoccupied palletizing point, the material does not need to be placed in a fixed position input by a human, which is conducive to the development and promotion of flexible production lines.

[0077] In one embodiment, such as Figure 8 As shown, step S140 is followed by steps S1401-S1402.

[0078] S1401. Determine whether a palletizing abnormality has occurred based on sensor detection information;

[0079] S1402. If an abnormality occurs, the control signal of the robot is triggered to control the robot to broadcast the abnormality.

[0080] In this embodiment, the sensor detection information is information detected by sensors pre-installed on the robot. The system determines whether a palletizing anomaly has occurred based on the sensor detection information. For example, the material position sensor can detect whether the material is at a predetermined position. If no material to be palletized is found at the predetermined position, a palletizing anomaly is determined. A palletizing anomaly refers to any abnormal situation that occurs during the robot's palletizing process, including but not limited to abnormal shutdown, missing material, or palletizing completion. If an anomaly occurs, the robot's control signal is triggered. When the control signal is triggered, the robot is instructed to broadcast the anomaly. Specifically, the anomaly can be broadcast using a speech synthesis model within the preset language conversion model in step S110. By determining whether an anomaly has occurred and controlling the robot to broadcast the anomaly when it occurs, the user is promptly alerted to the robot's abnormal palletizing status, facilitating timely adjustments to the robot or the palletizing task.

[0081] Figure 9 This is a schematic block diagram of a material palletizing device 200 based on a large model provided in an embodiment of the present invention. Figure 9 As shown, corresponding to the above-described material palletizing method based on a large model, the present invention also provides a material palletizing device based on a large model. This material palletizing device includes a unit for executing the above-described material palletizing method based on a large model, and the device can be configured in a terminal such as a desktop computer, tablet computer, or laptop computer. For details, please refer to... Figure 9 The material palletizing device based on the large model includes a conversion unit 210, an analysis unit 220, a generation unit 230, and a palletizing unit 240.

[0082] The conversion unit 210 is used to convert user-inputted voice commands into text commands through a preset language interaction model.

[0083] The analysis unit 220 is used to analyze and process the text instructions and the corresponding target acquisition images through a preset image processing model to obtain the spatial information of the material to be palletized and the status information of the palletizing storage area.

[0084] In one embodiment, the analysis unit 220 includes a search unit, a search unit, and a determination unit.

[0085] The search unit is used to search for the corresponding material to be stacked among several materials based on the feature information of the material to be stacked, and to acquire an image of the material to be stacked.

[0086] The searching unit is used to find the corresponding palletizing storage area according to the palletizing requirement information, and to acquire an image of the palletizing storage area.

[0087] The determining unit is used to determine the target acquisition image based on the image of the material to be palletized and the image of the palletizing storage area.

[0088] In one embodiment, the analysis unit 220 includes a first processing unit and a second processing unit.

[0089] The first processing unit is used to analyze and process the feature information of the material to be palletized and the image of the material to be palletized through the preset image processing model to obtain the position information and size information of the material to be palletized.

[0090] The second processing unit is used to analyze and process the palletizing requirement information and the image of the palletizing storage area through the preset image processing model to obtain the location information of the palletizing storage area and the quantity information of the palletized materials.

[0091] The generation unit 230 is used to generate palletizing control code by using the text instructions, the spatial information of the material to be palletized, and the status information of the palletizing storage area through a preset large model.

[0092] In one embodiment, the generation unit 230 includes a construction unit and an acquisition unit.

[0093] The construction unit is used to construct the preset large model based on the text instructions, the spatial information of the material to be palletized, the status information of the palletizing storage area, and the preset basic functions, preset output requirements, and preset samples.

[0094] The acquisition unit is used to input the prompt instruction into the preset large model to obtain the palletizing control code that meets the preset output requirements.

[0095] In one embodiment, the generation unit 230 includes a receiving unit, a first generation unit, a second generation unit, and a determining unit.

[0096] The receiving unit is used to take the size information of the material to be palletized, the palletizing requirement information, and the location information of the palletizing storage area as the receiving parameters of the palletizing point generation function;

[0097] The first generation unit is used to obtain a palletizing point generation code through the palletizing point generation function according to the received parameters, wherein the palletizing point generation code generates a palletizing point for each material to be palletized in the palletizing storage area.

[0098] The second generation unit is used to generate palletizing execution code based on the location information of the material to be palletized, the palletizing point, and the quantity information of the already palletized material through the palletizing execution function.

[0099] A determining unit is used to determine the palletizing control code based on the palletizing point generation code and the palletizing execution code.

[0100] The palletizing unit 240 is used to control the robot to palletize the material to be palletized according to the palletizing control code.

[0101] In one embodiment, the palletizing unit 240 includes a rejection unit and a moving unit.

[0102] The rejection unit is used to control the robot to obtain the material to be palletized based on the position information of the material to be palletized.

[0103] The moving unit is used to remove occupied palletizing points based on the quantity information of the already palletized materials, and move the materials to be palletized to unoccupied palletizing points.

[0104] In one embodiment, the palletizing unit 240 further includes a judgment unit and a broadcasting unit.

[0105] The judgment unit is used to determine whether a palletizing abnormality has occurred based on sensor detection information.

[0106] The broadcasting unit is used to trigger the robot's control signal if an abnormality occurs, and control the robot to broadcast the abnormality.

[0107] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the material palletizing device 200 based on the large model and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0108] The aforementioned material palletizing device based on a large model can be implemented as a computer program, which can be used in, for example... Figure 10 It runs on the computer device shown.

[0109] Please see Figure 10 , Figure 10This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0110] See Figure 10 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0111] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a material palletizing method based on a large model.

[0112] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0113] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a material palletizing method based on a large model.

[0114] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0115] The processor 502 is used to run a computer program 5032 stored in a memory to implement the steps of the above method.

[0116] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0117] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0118] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the steps of the method described above.

[0119] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0121] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0122] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0123] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0124] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A material palletizing method based on a large model, characterized in that, include: The user's voice commands are converted into text commands by a preset language interaction model. The text commands include the material characteristics and palletizing requirements. Based on the characteristic information of the material to be palletized, the corresponding material to be palletized is searched among several materials, and an image of the material to be palletized is acquired. Based on the palletizing requirements, the corresponding palletizing storage area is located, and an image of the palletizing storage area is acquired. Based on the image of the material to be palletized and the image of the palletizing storage area, a target image is determined. The characteristic information of the material to be palletized and the image of the material to be palletized are analyzed and processed using a preset image processing model to obtain the position and size information of the material to be palletized. The palletizing requirements and the image of the palletizing storage area are analyzed and processed using the preset image processing model to obtain the position information of the palletizing storage area and the quantity information of the material already palletized. Based on the text instructions, the spatial information of the material to be palletized, the status information of the palletizing storage area, and the prompt instructions to construct a preset large model using preset basic functions, preset output requirements, and preset samples, the preset basic functions include a palletizing point generation function and a palletizing execution function; the size information of the material to be palletized, the palletizing requirement information, and the location information of the palletizing storage area are used as the receiving parameters of the palletizing point generation function. Based on the received parameters, a palletizing point generation code is obtained through the palletizing point generation function, wherein the palletizing point generation code generates a palletizing point for each of the materials to be palletized in the palletizing storage area; a palletizing execution code is generated through the palletizing execution function based on the location information of the materials to be palletized, the palletizing points, and the quantity information of the already palletized materials; and a palletizing control code is determined based on the palletizing point generation code and the palletizing execution code. The robot is controlled to palletize the materials to be palletized according to the palletizing control code.

2. The method according to claim 1, characterized in that, The step of controlling the robot to palletize the material to be palletized according to the palletizing control code includes: The robot is controlled to acquire the material to be palletized based on its location information. Based on the quantity information of the already palletized materials, the occupied palletizing points are removed, and the materials to be palletized are moved to unoccupied palletizing points.

3. The method according to claim 1, characterized in that, After the step of controlling the robot to palletize the material to be palletized according to the palletizing control code, the following steps are included: Determine whether a palletizing abnormality has occurred based on sensor detection information; If an anomaly occurs, the robot's control signal is triggered, and the robot is controlled to broadcast the anomaly.

4. A material palletizing device based on a large model, characterized in that, include: The conversion unit is used to convert user-inputted voice commands into text commands through a preset language interaction model. The text commands include the material characteristics information to be palletized and the palletizing requirements information. The analysis unit is configured to: search for the corresponding material to be palletized among several materials based on the material's characteristic information; acquire an image of the material to be palletized; locate the corresponding palletizing storage area based on the palletizing requirement information; acquire an image of the palletizing storage area; determine the target acquisition image based on the image of the material to be palletized and the image of the palletizing storage area; analyze and process the material's characteristic information and the image of the material to be palletized using a preset image processing model to obtain the material's position and size information; and analyze and process the palletizing requirement information and the image of the palletizing storage area using the preset image processing model to obtain the location information of the palletizing storage area and the quantity of materials already palletized. The generation unit is used to construct a preset large model based on the text instructions, the spatial information of the material to be palletized, the status information of the palletizing storage area, and the preset basic functions, preset output requirements, and preset samples. The preset basic functions include a palletizing point generation function and a palletizing execution function. The size information of the material to be palletized, the palletizing requirement information, and the position information of the palletizing storage area are used as the receiving parameters of the palletizing point generation function. Based on the received parameters, a palletizing point generation code is obtained through the palletizing point generation function, wherein the palletizing point generation code generates a palletizing point for each of the materials to be palletized in the palletizing storage area; a palletizing execution code is generated through the palletizing execution function based on the location information of the materials to be palletized, the palletizing points, and the quantity information of the already palletized materials; and a palletizing control code is determined based on the palletizing point generation code and the palletizing execution code. The palletizing unit is used to control the robot to palletize the material to be palletized according to the palletizing control code.

5. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1-3.

6. A storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, can implement the method as described in any one of claims 1-3.

Images