Robot control methods, devices, storage media and electronic equipment
By creating a container model and obtaining target model data in the model editing page, the problem of the robot being unable to recognize objects of different shapes for handling was solved, and efficient and accurate handling tasks were achieved.
Patent Information
- Application Number
- CN202510742242.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-04
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2045-06-04
AI Technical Summary
Robots are unable to effectively identify objects of different shapes, affecting handling efficiency and accuracy.
By creating a container model in the model editing page, the target model data is obtained, and a handling task is sent to the robot to carry the target model data of the container model for identification.
This improves the efficiency and accuracy of robots in recognizing objects of different shapes, ensuring the successful execution of handling tasks.
Smart Images

Figure CN120779800B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of intelligent warehousing technology, specifically to a robot control method, device, storage medium, and electronic device. Background Technology
[0002] With the development of smart warehousing technology, robots are being widely used. Robots can be used to move objects such as boxes and shelves.
[0003] Currently, in order for robots to transport objects, they need to be able to identify those objects. However, in practical applications, there are objects of various shapes that robots cannot effectively identify, affecting the robot's transport efficiency and accuracy. Summary of the Invention
[0004] In view of this, the present disclosure provides a robot control method, device, storage medium and electronic device, the main purpose of which is to improve the current problem that robots cannot effectively identify objects of different shapes, which affects the robot's handling efficiency and accuracy.
[0005] In a first aspect, this disclosure provides a robot control method, including:
[0006] Obtain the container model corresponding to the transport object. The container model is created by the user in advance in the model editing page, which includes editing items for model attributes.
[0007] Based on the user's settings for the edited items, obtain the target model data of the container model;
[0008] The robot is sent a task to move the object being moved, the task carrying the target model data, which is used by the robot to identify the object being moved.
[0009] Optionally, obtaining the target model data of the container model based on the user's settings for the edited items includes:
[0010] Based on the user's settings for the edited items, the attribute characteristics of the container model are obtained;
[0011] The attribute features are determined as the target model data.
[0012] Optionally, before obtaining the container model corresponding to the transport object, the method further includes:
[0013] The model editing page corresponding to the container type selected by the user is displayed. The model editing page includes a 3D display of the basic model corresponding to the container type, as well as editing items for the model attributes of the basic model. The 3D display dynamically changes according to the user's settings for the editing items.
[0014] The container model is created based on the user's actions on the model editing page.
[0015] Optionally, after creating the container model based on the user's actions on the model editing page, the method further includes:
[0016] Obtain information about the moving site;
[0017] The container model created by the user is validated based on the transportation site information;
[0018] If the container model passes the verification, the container model created by the user will be entered into the database.
[0019] The step of obtaining the container model corresponding to the transport object includes:
[0020] Obtain the entered container model.
[0021] Optionally, the step of validating the container model created by the user based on the handling site information includes:
[0022] Based on the transport site information, obtain site environment data, which includes at least the path information that the robot can pass through, site area information, and obstacle distribution information;
[0023] The three-dimensional geometric parameters of the container model are dynamically fused with the site environment data to generate an interactive simulation scene.
[0024] A virtual handling test is performed in the interactive simulation scenario to determine whether the container model meets the handling stability constraints under different postures. The constraints include a center of gravity offset threshold, overturning moment limit, and contact surface sliding risk assessment.
[0025] If the container model passes the virtual handling test, then the container model is deemed to have passed the verification.
[0026] If the container model fails the virtual handling test, the user is prompted to adjust the parameters of the container model, and parameter adjustment suggestions are provided based on the results of the virtual handling test.
[0027] Optionally, the method further includes:
[0028] In response to a user's update command for the container model, the model editing page corresponding to the container model is displayed;
[0029] The container model is updated based on the user's actions on the model editing page.
[0030] Optionally, after sending the transport task of the object to be transported to the robot, the method further includes:
[0031] The object to be transported is displayed on the warehouse map based on the container model, and the movement trajectory of the robot transporting the object is displayed on the warehouse map. The movement trajectory is consistent with the actual transport trajectory of the object.
[0032] Optionally, the method further includes:
[0033] Determine the mapping relationship between model data of different container models and camera recognition patterns;
[0034] The mapping relationship is sent to the robot, and the mapping relationship is used by the robot to determine the camera recognition mode to be used when recognizing the transported object.
[0035] Optionally, determining the mapping relationship between model data of different container models and camera recognition patterns includes:
[0036] The camera parameters are determined based on the model data of the container model and the preset working environment. The camera parameters include one or more of the following: focal length, aperture size, ISO, exposure time, and white balance.
[0037] The model sample of the container model is photographed according to the camera parameters;
[0038] The camera recognition mode corresponding to the container model is determined based on the shooting results.
[0039] Optionally, determining the camera recognition mode corresponding to the container model based on the shooting results includes:
[0040] Image quality analysis was performed on the images of the model samples taken under different camera parameter conditions.
[0041] Obtain the target camera parameters corresponding to the target captured image whose image quality is greater than a preset threshold;
[0042] The camera recognition mode corresponding to the container model is determined based on the target camera parameters.
[0043] Optionally, determining the camera recognition mode corresponding to the container model based on the target camera parameters includes:
[0044] The target image is preprocessed;
[0045] The pre-processed target images are then identified using different visual algorithms.
[0046] A visual algorithm that obtains target recognition results that meet the requirements;
[0047] According to the target vision algorithm and the target camera parameters, the camera recognition mode corresponding to the container model is obtained.
[0048] Secondly, this disclosure provides a robot control method, including:
[0049] The system receives a transport task for a transported object. The transport task carries target model data of the container model corresponding to the transported object. The container model is created by the user in advance in the model editing page. The model editing page includes editing items for model attributes. The target model data is obtained based on the user's settings for the editing items.
[0050] The transport object is identified based on the target model data;
[0051] The transport task is executed based on the identification results.
[0052] Optionally, before identifying the transported object based on the target model data, the method further includes:
[0053] Receive the mapping relationship between model data of different container models and camera recognition patterns;
[0054] The step of identifying the transported object based on the target model data includes:
[0055] The camera recognition mode corresponding to the target model data is determined by querying the mapping relationship.
[0056] The object being transported is identified using the camera recognition mode corresponding to the target model data.
[0057] Optionally, the step of using the camera recognition mode corresponding to the target model data to identify the transported object includes:
[0058] Based on the camera recognition mode corresponding to the target model data, determine the camera parameters and vision algorithm;
[0059] The object being transported is identified based on the camera parameters and the vision algorithm.
[0060] The step of performing the transport task based on the identification result includes:
[0061] The location and edge information of the transported object are determined based on the recognition results;
[0062] The transport task of the transported object is executed based on the location information and edge information.
[0063] Optionally, the step of identifying the transported object based on the camera parameters and the visual algorithm includes:
[0064] The object being transported is photographed using a robotic camera device according to the camera parameters.
[0065] The visual algorithm is used to identify the captured image to obtain a target image with the outline of the transported object marked. The position and edge information of the transported object are then identified in the target image by combining the attribute features corresponding to the target model data.
[0066] Optionally, the method further includes:
[0067] If the object being transported fails to be identified after a preset number of attempts using the camera recognition mode determined based on the target model data, then a different camera recognition mode will be used to identify the object being transported.
[0068] Send the target camera recognition mode used when the object being transported is successfully identified, in order to update the mapping relationship between the model data of the container model and the camera recognition mode.
[0069] Thirdly, this disclosure provides a robot control device, comprising:
[0070] The processing module is configured to obtain the container model corresponding to the transport object, wherein the container model is created by the user in advance in the model editing page, and the model editing page includes editing items for model attributes; and to obtain the target model data of the container model based on the user's settings for the editing items.
[0071] The sending module is configured to send a handling task of the object to the robot. The handling task carries the target model data, which is used by the robot to identify the object.
[0072] Fourthly, this disclosure provides a robot control device, comprising:
[0073] The receiving module is configured to receive a transport task for a transported object. The transport task carries the target model data of the container model corresponding to the transported object. The container model is created by the user in advance in the model editing page. The model editing page includes editing items for model attributes. The target model data is obtained based on the user's settings for the editing items.
[0074] The execution module is configured to identify the transport object based on the target model data and execute the transport task based on the identification result.
[0075] Fifthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot control method described in the first or second aspect.
[0076] In a sixth aspect, this disclosure provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the robot control method described in the first or second aspect.
[0077] In a seventh aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the robot control method described in the first or second aspect.
[0078] By employing the above technical solution, this disclosure provides a robot control method, device, storage medium, and electronic device. First, it acquires a container model corresponding to the object being transported. This container model is created by the user in advance in a model editing page, which includes editing items for model attributes. Then, based on the user's settings for these editing items, it acquires the target model data of the container model. Next, it sends a transport task for the object to the robot. This transport task carries the target model data of the container model, which can be used by the robot to identify the object and execute the corresponding transport task. By applying the technical solution of this disclosure, the robot can quickly and effectively identify the object being transported based on the container model data carried in the transport task, and then execute the corresponding transport task. This container model can be created by the user in advance in the model editing page. For objects of different shapes, the user can create container models corresponding to these objects in advance in the model editing page. This allows the user to indicate the corresponding model data when issuing transport tasks to the robot, enabling the robot to effectively identify objects of different shapes and improving the robot's transport efficiency and accuracy.
[0079] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description
[0080] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0081] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0082] Figure 1 A schematic diagram of the structure of a warehousing system provided in an embodiment of this disclosure is shown;
[0083] Figure 2 A flowchart illustrating a robot control method provided in an embodiment of this disclosure is shown;
[0084] Figure 3 A schematic diagram of an example method provided by an embodiment of this disclosure is shown;
[0085] Figure 4 This illustration shows an example effect diagram provided by an embodiment of the present disclosure;
[0086] Figure 5 A schematic diagram of another example method provided by an embodiment of this disclosure is shown;
[0087] Figure 6 A schematic diagram illustrating yet another example provided by an embodiment of this disclosure is shown;
[0088] Figure 7 A schematic diagram of the structure of a robot control device provided in an embodiment of this disclosure is shown;
[0089] Figure 8 A schematic diagram of another robot control device provided in an embodiment of this disclosure is shown;
[0090] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation
[0091] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0092] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.
[0093] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.
[0094] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.
[0095] The warehousing system used in the embodiments of this disclosure will be described below with reference to the accompanying drawings.
[0096] Figure 1 This is a schematic diagram of a warehousing system used in an embodiment of this disclosure. Figure 1 As shown, the warehousing system may include at least: several carriers 10 and several robots 20. The carriers 10 can be placed in the storage area and can hold goods. In some examples, the carrier 10 may be a shelf with multiple storage positions, each storage position being used to place a box, which may contain goods or be an empty box. In addition, the carrier 10 may also be a box, pallet, etc.
[0097] The carrier 10 can be used as a transport object, and correspondingly, the robot 20 can be used to transport the carrier 10. In some examples, if the carrier 10 is a shelf, the robot 20 can transport the shelf from the entry point to a designated location in the warehouse, or deliver the shelf from the warehouse to the workstation for goods picking, etc. In some examples, if the carrier 10 is a box containing goods that needs to be transported to the workstation, the box can be placed on the buffer position of the shelf, and the robot 20 can pick up the box at the buffer position and deliver the box to the workstation for goods picking, etc.
[0098] In some embodiments, the carrier 10 serves as the object to be transported, and transport tasks can be assigned to the robot 20 for the carrier 10. However, in reality, there may be carriers 10 with different shapes (including unconventional shapes, etc.). For example, when the carrier 10 is a shelf, its shelf size may have various styles. In this case, the robot 20 may not be able to effectively identify these different shaped shelves, affecting the transport efficiency and accuracy of the robot 20.
[0099] Therefore, based on the above Figure 1 The warehousing system shown in this disclosure provides a robot control method that can be applied to server execution in warehouse management, such as servers equipped with a Warehouse Management System (WMS) or Warehouse Execution System (WES). Figure 2 As shown, this method can be used to improve the current problem that robots cannot effectively identify objects of different shapes, thus affecting the robot's handling efficiency and accuracy. The method includes the following steps 101 to 103:
[0100] Step 101: Obtain the container model corresponding to the transport object.
[0101] In some embodiments, the container model may be created by the user in advance in a model editing page, which may include editing items for model attributes, such as the size, structure, and settings of different faces of the transported object.
[0102] For example, if the object to be moved is a shelf, its corresponding container model can be a shelf model. The corresponding model attributes can include the length, width, and height of the shelf, the number of layers and columns on different sides of the shelf, the height of the layers, the width of the columns, the type of shelf, and so on. Users can pre-create the container model of the shelf in the model editing page so that when instructing the robot to move the shelf, the corresponding model data can be indicated at the same time, enabling the robot to quickly and accurately identify the shelf.
[0103] This disclosure allows for the acquisition of a container model corresponding to a transported object when a transport task needs to be assigned to a robot. In some examples, it can first determine whether a container model corresponding to the transported object exists. If it exists, it can be acquired directly; if it does not exist, the user can be prompted to create it on the model editing page, and the corresponding container model can be acquired after creation.
[0104] In some examples, users can instruct the handling of objects on the relevant management page of the warehouse system. For instance, they can instruct a robot to move an object from location A to location B. During this process, users can manually add container models corresponding to the objects being moved from a list of existing container models, such as different types of shelf models, box models, and pallet models. If a container model is not found in the list, it can be added in the model editing page using the "Add" function. After adding the container model for the object being moved, the user can instruct the robot to perform the movement.
[0105] Step 102: Based on the user's settings for editing model attributes, obtain the target model data of the container model.
[0106] In some embodiments, the attribute features of the container model can be obtained based on the user's settings for editing model attributes, such as the container model's size information, structural information, settings of different faces, weight, color, texture, etc.; and then these attribute features can be determined as the target model data of the container model corresponding to the transport object.
[0107] For example, if the object to be transported is a shelf, the user can pre-create a container model of the shelf in the model editing page. The settings of its attributes, such as the length, width and height of the shelf, the number of layers and columns on different sides of the shelf, the height of the layers, the width of the columns, and the type of the shelf, are used as the target model data for the container model corresponding to the shelf.
[0108] Step 103: Send the task of moving the object to the robot.
[0109] A handling task can be a task that instructs the robot to move an object. For example, if the object is a shelf, by issuing a corresponding handling task to the robot, the robot can be instructed to move the shelf from position 1 to position 2.
[0110] The handling task can carry target model data of the container model corresponding to the object being handled. This target model data can be used by the robot to identify the object being handled. For example, if the object being handled is a shelf, sending the shelf's length, width, and height dimensions, the number of layers and columns on different sides of the shelf, the height of the layers, the width of the columns, the type of shelf, the dimensions of the shelf legs, the weight of the shelf, its color, and texture to the robot can effectively improve the robot's efficiency and accuracy in identifying the shelf.
[0111] In some embodiments, the operation of each robot is scheduled through a Robot Management System (RMS), and correspondingly, the RMS system can send the task of transporting objects to the robots.
[0112] By applying the solution of this disclosure embodiment, the robot can quickly and effectively identify the object to be transported based on the container model data carried in the transport task, and then perform the corresponding transport task. The container model can be created by the user in advance in the model editing page. For transport objects of different shapes, the user can create corresponding container models for each object in advance in the model editing page. This allows the user to instruct the robot on the corresponding model data when issuing transport tasks, enabling the robot to effectively identify transport objects of different shapes and improving the robot's transport efficiency and accuracy.
[0113] A warehouse map, in an intelligent warehousing system, is a digital map used to represent the internal structure, storage area divisions, equipment locations, and route planning of a warehouse. It is not only a virtual representation of the warehouse's physical space but also a crucial foundation for automated operation and management. As an optional method, after sending a handling task to a robot, a generic icon for the object can be used to display it on the warehouse map, along with the robot's movement trajectory, which matches the actual movement trajectory of the object.
[0114] For example, if the object being moved is a shelf, regardless of its size, a universal shelf icon can be used to display it on the warehouse map. The map can also show the robot's movement trajectory as it moves the shelf. For instance, by displaying a robot icon on the map and combining it with the universal shelf icon, the effect of the robot moving the shelf can be visualized, and the corresponding movement trajectory matches the actual movement of the shelf. In this way, not only can the location of the object being moved be accurately tracked and visualized, but the robot's movement path during task execution can also be monitored and displayed in real time.
[0115] As an alternative approach, after sending a task to the robot to move an object, the object can be displayed on the warehouse map based on the container model corresponding to the object, and the robot's movement trajectory for moving the object can be displayed on the warehouse map. The movement trajectory is consistent with the actual movement trajectory of the object.
[0116] For example, if the object to be moved is a shelf, instead of using a generic shelf icon, this optional method can display the object on the warehouse map using a personalized shelf icon based on the model data of the shelf's container model. Furthermore, the movement trajectory of the robot moving the object can be displayed on the warehouse map. For instance, if a robot icon is displayed on the warehouse map, combining that robot icon with the personalized shelf icon will show the effect of the robot moving the shelf, and the corresponding movement trajectory will be consistent with the actual movement trajectory of the shelf. In this way, not only can the location of the object to be moved be accurately tracked and visualized, but the robot's movement path during task execution can also be monitored and displayed in real time.
[0117] Furthermore, to illustrate the creation process of the container model, such as Figure 3 As shown, the method may include the steps 201 to 202 as follows:
[0118] Step 201: Display the model editing page corresponding to the container type selected by the user.
[0119] Container types can include different shelf types, different box types, different pallet types, etc. Users can choose the shelf type, box type, or pallet type they want to create.
[0120] For example, when a user needs to create a container model, they can click on the corresponding function button on the warehouse system's management page to select the desired container type, and then enter the model editing page corresponding to that container type. The user can then set the parameter data of the container model on the model editing page to create the corresponding container model.
[0121] In some embodiments, the model editing page may include a 3D display of the base model corresponding to the container type, and editing items for the model attributes of the base model, wherein the 3D display can be dynamically changed according to the user's settings for the editing items.
[0122] For example, such as Figure 4 As shown, when a user creates a container model of a shelf, a schematic diagram of the basic shelf model will be displayed on the corresponding model editing page. The dimensions of this basic model can be the initial values for several editable attributes. These attributes may include whether the shelf supports movement, whether it can be assigned to robots, shelf dimensions, shelf leg dimensions, shelf layer settings, and so on. When users adjust these settings, especially those involving appearance features such as dimensions, the corresponding dynamic display will be shown in the schematic diagram, allowing users to clearly identify which parameters of the shelf are being adjusted and avoiding errors. Users can also manually adjust the display angle of the schematic diagram.
[0123] Step 202: Create a container model based on the user's actions on the model editing page.
[0124] For example, users can set various attribute information of the container model on the model editing page. When the create button is clicked, the corresponding container model can be created and saved (or entered).
[0125] In some embodiments, after creating a container model based on the user's operation on the model editing page, the transport site information can be obtained; then, the container model created by the user is verified based on the transport site information; if the container model passes the verification, the container model created by the user is entered into the database; if the container model fails the verification, the container model created by the user is not entered into the database. Only container models that are successfully entered into the database can be used subsequently. Accordingly, step 101 may specifically include: obtaining the container model corresponding to the transport object from the entered models, i.e., obtaining the entered container model, and then executing the processes shown in steps 102 to 103.
[0126] This approach avoids robot handling accidents caused by model errors, verifies feasibility in advance, reduces the issuance of invalid tasks, and improves overall operational efficiency.
[0127] There are several options for validating user-created container models based on transport site information. One option is to obtain site environment data based on the transport site information. This site environment data includes at least the robot's traversable path information, site area information, and obstacle distribution information. Then, the three-dimensional geometric parameters of the created container model are dynamically fused with the site environment data to generate an interactive simulation scenario. A virtual transport test is then performed in the interactive simulation scenario to determine whether the container model meets the transport stability constraints under different postures. These constraints include the center of gravity offset threshold, overturning moment limit, and contact surface slippage risk assessment. If the created container model passes the virtual transport test, the container model is considered valid. If the created container model fails the virtual transport test, the user is prompted to adjust the container model's parameters, and parameter adjustment suggestions are provided based on the results of the virtual transport test.
[0128] Based on the above-mentioned optional methods, this disclosure provides an intelligent verification scheme for container models, applicable to the preliminary preparation stage of robot handling tasks. This scheme constructs a virtual interactive scenario by combining actual environmental information of the handling site with user-created container model data, and conducts simulation tests within it, thereby comprehensively evaluating the portability of the container model.
[0129] For example, firstly, environmental data of the handling site is obtained based on the site information. This environmental data includes at least the path information that the robot can traverse (such as aisle width, turning radius, slope, etc.), site area information (such as shelf distribution, work area division, floor material, etc.), and obstacle distribution information (such as temporary parking equipment, the location of other robots, and fixed structures). Then, the three-dimensional geometric parameters of the user-created container model are dynamically fused with the aforementioned environmental data to generate a virtual simulation scene that includes the interaction between the container and the environment. This container model includes, but is not limited to, handling objects such as boxes, shelves, and pallets, and its three-dimensional geometric parameters at least cover key attributes such as dimensions (length × width × height), weight, center of gravity position, and gripping point coordinates. Then, a virtual handling test is performed in the interactive simulation scene. This test simulates the process of the robot handling the container in different postures to determine whether the container meets the preset handling stability constraints. These constraints include, but are not limited to: a center of gravity offset threshold (the center of gravity displacement of the container during handling must not exceed a set safety range); a tipping moment limit (the tipping moment experienced by the container during tilting, acceleration, or deceleration should be below a critical value); and a contact surface slippage risk assessment (analyzing the friction coefficient and force conditions between the container and the ground and clamps to assess the risk of slippage or detachment). If the container model does not exhibit any stability issues or path conflicts during the virtual handling test, it is considered to have passed the verification, and the container model is officially entered into the warehouse management system for subsequent task scheduling. Conversely, if the container model has potential risks, the user is prompted to adjust the container model's parameters, and parameter adjustment suggestions are provided based on the virtual test results, such as reducing height, adjusting width, or changing the center of gravity distribution, to help the user optimize the model design. Through this example solution, intelligent matching and risk prediction between the container model and the actual handling environment are achieved, improving the safety and success rate of handling tasks.
[0130] In some embodiments, in response to a user's update instruction for the container model, a model editing page corresponding to the container model is displayed; then, the container model is updated according to the user's operation on the model editing page.
[0131] For example, if a user needs to update an entered container model, such as changing some of its attributes, they can input adjustment commands for the container model through the warehouse system's management page. This will take them to the corresponding model editing page, where the pre-set attribute information is stored. Users can adjust these attributes according to their needs and then create the updated container model. This method satisfies users' needs for updating already created container models.
[0132] As an application scenario, when expanding a warehousing system (e.g., adding a new warehouse area), it's necessary to move some shelves, boxes, and other containers to the newly expanded warehouse area. Users can search among the already entered container models to determine if these container models exist. If the required container model is not found among the entered models, new shelf, box, and other container models can be created on the system's model editing page. For example, the shelf type (e.g., standard pallet racking, drive-in racking, etc.) can be selected; the height, width, number of layers, and number of the shelf can be edited; while editing these attributes, the changes in the 3D model can be viewed in real time to ensure that the created model is consistent with the actual physical structure; the created shelf model is then entered into the system, and the user can then call a robot to move the shelf from the operation page. The system can obtain the corresponding container model and the target model data of the container model, and then send a moving task to the robot. The moving task carries the target model data and the start and end points of the moving path. At the start point, the robot identifies the shelf based on the target model data, confirms that it is the object it needs to move, and then carries and moves it to the newly expanded warehouse area. For example, a robot can identify the corresponding object based on target model data (such as size, shape, position, etc.) and perform precise handling operations to move the newly built shelf to the designated coordinate point in the newly expanded area.
[0133] This approach provides standardized and visual modeling and management tools for new project implementation and new regional development. The tools are simple and convenient to use, enhancing the user experience.
[0134] Furthermore, in order to improve the efficiency and accuracy of robots in identifying and handling objects, as Figure 2 An alternative approach to the embodiment, such as Figure 5 As shown, the method includes the steps 301 to 302 as follows:
[0135] Step 301: Determine the mapping relationship between model data of different container models and camera recognition modes.
[0136] Camera recognition mode refers to the camera recognition mode used by the robot when identifying the object being transported. A suitable camera recognition mode can effectively improve the robot's efficiency and accuracy in identifying the object being transported.
[0137] Camera recognition modes may include at least: shape matching-based recognition modes (which use the outline or shape features of objects for recognition and are suitable for containers with unique geometric shapes), color / texture-based recognition modes (which identify specific transported objects by analyzing the color distribution or texture features of the container surface and are suitable for objects that have significant differences in color or texture), depth perception recognition modes (which use a depth camera to obtain three-dimensional information of the target object, which helps to more accurately determine the position, posture and volume of the object and are suitable for recognizing stacked goods), etc.
[0138] For example, different camera recognition modes can be used to identify the transport objects corresponding to different container models. The robot selects the camera recognition mode based on the judgment logic corresponding to the model data, which can quickly perform the identification of transport objects. For example, a suitable recognition algorithm can be selected based on color and / or texture to distinguish transport objects with obvious color differences or unique surface textures; or a suitable recognition algorithm can be selected based on shape and / or contour to identify objects with regular shapes that are easy to identify; or based on depth information, 3D reconstruction can be performed using depth data provided by a depth camera to identify transport objects with complex shapes, etc.
[0139] In some embodiments, determining the mapping relationship between model data of different container models and camera recognition modes may specifically include: firstly, determining camera parameters based on the model data of the container model and a preset working environment, the camera parameters including one or more of focal length, aperture size, ISO, exposure time, and white balance; then, performing test shooting on model samples of the container model according to the camera parameters; and finally, determining the camera recognition mode corresponding to the container model based on the shooting results.
[0140] In some examples, camera parameters are determined based on model data and a preset working environment. Specifically, an appropriate lens focal length can be selected based on the size of the container model and the shooting distance. The aperture value is adjusted to control the depth of field, ensuring that the key features of the container model are within the focal range. A suitable ISO is set according to the ambient lighting conditions to balance image quality and noise levels, and the shutter speed is adjusted to prevent overexposure or underexposure due to excessively long or short exposures. Accurate color reproduction is ensured by calibrating white balance settings, especially under conditions with multiple light sources or complex lighting. Based on machine learning or deep learning methods, sample datasets can be collected to train the model, enabling it to distinguish different container shapes. Images of the container are taken from multiple angles according to the determined camera parameters, acquiring images of the container under different conditions, covering all important features, and detailed annotations are performed. Image processing software or deep learning models are used to automatically evaluate the quality of each image, checking for blur, distortion, overexposure, and other issues. Key features are extracted from qualified test images to verify whether the model can effectively distinguish different types of container models. The model is continuously iterated and optimized based on problems discovered during the verification process until it can reliably help the robot identify container contents. If certain parameter settings are found to cause image quality problems, they need to be adjusted accordingly, and the test should be repeated until a satisfactory result is obtained.
[0141] For example, after test shooting, considering factors such as image quality and feature extraction success rate, check whether the image quality meets the requirements, especially in terms of edge sharpness and contrast. Evaluate the effectiveness of the current camera parameter settings, and select an optimal combination of camera parameters based on the test results to ensure stable recognition of various container models in practical applications. For specific types of container models, select the most suitable edge detection algorithm based on the characteristics of the container and the test results. Commonly used algorithms include Canny edge detection, Sobel operator, Laplacian operator, and deep learning algorithms; for more complex scenarios, deep learning methods can also be considered to improve recognition accuracy and speed. Selecting the most suitable recognition algorithm and parameter combination can ensure that the system can operate stably in real-world environments.
[0142] In some embodiments, determining the camera recognition mode corresponding to the container model based on the shooting results may specifically include: performing image quality analysis on the images of the model samples taken under different camera parameter conditions; obtaining the target camera parameters used for the target shooting image whose image quality is greater than a preset threshold; and determining the camera recognition mode corresponding to the container model based on the target camera parameters.
[0143] In some examples, to ensure the robot's accuracy and stability in recognizing the objects being handled, reasonable image quality thresholds (such as sharpness > 70 points, color accuracy < 5ΔE, etc.) can be set according to industry standards or actual needs. If the initial assessment finds the thresholds to be too high or too low, appropriate adjustments can be made based on the actual situation. Based on the above evaluation indicators, target images that meet the preset thresholds are selected from all captured images, and the camera parameters used for each target image are recorded; these are the target camera parameters.
[0144] For example, common features (such as specific focal lengths and aperture combinations) can be extracted from the selected target camera parameters as the basis for optimization. Additional tests can also be conducted on the selected target camera parameter combinations to ensure they maintain good image quality under different sample and environmental conditions. Based on the test results, an optimal set of camera parameter combinations is selected and combined with a camera recognition algorithm to form the optimal camera recognition mode for the current sample, ensuring stable recognition of various container models in practical applications.
[0145] In some embodiments, determining the camera recognition mode corresponding to the container model based on the target camera parameters includes: preprocessing the target captured image; recognizing the preprocessed target captured image using different visual algorithms; obtaining the target visual algorithm whose recognition result meets the requirements; and obtaining the camera recognition mode corresponding to the container model according to the target visual algorithm and the target camera parameters.
[0146] For example, edge and other shape features of the preprocessed target image can be identified using Canny edge detection, Sobel operator, Laplacian operator, or deep learning methods. Key parameters (such as threshold and kernel size) of each algorithm can be adjusted to obtain the best results. Then, quantitative indicators such as intersection-over-union ratio and F1 score are used to evaluate the detection results of each algorithm. Combining the recognition accuracy requirements of the transported object, the algorithm that can stably detect all key features or has a high overall computational efficiency is selected as the target vision algorithm. Then, the camera recognition mode corresponding to the container model is determined by combining the target vision algorithm and the target camera parameters.
[0147] Step 302: Send the mapping relationship between the model data of different container models and the camera recognition mode to the robot.
[0148] This mapping relationship can be used by the robot to determine the camera recognition mode used when identifying the object being transported.
[0149] In some examples, once the camera recognition mode corresponding to a certain container model is confirmed to be effective, it can be integrated into the RMS system, making it part of the optional task instructions. With the accumulation of more data and the development of technology, the recognition modes of various container types are continuously monitored and updated to maintain the system's advanced nature and adaptability. The optimal camera recognition mode is selected for each container type, and a mapping table is maintained in the database, recording the data of each container model and its corresponding suitable camera recognition mode (including camera parameters and vision algorithm configuration). Special modes can also be saved for different colored container types. The mapping relationship between the model data of different container models and their camera recognition modes is sent to the robot. The robot can directly determine the camera recognition mode based on the model data issued by the RMS, greatly improving the robot's recognition accuracy and work efficiency in actual operation.
[0150] By applying the technical solution of this disclosure embodiment, the mapping relationship between model data of different container models and camera recognition modes can be sent to the robot, so that the robot can determine the camera recognition mode used when recognizing the transported object, and then use the appropriate camera recognition mode to recognize the transported object, thereby effectively improving the robot's recognition efficiency and accuracy of transported objects.
[0151] Furthermore, to illustrate the specific execution process on the robot side, Figure 6 A flowchart illustrating a robot control method according to an embodiment of the present disclosure is shown. Applied to robot-side execution, it may include steps 401 to 403.
[0152] Step 401: The robot receives the task of transporting the object.
[0153] The transport task carries target model data for the container model corresponding to the transported object. This container model can be pre-created by the user in the model editing page, which includes editable items for model attributes. The target model data can be obtained based on the user's settings for these editable items. For a detailed explanation, please refer to [link to relevant documentation]. Figures 1 to 5 The details of the embodiments shown will not be repeated here.
[0154] For example, a transport task can be a task of moving an object. After receiving a transport task, the robot performs the task of moving the object from position 1 to position 2.
[0155] Step 402: The robot identifies the object to be transported based on the target model data.
[0156] Step 403: The robot performs the task of moving the object based on the recognition results.
[0157] For example, if the object to be moved is a shelf, the robot can determine its shape, specifications, weight, and other information based on target model data. Then, using sensors (such as cameras, LiDAR, and ultrasonic sensors) on its equipment, the robot scans a designated area (carried during the transport task to indicate the shelf's location) to obtain an actual image of the object. Then, using computer vision technology and machine learning algorithms, the robot compares the scanned data with the previously received target model data. This includes extracting key features from the image and using edge detection, shape matching, and other techniques to identify the object's shape. Based on a pre-trained model (such as a convolutional neural network, CNN), the robot identifies and classifies the object, thus recognizing the shelf. Based on this information, the robot formulates a specific transport plan and begins executing the transport task. This includes moving to the target location, adjusting its posture to the optimal gripping angle, performing the gripping action, and transporting the goods to the designated location to release them—a series of operations.
[0158] In some embodiments, the robot may pre-receive the mapping relationship between model data of different container models and camera recognition modes; accordingly, step 402 may specifically include: determining the camera recognition mode corresponding to the target model data by querying the mapping relationship; and then using the camera recognition mode corresponding to the target model data to identify the transported object. This allows the use of a suitable camera recognition mode to identify the transported object, thereby effectively improving the robot's efficiency and accuracy in identifying the transported object. In some examples, model data is used by the robot to determine the camera recognition mode used when identifying the transported object. Different camera recognition modes can be used to identify different container models. By selecting the camera recognition mode according to the judgment logic corresponding to the model identifier, the identification of the transported object can be performed quickly. When the robot receives a task instruction, it acquires the camera recognition mode for the target container data, which includes pre-adjusted camera hardware parameters (such as focal length, aperture, ISO, etc.), and applies these parameters to its built-in or external camera system to ensure optimized image quality when photographing the transported object.
[0159] For example, the camera recognition patterns corresponding to container models are integrated and made part of the optional task instructions. With the accumulation of more data and the development of technology, the recognition patterns of various container types are continuously monitored and updated to maintain the system's advanced nature and adaptability. Suitable camera recognition patterns are created for each container type, and a mapping table is maintained in the database, recording the data of each container model and its corresponding suitable camera recognition pattern (including camera parameters and vision algorithm configuration). Special patterns can also be saved for different colored containers. The robot receives the mapping relationship between the model data of different container models and the camera recognition patterns. The robot can directly determine the camera recognition pattern from the model data issued by the system, greatly improving the robot's recognition accuracy and work efficiency in actual operation.
[0160] In some examples, the camera recognition mode corresponding to the target model data is used to identify the transported object. This includes: determining camera parameters and a vision algorithm based on the camera recognition mode corresponding to the target model data; then identifying the transported object based on the camera parameters and the vision algorithm. Accordingly, step 403 includes: determining the position information and edge information of the transported object based on the identification result; and then performing the transport task based on the position information and edge information. For example, the robot formulates a specific transport plan based on the identified position information and edge information of the transported object and begins to execute the transport task.
[0161] Optionally, the above-mentioned identification of the transported object based on camera parameters and vision algorithms includes: taking pictures of the transported object using a robot camera device according to the camera parameters; then identifying the captured image according to the vision algorithm to obtain a target image with the outline position marked on the transported object; and combining the attribute features corresponding to the target model data to identify the position information and edge information of the transported object in the target image.
[0162] For example, object recognition based on camera parameters and visual algorithms involves adjusting the depth camera settings according to the determined camera parameters, loading the specified visual algorithm and its parameter configuration, and capturing images of the object according to the configured parameters. The robot then identifies the object and performs edge detection based on the camera parameters and visual algorithm configured in the camera recognition mode, extracting key features. The robot performs edge detection based on the selected visual algorithm, using image processing techniques or deep learning models. After algorithmic processing, a new image marked with potential edge locations (contour locations) is obtained, i.e., the target image. Meaningful information, such as line segments and curves, is extracted from the target image. This information helps determine the specific contour of the object being transported. Then, using a pre-built container model, an attempt is made to match the edge features in the image with the expected container shape. If there are any uncertainties or the initial results are inaccurate, the container model can be checked and corrected based on known container dimensions, relative positions, and other information. In this way, accurate object recognition can be achieved.
[0163] Furthermore, in some embodiments, during the process of identifying the transported object using a camera recognition mode determined based on the target model data, if identification fails after a preset number of attempts, a different camera recognition mode is used to identify the transported object. Then, the target camera recognition mode used when the transported object was successfully identified is sent to update the mapping relationship between the container model's model data and the camera recognition mode. In this way, the mapping relationship between the container model's model data and the camera recognition mode can be updated in a timely manner, facilitating the robot to use the updated mapping relationship and obtain a more suitable camera recognition mode when performing other transport tasks.
[0164] By applying the solution of this disclosure embodiment, the robot can quickly and effectively identify the object to be transported based on the container model data carried in the transport task, and then perform the corresponding transport task. The container model can be created by the user in advance in the model editing page. For transport objects of different shapes, the user can create corresponding container models for each object in advance in the model editing page. This allows the user to instruct the robot on the corresponding model data when issuing transport tasks, enabling the robot to effectively identify transport objects of different shapes and improving the robot's transport efficiency and accuracy.
[0165] Furthermore, as Figures 1 to 6 The specific implementation of the example shown in this disclosure provides a robot control device that can be applied to a warehouse management server, such as... Figure 7 As shown, the device includes a processing module 501 and a transmitting module 502.
[0166] Processing module 501 is configured to obtain the container model corresponding to the transported object. The container model is created by the user in advance in the model editing page, which includes editing items for model attributes. Based on the user's settings for the editing items, the target model data of the container model is obtained.
[0167] The sending module 502 is configured to send a handling task of a transported object to the robot. The handling task carries target model data, which is used by the robot to identify the transported object.
[0168] In some embodiments of this disclosure, the processing module 501 is specifically configured to obtain the attribute features of the container model based on the user's settings for the edit items; and determine the attribute features as target model data.
[0169] In some embodiments of this disclosure, the processing module 501 is further configured to display a model editing page corresponding to the container type selected by the user. The model editing page includes a 3D display of the basic model corresponding to the container type and editing items for the model attributes of the basic model. The 3D display dynamically changes according to the user's settings for the editing items. The container model is created based on the user's operations on the model editing page.
[0170] In some embodiments of this disclosure, the processing module 501 is further configured to acquire transport site information; verify the container model created by the user based on the transport site information; if the container model passes the verification, then input the container model created by the user; correspondingly, the processing module 501 is also configured to acquire the input container model.
[0171] In some embodiments of this disclosure, the processing module 501 is further configured to acquire site environment data based on the transport site information, the site environment data including at least the path information that the robot can pass through, site area information, and obstacle distribution information; dynamically fuse the three-dimensional geometric parameters of the container model with the site environment data to generate an interactive simulation scene; perform a virtual transport test in the interactive simulation scene to determine whether the container model meets the transport stability constraints under different postures, the constraints including the center of gravity offset threshold, overturning moment limit, and contact surface sliding risk assessment; if the container model passes the virtual transport test, it is determined that the container model has passed the verification; if the container model fails the virtual transport test, the user is prompted to adjust the parameter information of the container model, and parameter adjustment suggestions are provided based on the results of the virtual transport test.
[0172] In some embodiments of this disclosure, the processing module 501 is further configured to, in response to a user's update instruction for the container model, display the model editing page corresponding to the container model; and update the container model according to the user's operation on the model editing page.
[0173] In some embodiments of this disclosure, the processing module 501 is further configured to display the transported object in the warehouse map based on the container model, and to display the motion trajectory of the robot transporting the transported object in the warehouse map, wherein the motion trajectory is consistent with the actual transport trajectory of the transported object.
[0174] In some embodiments of this disclosure, the processing module 501 is further configured to determine the mapping relationship between model data of different container models and camera recognition modes; the sending module 502 is further configured to send the mapping relationship to the robot, the mapping relationship being used by the robot to determine the camera recognition mode used when recognizing the transported object.
[0175] In some embodiments of this disclosure, the processing module 501 is specifically configured to determine camera parameters based on the model data of the container model and a preset working environment. The camera parameters include one or more of focal length, aperture size, ISO, exposure time, and white balance. The module then takes a picture of a model sample of the container model based on the camera parameters and determines the camera recognition mode corresponding to the container model based on the shooting results.
[0176] In some embodiments of this disclosure, the processing module 501 is specifically configured to perform image quality analysis on the images of the model samples taken under different camera parameter conditions; obtain the target camera parameters used for the target captured image whose image quality is greater than a preset threshold; and determine the camera recognition mode corresponding to the container model based on the target camera parameters.
[0177] In some embodiments of this disclosure, the processing module 501 is specifically configured to preprocess the target captured image; identify the preprocessed target captured image using different visual algorithms; obtain the target visual algorithm whose identification result meets the requirements; and obtain the camera recognition mode corresponding to the container model according to the target visual algorithm and the target camera parameters.
[0178] It should be noted that other corresponding descriptions of the functional units involved in the robot control device provided in this disclosure embodiment can be found by referring to... Figures 1 to 6 The corresponding description in [the document] will not be repeated here.
[0179] Furthermore, as Figures 1 to 6 The specific implementation of the example shown in this disclosure provides a robot control device that can be applied to robots, such as... Figure 8 As shown, the device includes a receiving module 601 and an execution module 602.
[0180] The receiving module 601 is configured to receive the transport task of the transport object. The transport task carries the target model data of the container model corresponding to the transport object. The container model is created by the user in advance in the model editing page. The model editing page includes the editing items of the model attributes. The target model data is obtained based on the user's setting information of the editing items.
[0181] The execution module 602 is configured to identify the transported object based on the target model data and execute the transport task based on the identification result.
[0182] In some embodiments of this disclosure, the receiving module 601 is further configured to receive the mapping relationship between model data of different container models and camera recognition modes; the execution module 602 is specifically configured to determine the camera recognition mode corresponding to the target model data by querying the mapping relationship; and to identify the transported object using the camera recognition mode corresponding to the target model data.
[0183] In some embodiments of this disclosure, the execution module 602 is specifically configured to determine camera parameters and a visual algorithm based on the camera recognition mode corresponding to the target model data; to identify the transported object based on the camera parameters and the visual algorithm; and correspondingly, the execution module 602 is further configured to determine the position information and edge information of the transported object based on the recognition result; and to perform the transport task of the transported object based on the position information and edge information.
[0184] In some embodiments of this disclosure, the execution module 602 is specifically configured to use a robot camera device to take pictures of the transported object according to camera parameters; to identify the captured image according to a visual algorithm to obtain a target image with the outline position marked on the transported object; and to identify the position information and edge information of the transported object in the target image by combining the attribute features corresponding to the target model data.
[0185] In some embodiments of this disclosure, the execution module 602 is further configured to, during the process of identifying the transported object using a camera recognition mode determined based on the target model data, if the identification fails after a preset number of attempts, use a different camera recognition mode to identify the transported object; and send the target camera recognition mode used when the identification of the transported object is successful, so as to update the mapping relationship between the model data of the container model and the camera recognition mode.
[0186] Based on the above, Figures 1 to 6 As illustrated in the example, correspondingly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the above-described... Figures 1 to 6 The example method shown.
[0187] Based on the above, Figures 1 to 6As illustrated, correspondingly, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described... Figures 1 to 6 The example method shown.
[0188] Based on this understanding, the technical solution of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods of various implementation scenarios of this disclosure.
[0189] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0190] Figure 9 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0191] like Figure 9 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in ROM (Read-Only Memory) 1002 or loaded from storage unit 1008 into RAM (Random Access Memory) 1003. RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. I / O (Input / Output) interface 1005 is also connected to bus 1004.
[0192] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0193] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the methods mentioned earlier. For example, in some embodiments, the methods mentioned earlier can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the aforementioned method by any other suitable means (e.g., by means of firmware).
[0194] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0195] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0196] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0197] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0198] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.
[0199] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.
[0200] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.
[0201] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0202] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A robot control method, characterized in that, include: Display the model editing page corresponding to the container type selected by the user, and create the container model according to the user's operations on the model editing page; The validation of a user-created container model based on transport site information includes: acquiring site environment data based on the transport site information, the site environment data including at least robot-accessible path information, site area information, and obstacle distribution information; dynamically fusing the three-dimensional geometric parameters of the container model with the site environment data to generate an interactive simulation scenario; performing a virtual transport test in the interactive simulation scenario to determine whether the container model meets the transport stability constraints under different postures, the constraints including a center of gravity offset threshold, overturning moment limit, and contact surface sliding risk assessment; if the container model passes the virtual transport test, the container model is determined to have passed the validation. If the user-created container model passes the verification, the user-created container model will be entered into the database. Obtain the container model corresponding to the transport object. The container model is created and entered by the user in advance in the model editing page. The model editing page includes editing items for model attributes. Based on the user's settings for the edited items, obtain the target model data of the container model; The robot is sent a task to move the object being moved, the task carrying the target model data, which is used by the robot to identify the object being moved.
2. The method according to claim 1, characterized in that, The step of obtaining the target model data of the container model based on the user's settings for the edited items includes: Based on the user's settings for the edited items, the attribute characteristics of the container model are obtained; The attribute features are determined as the target model data.
3. The method according to claim 1, characterized in that, The model editing page includes a 3D display of the basic model corresponding to the container type, as well as editing items for the model attributes of the basic model. The 3D display dynamically changes based on the user's settings for the editing items.
4. The method according to claim 1, characterized in that, The method further includes: If the container model created by the user fails the virtual transport test, the user will be prompted to adjust the parameters of the container model, and parameter adjustment suggestions will be provided based on the results of the virtual transport test.
5. The method according to claim 1, characterized in that, The method further includes: In response to a user's update command for the container model, the model editing page corresponding to the container model is displayed; The container model is updated based on the user's actions on the model editing page.
6. The method according to claim 1, characterized in that, After sending the transport task of the object to be transported to the robot, the method further includes: The object to be transported is displayed on the warehouse map based on the container model, and the movement trajectory of the robot transporting the object is displayed on the warehouse map. The movement trajectory is consistent with the actual transport trajectory of the object.
7. The method according to claim 1, characterized in that, The method further includes: Determine the mapping relationship between model data of different container models and camera recognition patterns; The mapping relationship is sent to the robot, and the mapping relationship is used by the robot to determine the camera recognition mode to be used when recognizing the transported object.
8. The method according to claim 7, characterized in that, Determining the mapping relationship between model data of different container models and the camera recognition pattern includes: The camera parameters are determined based on the model data of the container model and the preset working environment. The camera parameters include one or more of the following: focal length, aperture size, ISO, exposure time, and white balance. The model sample of the container model is photographed according to the camera parameters; The camera recognition mode corresponding to the container model is determined based on the shooting results.
9. The method according to claim 8, characterized in that, The step of determining the camera recognition mode corresponding to the container model based on the shooting results includes: Image quality analysis was performed on the images of the model samples taken under different camera parameter conditions. Obtain the target camera parameters corresponding to the target captured image whose image quality is greater than a preset threshold; The camera recognition mode corresponding to the container model is determined based on the target camera parameters.
10. The method according to claim 9, characterized in that, Determining the camera recognition mode corresponding to the container model based on the target camera parameters includes: The target image is preprocessed; The pre-processed target images are then identified using different visual algorithms. A visual algorithm that obtains target recognition results that meet the requirements; According to the target vision algorithm and the target camera parameters, the camera recognition mode corresponding to the container model is obtained.
11. A robot control method, characterized in that, include: The system receives a transport task for a transport object as described in claim 1. The transport task carries target model data of a container model corresponding to the transport object. The container model is created in advance by the user in a model editing page. The model editing page includes editing items for model attributes. The target model data is obtained based on the user's settings for the editing items. The transport object is identified based on the target model data; The transport task is executed based on the identification results.
12. The method according to claim 11, characterized in that, Before identifying the transported object based on the target model data, the method further includes: Receive the mapping relationship between model data of different container models and camera recognition patterns; The step of identifying the transported object based on the target model data includes: The camera recognition mode corresponding to the target model data is determined by querying the mapping relationship. The object being transported is identified using the camera recognition mode corresponding to the target model data.
13. The method according to claim 12, characterized in that, The step of using the camera recognition mode corresponding to the target model data to identify the transported object includes: Based on the camera recognition mode corresponding to the target model data, determine the camera parameters and vision algorithm; The object being transported is identified based on the camera parameters and the vision algorithm. The step of performing the transport task based on the identification result includes: The location and edge information of the transported object are determined based on the recognition results; The transport task of the transported object is executed based on the location information and edge information.
14. The method according to claim 13, characterized in that, The process of identifying the transported object based on the camera parameters and the visual algorithm includes: The object being transported is photographed using a robotic camera device according to the camera parameters. The visual algorithm is used to identify the captured image to obtain a target image with the outline of the transported object marked. The position and edge information of the transported object are then identified in the target image by combining the attribute features corresponding to the target model data.
15. The method according to claim 12, characterized in that, The method further includes: If the object being transported fails to be identified after a preset number of attempts using the camera recognition mode determined based on the target model data, then a different camera recognition mode will be used to identify the object being transported. Send the target camera recognition mode used when the object being transported is successfully identified, in order to update the mapping relationship between the model data of the container model and the camera recognition mode.
16. A robot control device, characterized in that, include: The processing module is configured to display the model editing page corresponding to the type of container selected by the user, and to create the container model based on the user's operations on the model editing page. The processing module is further configured to verify the user-created container model based on the handling site information, including: acquiring site environment data based on the handling site information, the site environment data including at least robot-accessible path information, site area information, and obstacle distribution information; dynamically fusing the three-dimensional geometric parameters of the container model with the site environment data to generate an interactive simulation scenario; performing a virtual handling test in the interactive simulation scenario to determine whether the container model meets the handling stability constraints under different postures, the constraints including a center of gravity offset threshold, overturning moment limit, and contact surface sliding risk assessment; if the container model passes the virtual handling test, the container model is determined to have passed the verification. The processing module is also configured to input the user-created container model if the user-created container model passes the verification. The processing module is further configured to obtain the container model corresponding to the transport object, wherein the container model is created and entered by the user in advance in the model editing page, and the model editing page includes editing items for model attributes; and to obtain the target model data of the container model based on the user's settings for the editing items. The sending module is configured to send a handling task of the object to the robot. The handling task carries the target model data, which is used by the robot to identify the object.
17. A robot control device, characterized in that, include: The receiving module is configured to receive a transport task of a transport object in the method as described in claim 1, wherein the transport task carries target model data of a container model corresponding to the transport object, the container model is created in advance by the user in a model editing page, the model editing page includes editing items for model attributes, and the target model data is obtained based on the user's settings for the editing items; The execution module is configured to identify the transported object based on the target model data; The transport task is executed based on the identification results.
18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 15.
19. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 15.
20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 15.
Citation Information
Patent Citations
Robot automatic control method, device and equipment and storage medium
CN119717630A
Robot control method and device, storage medium and electronic equipment
CN120044846A
