Robot control method and device, electronic equipment and storage medium

Abstract

The invention provides a robot control method and device, electronic equipment and a storage medium. Comprising the following steps: in response to a control instruction for a robot, obtaining a local Voronoi diagram of a current position of the robot, a panoramic map of the current position and a pre-constructed local semantic map of the current position; based on the local Voronoi map and the panoramic map, determining a feasible region of the robot at the current position and a region description of the feasible region; determining a feasible path of the robot at the current position and path description of the feasible path based on the local semantic map and the region description; determining a path score of the feasible path based on the control instruction, the path description and the local Voronoi diagram; and determining a target navigation point of the robot based on the path score, and controlling the robot to move to the target navigation point. The intelligent degree and the control efficiency of the robot can be improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot control method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, robotics technology has become mature, and there are more and more types of robots, including industrial robots, household robots, military robots, etc.; all walks of life are constantly developing robots suitable for their industries, hoping that robots can replace humans to complete repetitive, tedious, and dangerous operations; however, during the movement of the robot, it is necessary to accurately determine the robot's navigation point in order to achieve control of the robot.

[0003] In the related art, the navigation points of the robot are usually set by the user with specific coordinates, or the robot is required to scan the entire environment in advance and build a map of the entire environment to determine the navigation points and movement paths of the robot; however, the robot pre-builds a map of the entire environment with a low degree of intelligence, which reduces the control efficiency of the robot. Summary of the invention

[0004] The embodiments of the present application provide a robot control method, device, electronic device and storage medium, which improve the intelligence level and control efficiency of the robot by obtaining a local map of the robot's current position and determining the robot's target navigation point.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] An embodiment of the present application provides a robot control method, the method comprising: in response to a control instruction for the robot, obtaining a local Voronoi map of the current position of the robot, a panoramic map of the current position, and a pre-constructed local semantic map of the current position; based on the local Voronoi map and the panoramic map, determining a feasible area of ​​the robot at the current position and an area description of the feasible area; based on the local semantic map and the area description, determining a feasible path of the robot at the current position and a path description of the feasible path; based on the control instruction, the path description, and the local Voronoi map, determining a path score of the feasible path; based on the path score, determining a target navigation point of the robot, and controlling the robot to move to the target navigation point.

[0007] An embodiment of the present application provides a robot control device, including: an acquisition module, which is used to acquire a local Voronoi map of the current position of the robot, a panoramic map of the current position, and a pre-constructed local semantic map of the current position in response to a control instruction for the robot; a first determination module, which is used to determine a feasible area of ​​the robot at the current position and an area description of the feasible area based on the local Voronoi map and the panoramic map; a second determination module, which is used to determine a feasible path of the robot at the current position and a path description of the feasible path based on the local semantic map and the area description; a third determination module, which is used to determine a path score of the feasible path based on the control instruction, the path description, and the local Voronoi map; and a control module, which is used to determine a target navigation point of the robot based on the path score, and control the robot to move to the target navigation point.

[0008] In the above scheme, the first determination module is also used to: construct a coordinate system with the current position of the robot as the coordinate origin; based on the coordinate system, perform map matching on the local Voronoi map and the panoramic map to obtain a feasible area of ​​the robot at the current position; and generate an area description of the feasible area through a pre-constructed first language model.

[0009] In the above scheme, the first determination module is also used to: determine the coordinate values ​​of each point in the panoramic map and the local Voronoi map based on the coordinate system; perform point-by-point matching of the panoramic map and the local Voronoi map based on the coordinate values ​​to obtain a matching area between the panoramic map and the local Voronoi map; and determine a feasible area of ​​the robot at its current position from the matching area.

[0010] In the above scheme, the local Voronoi map includes multiple paths; the second determination module is also used to: determine the first description text of each path in the local Voronoi map through the local semantic map; the first description text includes the objects included in each path and the coordinate values ​​of each of the objects; perform text splicing on the first description text and the area description to obtain a first prompt word; and determine the feasible path of the robot at the current position from the multiple paths based on the first prompt word through a pre-constructed second language model, as well as the path description of the feasible path.

[0011] In the above scheme, the third determination module is also used to: perform text conversion on the control instruction to obtain a second description text; determine a second prompt word based on the path description and the second description text; determine the text similarity between the path description and the second description text based on the second prompt word through a pre-constructed third language model; and determine the path score of the feasible path based on the local Voronoi diagram and the text similarity.

[0012] In the above scheme, the feasible path includes N nodes; the third determination module is also used to: determine the access status of each node in the feasible path based on the local Voronoi diagram; the access status includes nodes that have been visited and nodes that have not been visited; score the corresponding nodes based on the access status of each node to obtain the score value of each node; perform weighted summation on the score values ​​of all nodes in the feasible path to obtain a comprehensive score of the feasible path; and determine the product of the text similarity and the comprehensive score as the path score of the feasible path.

[0013] In the above solution, the third determination module is further used to: determine the feasible path with the highest path score as the target path; and determine the node in the target path that is closest to the current location as the target navigation point.

[0014] In the above scheme, the device also includes an updating module, which is used to: when the robot moves to the target navigation point, update the access status of the target navigation point; based on the updated access status of the target navigation point, update the local Voronoi diagram.

[0015] In the above scheme, the acquisition module is also used to: obtain the position data of the robot's current position and the environmental perception data of the current position; based on the position data and the environmental perception data, determine the local grid map of the robot's current position through a grid map generation tool; and convert the local grid map into a local Voronoi map of the robot's current position.

[0016] In the above scheme, the acquisition module is also used to: preprocess the local grid map to obtain a preprocessed grid map; skeletonize the preprocessed grid map to obtain an initial Voronoi diagram; determine the coordinate value of each path intersection in the initial Voronoi diagram, as well as the coordinate value of the robot's current position; and determine the local Voronoi diagram of the robot's current position based on the initial Voronoi diagram, the coordinate values ​​of the path intersections, and the coordinate values ​​of the current position.

[0017] In the above scheme, the acquisition module is also used to: acquire scene images collected after the robot is rotated at different rotation angles at its current position; perform image stitching on all scene images according to the rotation angle corresponding to each scene image through a preset image stitching algorithm to obtain a stitched image; perform image fusion on the image overlapping areas in the stitched image to obtain a panoramic map of the robot's current position.

[0018] An embodiment of the present application provides an electronic device, comprising: a memory for storing computer executable instructions; and a processor for implementing the robot control method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0019] An embodiment of the present application provides a computer-readable storage medium storing a computer program or executable instructions. When the computer program or computer executable instructions are executed by a processor, the robot control method provided in the embodiment of the present application is implemented.

[0020] An embodiment of the present application provides a computer program product, which includes computer executable instructions, and the computer executable instructions are stored in a computer-readable storage medium; wherein, when a processor of an electronic device reads the computer executable instructions from the computer-readable storage medium and executes the computer executable instructions, the robot control method provided in the embodiment of the present application is implemented.

[0021] The embodiments of the present application have the following beneficial effects:

[0022] When controlling the robot, by obtaining the local Voronoi map, panoramic map and local semantic map of the robot's current position, the robot can accurately understand the robot's surrounding environment, and the accuracy of the robot's path planning is improved through multi-dimensional information fusion; moreover, by obtaining the local Voronoi map and panoramic map, the robot can determine the feasible area in the current environment, so that the robot can respond quickly when performing tasks, avoid invalid paths, and improve navigation efficiency; and, by calculating the path score of each path, the robot can select the best walking path, thereby reducing energy consumption and improving the efficiency of task completion; in addition, the present application only uses the local map of the robot's current position to determine the robot's target navigation point, and there is no need for the robot to pre-scan the entire environment, thereby improving the robot's intelligence and control efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is an optional flowchart of the robot control method provided in the embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of a process of obtaining a local Voronoi diagram of a robot's current position provided in an embodiment of the present application;

[0025] Figure 3 It is a flowchart of determining a feasible area of ​​a robot at a current position and describing an area of ​​the feasible area provided by an embodiment of the present application;

[0026] Figure 4 It is a flowchart of determining a feasible path for a robot at a current location based on a local semantic map and a regional description and a path description of the feasible path provided by an embodiment of the present application;

[0027] Figure 5 is a system architecture diagram of a robot control method provided in an embodiment of the present application;

[0028] Figure 6 is a structural block diagram of a robot control device provided in an embodiment of the present application;

[0029] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0033] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0034] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0035] When determining the target navigation point of the robot, first, in response to the control instruction for the robot, a local Voronoi map of the current position of the robot, a panoramic map of the current position and a pre-built local semantic map of the current position are obtained; then, based on the local Voronoi map and the panoramic map, a feasible area of ​​the robot at the current position and a regional description of the feasible area are determined; based on the local semantic map and the regional description, a feasible path of the robot at the current position and a path description of the feasible path are determined; based on the control instruction, the path description and the local Voronoi map, a path score of the feasible path is determined; based on the path score, the target navigation point of the robot is determined, and the robot is controlled to move to the target navigation point. In this way, by obtaining the local Voronoi map, panoramic map and local semantic map of the robot's current position, the robot can accurately understand the robot's surrounding environment, and the accuracy of the robot's path planning is improved through multi-dimensional information fusion; moreover, by obtaining the local Voronoi map and panoramic map, the robot can determine the feasible area in the current environment, so that the robot can respond quickly when performing tasks, avoid invalid paths, and improve navigation efficiency; and, by calculating the path score of each path, the robot can select the best walking path, thereby reducing energy consumption and improving the efficiency of task completion; in addition, the present application only uses the local map of the robot's current position to determine the robot's target navigation point, and there is no need for the robot to pre-scan the entire environment, thereby improving the robot's intelligence and control efficiency.

[0036] The robot control method provided in the embodiments of the present application can be applied to electronic devices such as robots, laptop computers, tablet computers, desktop computers, smart home appliances, and smart vehicle-mounted devices. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0037] The robot control method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0038] Figure 1 1 is an optional flow chart of a robot control method provided in an embodiment of the present application, and the method can be applied to electronic devices. The following will be described by taking the electronic device as a robot as an example. Figure 1 As shown, the method includes the following steps S101 to S105:

[0039] Step S101, in response to a control instruction for the robot, obtain a local Voronoi map of the robot's current position, a panoramic map of the current position, and a pre-constructed local semantic map of the current position.

[0040] The control instructions of a robot refer to the commands or signals used to guide the behavior and actions of the robot. The control instructions of the robot can come from user input, including voice information or text description information input by the user; they can also come from preset programs or intelligent algorithms; the control instructions of the robot usually include motion instructions (such as forward, backward, turning, etc.), task instructions (such as grasping, grasping animals, cleaning, taking pictures, etc.), navigation instructions (such as navigating to specific coordinates, avoiding obstacles, etc.) and status instructions (such as battery power, sensor status, task completion status, etc.), etc. The specific control instructions are not limited in this application.

[0041] A Voronoi map is a set of continuous polygonal structures composed of perpendicular bisectors connecting two adjacent straight lines. The Voronoi map divides the plane into multiple polygonal regions, each of which contains a generator. The distance from any point in the region to the generator is shorter than the distance to other generators. A local Voronoi map refers to a Voronoi map constructed within a limited range of the robot's current position. A panoramic map of the current position refers to a panoramic image generated by wide-angle photography or image stitching. A semantic map refers to a map generated on the basis of a traditional map by adding semantic information about objects in the environment, such as the category, location, and attributes of the objects. A local semantic map refers to a semantic map constructed within a limited range of the robot's current position.

[0042] In some embodiments, see Figure 2 , Figure 2 is a schematic diagram of a process for obtaining a local Voronoi diagram of the current position of a robot provided in an embodiment of the present application, Figure 2 It is shown that in step S101, the local Voronoi diagram of the current position of the robot is obtained, which can be achieved by following the steps S1011 to S1013:

[0043] Step S1011, obtaining the position data of the robot's current position and the environmental perception data of the robot's current position.

[0044] Position data refers to the coordinate information of the robot's current position, which is obtained through the positioning system (Global Positioning System (GPS), inertial navigation system, etc.) installed inside the robot. Environmental perception data refers to the information about the robot's surrounding environment obtained by the robot through sensors. Environmental perception data may include obstacle information (such as the location, size, shape, etc. of obstacles), environmental characteristics (including terrain (such as flat or sloped), and static elements (such as walls or doors)), etc. The specific environmental perception data is not limited in this application; environmental perception data is obtained through various sensors installed inside the robot. The specific process may include: emitting a laser beam through a laser radar and measuring the time it takes to reflect back to generate point cloud data of the environment; or measuring distance through an ultrasonic sensor to detect obstacles at close range.

[0045] Step S1012, based on the position data and the environment perception data, a local grid map of the current position of the robot is determined by a grid map generation tool.

[0046] A raster map is a grid-based map representation method, where each grid represents a small area in the environment and contains status information about the area, such as the occupancy status of the area; a raster map generation tool refers to an algorithm or software tool used to create a raster map. The raster map generation tool uses data collected by sensors to divide the environment into grids and assign status information to each grid.

[0047] In some embodiments, the position data and environmental perception data of the robot's current position are obtained, and then the obtained data is processed, including filtering, denoising, etc.; then, according to a preset map resolution and map size, the robot's current environment is divided into uniform grids, and the state information of each grid is judged according to the perception data. The state information may be whether the grid is occupied (i.e., whether there is an object); finally, according to the divided grids and the state information of each grid, a local grid map is constructed through a grid map generation tool.

[0048] Step S1013, converting the local grid map into a local Voronoi map of the current position of the robot.

[0049] By processing the constructed local grid map, a local Voronoi diagram is generated that can reflect the spatial relationship around the robot's current location, thereby helping the robot to better understand the environment around its current location.

[0050] In some embodiments, step S1013 can be implemented by the following method: first, preprocess the local grid map to obtain a preprocessed grid map; then, skeletonize the preprocessed grid map to obtain an initial Voronoi diagram; then, determine the coordinate value of each path intersection in the initial Voronoi diagram, as well as the coordinate value of the robot's current position; finally, based on the initial Voronoi diagram, the coordinate values ​​of the path intersections and the coordinate values ​​of the current position, determine the local Voronoi diagram of the robot's current position.

[0051] Here, preprocessing the local grid map refers to the process of optimizing the original local grid map, including noise removal, data smoothing, etc. Skeletonization is an image processing technique used to extract the main structural features of an object and form a thin "skeleton" representation. The main features of skeletonization include structural simplification, that is, simplifying the edges or contours of the object into center lines or thin lines, removing unnecessary details in the object; and in the skeletonization process, try to maintain the connectivity and topological features of the object so that the shape attributes of the object are not affected. Path intersections represent the points where at least every two paths intersect in the Voronoi diagram.

[0052] In some embodiments, the sweeping robot works in an indoor environment. First, the data of the position of the sweeping robot is obtained through the sensor installed on the sweeping robot to construct a local grid map; then, the local grid map is preprocessed to remove the noise in the local grid map; next, the preprocessed local grid map is skeletonized to obtain an initial Voronoi map; then, a coordinate system is constructed with the current position of the sweeping robot to determine the coordinate values ​​of the path intersection points in the initial Voronoi map; finally, on the basis of the initial Voronoi map, the local Voronoi map of the current position of the sweeping robot is determined according to the coordinate values ​​of the path intersection points and the coordinate values ​​of the current position of the sweeping robot.

[0053] In some embodiments, obtaining a panoramic map of the robot's current position in step S101 can be achieved by the following method: first, obtaining scene images captured after the robot is rotated at different rotation angles at its current position; then, using a preset image stitching algorithm, all scene images are stitched according to the rotation angle corresponding to each scene image to obtain a stitched image; finally, image fusion is performed on the image overlapping areas in the stitched image to obtain a panoramic map of the robot's current position.

[0054] Image stitching algorithm refers to an algorithm that merges multiple images into a continuous image, which can usually use feature matching and geometric transformation; image overlapping area refers to the overlapping part between different images in the stitched image, and the overlapping part usually contains the same scene features; image fusion refers to combining the image information of the overlapping area of ​​the image together to eliminate visual discontinuities and differences, making the final image more natural and coherent.

[0055] In some embodiments, a mobile robot performs panoramic image acquisition within a limited range. First, the mobile robot rotates at different angles at its current position, where the angle of each rotation may be the same or different. For example, the mobile robot takes an image each time it rotates 30 degrees at its current position until the shooting is completed after rotating 360 degrees. A total of 12 images are taken, and the 12 images are stitched together to obtain a stitched image. Since the same object or scene may exist in two adjacent images of the 12 images, such as picture 1 contains chair 1, and picture 2 also contains chair 1 and the scene near chair 1, after completing the image stitching, it is necessary to perform image fusion on the overlapping areas in the stitched images to generate a final panoramic map of the current position of the mobile robot.

[0056] In some embodiments, a local semantic map of the robot's current location can be constructed by the following method: first, data of the surrounding environment, including depth information, color information, etc., is obtained through sensors installed on the robot; then, meaningful features are extracted from the acquired data, for example, furniture and other objects in the room are identified through image recognition technology; then, the extracted features are matched with predefined semantic categories to generate annotation information, for example, the identified table is marked as furniture; finally, based on the annotation information, each feature and its relative position are integrated into the local semantic map; the local semantic map not only shows the spatial layout of the robot's current location, but also contains the semantic information of each object.

[0057] In some embodiments, the local semantic map of the robot's current location can also be manually set by the user, and the objects existing in the environment surrounding the robot's current location and their specific locations are manually marked in the corresponding map to generate a local semantic map.

[0058] Step S102, based on the local Voronoi diagram and the panoramic map, determine the feasible area of ​​the robot at the current position and the area description of the feasible area.

[0059] The feasible area refers to the area where there is a walking path determined by the local Voronoi diagram and the panoramic map; the area description refers to the specific explanation of the feasible area, including the size of the feasible area, the objects and features contained in the feasible area, such as: the feasible area 1 contains a three-legged table with a transparent water cup placed on the table.

[0060] In some embodiments, see Figure 3 , Figure 3 It is a flowchart of determining a feasible area of ​​a robot at a current position and describing an area of ​​the feasible area provided by an embodiment of the present application; Figure 3 It is shown that in step S102, based on the local Voronoi diagram and the panoramic map, the feasible area of ​​the robot at the current position and the regional description of the feasible area are determined, which can be achieved by the following steps S1021 to S1023:

[0061] Step S1021, constructing a coordinate system with the current position of the robot as the coordinate origin.

[0062] In some embodiments, the position of the robot is the origin of the coordinate system; the constructed coordinate system can be a two-dimensional coordinate system or a three-dimensional coordinate system; when constructing a two-dimensional coordinate system, that is, the coordinate of the current position of the robot is (0, 0), the x-axis and y-axis directions of the coordinate system are determined according to the current position of the robot, such as assuming that the x-axis points to the right side of the robot and the y-axis points to the front of the robot; when constructing a three-dimensional coordinate system, that is, the coordinate of the current position of the robot is (0, 0, 0), the x-axis, y-axis and z-axis directions of the coordinate system are determined according to the current position of the robot, such as assuming that the x-axis points to the right side of the robot, the y-axis points to the front of the robot, and the z-axis points to the top of the robot. Here, the directions of the x-axis, y-axis and z-axis are not limited in this application.

[0063] In some embodiments, when constructing a coordinate system, the current position of other objects can be used as the coordinate origin, and the coordinates of the current position of the robot can be determined according to the distance between the robot and the other objects.

[0064] Step S1022: Based on the coordinate system, map matching is performed on the local Voronoi map and the panoramic map to obtain a feasible area for the robot at its current location.

[0065] Map matching refers to aligning the data of two or more maps according to a constructed coordinate system in order to compare and integrate data information.

[0066] In some embodiments, step S1022 can be implemented by the following method: first, based on the coordinate system, determine the coordinate values ​​of each point in the panoramic map and the local Voronoi map; then, based on the coordinate values, match the panoramic map and the local Voronoi map point by point to obtain the matching area between the panoramic map and the local Voronoi map; finally, determine the feasible area of ​​the robot at its current position from the matching area.

[0067] Point-by-point matching refers to the process of matching points with the same coordinate values ​​in the panoramic map and the local Voronoi map according to the coordinate system.

[0068] In some embodiments, based on the constructed coordinate system, the panoramic map and the local Voronoi map are placed on the coordinate system in the same direction, and then the coordinate values ​​of each point on the panoramic map and the local Voronoi map are matched. For example, the coordinates of the points included in path 1 in the panoramic map are (2, 3), (2, 4) and (2, 5). According to the coordinates of each point in path 1, the area of ​​the coordinate points corresponding to path 1 is searched in the local Voronoi map, and so on, to obtain multiple matching areas; finally, the area where the walking path exists is searched in the multiple matching areas, and the area where the walking path exists is determined as a feasible area.

[0069] Step S1023: Generate an area description of the feasible area by using the pre-built first language model.

[0070] The pre-built first language model refers to a pre-trained language model that can generate natural language descriptions. The specific model may be a deep learning-based architecture, such as transformer or LSTM. The specific model is not limited in this application.

[0071] In some embodiments, the determined feasible region is input into a pre-built first language model to generate a region description of the feasible region. For example, after the feasible region is input into the pre-built first language model, the generated region description may be "path 1, door frame 1, and water cup 1 exist in feasible region 1".

[0072] Step S103, based on the local semantic map and the area description, determine a feasible path for the robot at the current location and a path description of the feasible path.

[0073] In some embodiments, the local Voronoi graph includes multiple paths; see Figure 4 , Figure 4 It is a flowchart of determining a feasible path for a robot at a current location based on a local semantic map and a regional description and a path description of the feasible path provided by an embodiment of the present application; Figure 4It is shown that in step S103, based on the local semantic map and the region description, the feasible path of the robot at the current position and the path description of the feasible path are determined, which can be achieved by the following steps S1031 to S1033:

[0074] Step S1031, determining the first description text of each path in the local Voronoi map through the local semantic map; the first description text includes the objects included in each path and the coordinate value of each object.

[0075] In some embodiments, since the local Voronoi map contains multiple paths, first, a coordinate system is constructed based on the current position of the robot; then, the local semantic map and the local Voronoi map are matched according to the coordinate system. The specific matching method is shown in step S1022, and the present application will not repeat it here; after the matching is completed, the objects included in each path in the local Voronoi map and the coordinate values ​​of each object can be determined according to the local semantic map, such as: the first description text generated is {door frame 1; (4, 4); water cup 1; (5, 5)}.

[0076] Step S1032: perform text splicing on the first description text and the area description to obtain a first prompt word.

[0077] In some embodiments, after obtaining the first description text and the area description, the first description text and the area description are filled into a preset first prompt word template to obtain a first prompt word, such as the first prompt word is: "Describe this path according to the following content, {path 1, door frame 1 and cup 1 exist in feasible area 1}, {door frame 1; (4, 4); cup 1; (5, 5)}".

[0078] Step S1033, using the pre-constructed second language model, based on the first prompt word, determine a feasible path for the robot at the current position from multiple paths, as well as a path description of the feasible path.

[0079] The pre-built second language model is a pre-trained large language model for generating path descriptions of feasible paths. The specific model is not limited in this application.

[0080] In some embodiments, the first prompt word is input into a pre-constructed second language model, where there is at least one first prompt word; when there are multiple first prompt words, the feasible paths for the robot's current location are determined based on the multiple first prompt words. For example, there are three first prompt words, the first first prompt word is: "Describe this path according to the following content, {path 1, door frame 1 and water cup 1 exist in feasible area 1}, {door frame 1; (4, 4); water cup 1; (5, 5)}", and the second first prompt word is: "According to the following The content describes this path, {path 2, bookshelf 1 and cup 1 exist in feasible area 2}, {bookshelf 1; (3, 3); cup 1; (5, 5)}", and the first prompt word of the third item is: "Describe this path according to the following content, {path 3, chair 1; and cup 1 exist in feasible area 3}, {chair 1; (2, 2); cup 1; (5, 5)}"; the three prompt words are analyzed by the pre-built second language model, and the feasible path is determined to be path 1, because there are no obstacles in path 1 and it can pass directly. After determining the feasible path, a path description of the feasible path is generated, for example: "The feasible path is path 1, there are door frame 1 and cup 1 in path 1, the coordinates of door frame 1 are (4, 4), the coordinates of cup 1 are (5, 5), and the robot can get cup 1 through door frame 1 on path 1".

[0081] Step S104, determining the path score of the feasible path based on the control instruction, the path description and the local Voronoi diagram.

[0082] The path score of a feasible path refers to a quantitative evaluation of the feasible path. The optimal path for the robot to walk can be determined through the path score.

[0083] In some embodiments, step S104 can be implemented by the following steps: first, performing text conversion on the control instruction to obtain a second description text; then, determining a second prompt word based on the path description and the second description text; then, determining the text similarity between the path description and the second description text based on the second prompt word through a pre-constructed third language model; finally, determining a path score of the feasible path based on the local Voronoi diagram and the text similarity.

[0084] Text conversion refers to converting the voice control instructions input by the user into text data; the second description text is the text data obtained after the voice control instructions input by the user are converted into text; the second prompt word is a prompt word used to guide the third language model to calculate text similarity; the pre-built third language model is a pre-trained large language model used to calculate the text similarity between the path description and the second description text; text similarity is an indicator to measure whether two short texts are semantically similar.

[0085] In some embodiments, the control instruction is converted into text to obtain a second description text: "Help me get the cup"; then, the path description and the second description text are filled into a pre-constructed second prompt word template to obtain a second prompt word, and the second prompt word is: "User demand: 'Help me get the cup'. According to the following path description, the text similarity between each path description and the user demand is determined. The path description is 'The feasible path is path 1, there are door frame 1 and cup 1 in path 1, the coordinates of door frame 1 are (4, 4), the coordinates of cup 1 are (5, 5), and the robot can get cup 1 through door frame 1 on path 1'"; then, the path description and the second description text are pre-processed, such as removing punctuation marks, removing stop words, etc.; then, the pre-processed path description and the second description text are input into a pre-constructed third language model to generate a text vector; finally, the cosine similarity is used to calculate the text similarity between the description text and the second description text; the specific method for calculating text similarity is not limited in this application.

[0086] In some embodiments, a feasible path includes N nodes; determining the path score of the feasible path based on the local Voronoi diagram and text similarity can be achieved by the following method: first, determining the access status of each node in the feasible path based on the local Voronoi diagram; the access status includes whether the node has been visited and whether the node has not been visited; then, scoring the corresponding nodes based on the access status of each node to obtain a score value for each node; then, performing weighted summation on the score values ​​of all nodes in the feasible path to obtain a comprehensive score of the feasible path; finally, determining the product of the text similarity and the comprehensive score as the path score of the feasible path.

[0087] A node refers to a key point in a path, which may be an intersection or a turning point, and this application does not limit this; an access status refers to whether the robot has passed through the node before the current moment. If it has passed through, the access status is visited; if not, the access status is not visited.

[0088] In some embodiments, feasible path 1 includes a total of 5 nodes, namely nodes A, B, C, D, and E. Through the Voronoi diagram, it can be determined that the access status of nodes C, D, and E is visited, then nodes C, D, and E are respectively assigned 0 points, and the access status of nodes A and B is not visited, then nodes A and B are respectively assigned 1 point; then, the scores of nodes A, B, C, D, and E are summed and multiplied by a preset weighted weight. If the weighted weight is 0.4, the comprehensive score of feasible path 1 is 0.4×(1+1+0+0+0)=0.8; finally, the product of the text similarity and the comprehensive score is determined as the path score of feasible path 1. If the text similarity is 0.7, the path score of feasible path 1 is 0.7×0.8=0.56.

[0089] Step S105, based on the path score, determine the target navigation point of the robot, and control the robot to move to the target navigation point.

[0090] The target navigation point refers to the target node that the robot needs to move to next.

[0091] In some embodiments, determining the target navigation point of the robot based on the path score in step S105 can be achieved by the following method: first, determining the feasible path with the highest path score as the target path; then, determining the node in the target path that is closest to the current position as the target navigation point.

[0092] In some embodiments, after calculating the path score of each feasible path, the feasible path with the highest path score is determined as the target path; if the path score of feasible path 1 is 0.78 and the path score of feasible path 2 is 0.56, feasible path 1 is used as the target path; next, the distance between each node in the target path and the current position of the robot can be determined by sensors, and the node closest to the current position is used as the target navigation point.

[0093] In some embodiments, after determining the target navigation point, the local Voronoi map can also be updated according to the determined target navigation point. This can be achieved by the following method: first, when the robot moves to the target navigation point, the access status of the target navigation point is updated; then, based on the updated access status of the target navigation point, the local Voronoi map is updated.

[0094] In some embodiments, when the target navigation point has not been visited by the robot before, after the robot reaches the target navigation point, the access status of the target navigation point is updated to visited, and the score of the target navigation point is set to 0; then, based on the updated access status of the target navigation point, the original access status of the target navigation point stored in the local Voronoi map is replaced with the updated access status.

[0095] The robot control method provided by the embodiment of the present application can be widely used in a variety of scenarios, such as: warehousing and logistics management, smart home cleaning, indoor navigation and service. In the warehousing and logistics management scenario, the robot can be a handling robot. The handling robot analyzes and determines the feasible path and target navigation point by obtaining the local Voronoi map, panoramic map and pre-built local semantic map of the current location of the handling robot, and can evaluate the feasible area in real time, avoid obstacles, and improve the handling efficiency. In the smart home cleaning scenario, the robot can be a sweeping robot. The sweeping robot can determine the feasible area and feasible path by obtaining the local Voronoi map, panoramic map and pre-built local semantic map of the current location of the sweeping robot, thereby determining the target navigation point, controlling the sweeping robot to move, and improving work efficiency. In the indoor navigation and service scenario, the robot can be a service robot. The service robot can find the nearest target location (such as a ward, a store) by obtaining the local Voronoi map, panoramic map and pre-built local semantic map of the current location of the handling robot, and control the robot to move to the target location to improve the service quality.

[0096] The robot control method provided in the embodiment of the present application enables the robot to accurately understand the robot's surrounding environment by acquiring the local Voronoi map, panoramic map and local semantic map of the robot's current position, and improves the accuracy of the robot's path planning through multi-dimensional information fusion; moreover, by acquiring the local Voronoi map and panoramic map, the robot can determine the feasible area in the current environment, so that the robot can respond quickly when performing tasks, avoid invalid paths, and improve navigation efficiency; and, by calculating the path score of each path, the robot can select the best walking path, thereby reducing energy consumption and improving the efficiency of task completion; in addition, the present application only uses the local map of the robot's current position to determine the robot's target navigation point, and there is no need for the robot to pre-scan the entire environment, thereby improving the robot's intelligence and control efficiency.

[0097] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0098] The embodiment of the present application provides a robot control method. Through the robot control method provided by the embodiment of the present application, a more intelligent intelligent interaction and autonomous navigation system can be built for the robot, so that the robot can understand vague human instructions, such as: "Help me find where the cup is?" Based on this instruction, the robot can autonomously explore the environment in an unknown environment and record the exploration status of the environment until the required item is found.

[0099] See also Figure 5 , Figure 5This is a system architecture diagram of the robot control method provided by the embodiment of the present application; the specific implementation steps are as follows:

[0100] Step 1: First, skeletonize the local grid map 301 of the robot's current position to obtain the initial Voronoi diagram of the robot's current position; then, determine the coordinate value of each path intersection in the initial Voronoi diagram; then, combine the coordinate value of the robot's current position to generate a local Voronoi diagram 302; and update the local Voronoi diagram 302 by incremental update. The local Voronoi diagram 302 is used to store the access status and score of each node.

[0101] Step 2: Determine the objects and coordinate values ​​of each path 303 in the local Voronoi diagram 302 through the local semantic map 305 of the current position of the robot, and generate the first description text 304. In addition, the panoramic map 306 is divided into regions according to the path, the feasible region 307 is determined, and the region description 308 of the feasible region 307 is obtained through the multimodal visual language model (the above-mentioned first language model). Generate a first prompt word 309 based on the first description text 304 and the region description 308, and the first prompt word 309 is as follows: "Describe this path according to the following content, {region description}, {first description text}"; input the first prompt word 309 into the second largest language model 310 to obtain the path description 311.

[0102] Step 3: The control instruction 312 input by the user may be: "I want to eat an apple". The control instruction 312 is then converted into a second description text 313, and a second prompt word 314 is generated based on the second description text 313 and the path description 311. The second prompt word is as follows: "User needs: {second description text}. Based on the description of the following path, a probability (the above text similarity) is output for each path that meets the user needs. The path description is: {path description}". Then, the second prompt word 314 is input into the third language model 315, and the text similarity between the second description text 313 and the path description 311 is determined by the third language model 315.

[0103] Step 4: According to the local Voronoi diagram 302, the access status and corresponding score of the node of each feasible path are determined. If the current node has been visited, it will get 0 points; if it has not been visited, it will get 1 point. The scores of all nodes are added and multiplied by the preset weight to obtain the comprehensive score of each path. The comprehensive score is multiplied by the text similarity output by the third language model 315 to obtain the path score 316 of the feasible path, and the path with the highest path score is selected as the target path 317; the node closest to the current position of the robot in the target path 317 is used as the target navigation point 318.

[0104] Step 5: When the robot moves to the target navigation point 318, the access status of the target navigation point is updated to obtain an updated local Voronoi diagram 319.

[0105] The robot control method provided in the embodiment of the present application enables the mobile robot to better realize intelligent interaction with human users in a real artificial environment. Without manually setting complex configurations, the autonomous navigation behavior of the mobile robot can be achieved by means of the fusion of speech and visual semantic information. Due to the integration of the large language model, the interaction of the navigation process becomes more intelligent, and the mobile robot can navigate in an environment without a priori maps.

[0106] Based on the robot control method described in the above embodiment, Figure 6 A structural block diagram of a robot control device 100 provided in an embodiment of the present application is shown. The robot control device may be a device in an electronic device (e.g., a robot). The robot control device may be implemented in software, which may be software in the form of programs and plug-ins, etc., including the following software modules: an acquisition module 101, a first determination module 102, a second determination module 103, a third determination module 104, and a control module 105. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0107] Among them, the acquisition module 101 is used to obtain the local Voronoi map of the current position of the robot, the panoramic map of the current position and the pre-built local semantic map of the current position in response to the control instruction for the robot; the first determination module 102 is used to determine the feasible area of ​​the robot at the current position and the regional description of the feasible area based on the local Voronoi map and the panoramic map; the second determination module 103 is used to determine the feasible path of the robot at the current position and the path description of the feasible path based on the local semantic map and the regional description; the third determination module 104 is used to determine the path score of the feasible path based on the control instruction, the path description and the local Voronoi map; the control module 105 is used to determine the target navigation point of the robot based on the path score, and control the robot to move to the target navigation point.

[0108] In some embodiments, the first determination module is further used to: construct a coordinate system with the current position of the robot as the coordinate origin; based on the coordinate system, perform map matching on the local Voronoi map and the panoramic map to obtain a feasible area of ​​the robot at the current position; and generate an area description of the feasible area through a pre-constructed first language model.

[0109] In some embodiments, the first determination module is further used to: determine the coordinate values ​​of each point in the panoramic map and the local Voronoi map based on the coordinate system; perform point-by-point matching of the panoramic map and the local Voronoi map based on the coordinate values ​​to obtain a matching area between the panoramic map and the local Voronoi map; and determine a feasible area for the robot at its current location from the matching area.

[0110] In some embodiments, the local Voronoi map includes multiple paths; the second determination module is further used to: determine a first description text for each path in the local Voronoi map through the local semantic map; the first description text includes the objects included in each path and the coordinate values ​​of each object; perform text splicing on the first description text and the area description to obtain a first prompt word; and determine a feasible path for the robot at the current position from the multiple paths based on the first prompt word through a pre-constructed second language model, as well as a path description of the feasible path.

[0111] In some embodiments, the third determination module is further used to: perform text conversion on the control instruction to obtain a second description text; determine a second prompt word based on the path description and the second description text; determine the text similarity between the path description and the second description text based on the second prompt word through a pre-constructed third language model; and determine the path score of the feasible path based on the local Voronoi diagram and the text similarity.

[0112] In some embodiments, the feasible path includes N nodes; the third determination module is further used to: determine the access status of each node in the feasible path based on the local Voronoi diagram; the access status includes nodes visited and nodes not visited; score the corresponding nodes based on the access status of each node to obtain the score value of each node; perform weighted summation on the score values ​​of all nodes in the feasible path to obtain a comprehensive score of the feasible path; and determine the product of the text similarity and the comprehensive score as the path score of the feasible path.

[0113] In some embodiments, the third determination module is further used to: determine the feasible path with the highest path score as the target path; and determine the node in the target path that is closest to the current location as the target navigation point.

[0114] In some embodiments, the device further includes an updating module for: updating an access status of the target navigation point when the robot moves to the target navigation point; and updating the local Voronoi diagram based on the updated access status of the target navigation point.

[0115] In some embodiments, the acquisition module is also used to: acquire the position data of the robot's current position and the environmental perception data of the current position; based on the position data and the environmental perception data, determine the local grid map of the robot's current position through a grid map generation tool; and convert the local grid map into a local Voronoi map of the robot's current position.

[0116] In some embodiments, the acquisition module is also used to: preprocess the local grid map to obtain a preprocessed grid map; skeletonize the preprocessed grid map to obtain an initial Voronoi diagram; determine the coordinate values ​​of each path intersection in the initial Voronoi diagram, as well as the coordinate values ​​of the robot's current position; and determine the local Voronoi diagram of the robot's current position based on the initial Voronoi diagram, the coordinate values ​​of the path intersections, and the coordinate values ​​of the current position.

[0117] In some embodiments, the acquisition module is also used to: acquire scene images collected after the robot is rotated at different rotation angles at its current position; perform image stitching on all scene images according to the rotation angle corresponding to each scene image through a preset image stitching algorithm to obtain a stitched image; perform image fusion on the image overlapping areas in the stitched image to obtain a panoramic map of the robot's current position.

[0118] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0119] An embodiment of the present application provides an electronic device, which may be a robot. Figure 7 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 7 As shown, the electronic device 130 includes: at least one processor 131 ( Figure 7 Only one is shown), a memory 132, and computer executable instructions 133 stored in the memory 132 and executable on at least one processor 131, the processor 131 executing the executable instructions 133 to implement the steps of any of the above-mentioned robot control method embodiments.

[0120] The electronic device may include but is not limited to a processor 131 and a memory 132. Those skilled in the art may understand that: Figure 7This is merely an example of the electronic device 130 and does not constitute a limitation on the electronic device 130 . The electronic device 130 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0121] The processor 131 may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPG), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0122] In some embodiments, the memory 132 may be an internal storage unit of the electronic device 130, such as a hard disk or memory of the electronic device 130. In other embodiments, the memory 132 may also be an external storage device of the electronic device 130, such as a plug-in hard disk, a smart memory card (SMC, Smart Media Card), a secure digital (SD, Secure Digital) card, a flash card, etc. equipped on the electronic device 130. Further, the memory 132 may also include both an internal storage unit of the electronic device 130 and an external storage device. The memory 132 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program. The memory 132 may also be used to temporarily store data that has been output or is to be output.

[0123] The present application embodiment provides a computer-readable storage medium in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will be caused to execute the robot control method provided by the present application embodiment, for example, Figure 1 The robot control method is shown.

[0124] The embodiment of the present application provides a computer program product, which includes computer executable instructions, and the computer executable instructions are stored in a computer readable storage medium. The processor of the electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the robot control method described in the embodiment of the present application.

[0125] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0126] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0127] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0128] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0129] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A robot control method, characterized in that: The method comprises: In response to a control instruction for the robot, obtaining a local Voronoi map of the current position of the robot, a panoramic map of the current position, and a pre-constructed local semantic map of the current position; Determining a feasible area of ​​the robot at the current location and an area description of the feasible area based on the local Voronoi diagram and the panoramic map; Determining a feasible path for the robot at the current position and a path description of the feasible path based on the local semantic map and the area description; Determining a path score of the feasible path based on the control instruction, the path description and the local Voronoi diagram; Based on the path score, a target navigation point of the robot is determined, and the robot is controlled to move to the target navigation point.

2. The method according to claim 1, characterized in that The determining, based on the local Voronoi diagram and the panoramic map, a feasible area of ​​the robot at the current location and an area description of the feasible area includes: Constructing a coordinate system with the current position of the robot as the coordinate origin; Based on the coordinate system, map matching is performed on the local Voronoi map and the panoramic map to obtain a feasible area of ​​the robot at the current position; A region description of the feasible region is generated by using a pre-built first language model.

3. The method according to claim 2, characterized in that The performing map matching on the local Voronoi map and the panoramic map based on the coordinate system to obtain a feasible area of ​​the robot at the current position includes: Based on the coordinate system, determining the coordinate values ​​of each point in the panoramic map and the local Voronoi map; Matching the panoramic map and the local Voronoi map point by point based on the coordinate values ​​to obtain a matching area between the panoramic map and the local Voronoi map; A feasible area for the robot at the current position is determined from the matching area.

4. The method according to claim 1, characterized in that: The local Voronoi graph includes multiple paths; The determining, based on the local semantic map and the area description, a feasible path of the robot at the current location and a path description of the feasible path comprises: Determining a first description text of each path in the local Voronoi map through the local semantic map; the first description text includes objects included in each path and coordinate values ​​of each object; Performing text splicing on the first description text and the area description to obtain a first prompt word; By using a pre-constructed second language model, a feasible path for the robot at the current position is determined from the multiple paths based on the first prompt word, as well as a path description of the feasible path.

5. The method according to claim 1, characterized in that The step of determining a path score of the feasible path based on the control instruction, the path description and the local Voronoi diagram comprises: Performing text conversion on the control instruction to obtain a second description text; Determine a second prompt word based on the path description and the second description text; Determining the text similarity between the route description and the second description text based on the second prompt word by using a pre-built third language model; A path score of the feasible path is determined based on the local Voronoi diagram and the text similarity.

6. The method according to claim 5, characterized in that The feasible path includes N nodes; and determining the path score of the feasible path based on the local Voronoi diagram and the text similarity includes: Determine the access status of each of the nodes in the feasible path based on the local Voronoi diagram; the access status includes a node that has been visited and a node that has not been visited; Scoring the corresponding nodes based on the access status of each node to obtain a scoring value of each node; Performing a weighted summation on the score values ​​of all nodes in the feasible path to obtain a comprehensive score of the feasible path; The product of the text similarity and the comprehensive score is determined as the path score of the feasible path.

7. The method according to claim 6, characterized in that Determining a target navigation point of the robot based on the path score includes: The feasible path with the highest path score is determined as the target path; The node in the target path that is closest to the current location is determined as the target navigation point.

8. The method according to claim 7, characterized in that The method further comprises: When the robot moves to the target navigation point, updating the access status of the target navigation point; The local Voronoi diagram is updated based on the updated access status of the target navigation point.

9. The method according to any one of claims 1 to 8, characterized in that: The obtaining of the local Voronoi diagram of the current position of the robot comprises: Acquire location data of the robot's current location and environmental perception data of the robot's current location; Based on the position data and the environmental perception data, determining a local grid map of the current position of the robot by a grid map generation tool; The local grid map is converted into a local Voronoi map of the current position of the robot.

10. The method according to claim 9, characterized in that The converting the local grid map into a local Voronoi map of the current position of the robot comprises: Preprocessing the local grid map to obtain a preprocessed grid map; Skeletonizing the preprocessed grid map to obtain an initial Voronoi diagram; Determine the coordinate value of each path intersection in the initial Voronoi diagram and the coordinate value of the current position of the robot; Based on the initial Voronoi diagram, the coordinate values ​​of the path intersections and the coordinate values ​​of the current position, a local Voronoi diagram of the current position of the robot is determined.

11. The method according to any one of claims 1 to 8, characterized in that: Obtaining a panoramic map of the robot's current location, including: Obtain scene images collected after the robot rotates at different rotation angles at its current position; By using a preset image stitching algorithm, all scene images are stitched according to the rotation angle corresponding to each scene image to obtain a stitched image; Image fusion is performed on the image overlapping areas in the stitched images to obtain a panoramic map of the current position of the robot.

12. A robot control device, characterized in that: include: an acquisition module, configured to acquire, in response to a control instruction for the robot, a local Voronoi map of the current position of the robot, a panoramic map of the current position, and a pre-constructed local semantic map of the current position; A first determination module is used to determine a feasible area of ​​the robot at the current position and an area description of the feasible area based on the local Voronoi diagram and the panoramic map; A second determination module, configured to determine a feasible path of the robot at the current position and a path description of the feasible path based on the local semantic map and the area description; A third determination module, configured to determine a path score of the feasible path based on the control instruction, the path description and the local Voronoi diagram; A control module is used to determine a target navigation point of the robot based on the path score, and control the robot to move to the target navigation point.

13. An electronic device, characterized in that: include: A memory for storing computer executable instructions; A processor, configured to implement the robot control method according to any one of claims 1 to 11 when executing the computer executable instructions stored in the memory.

14. A computer-readable storage medium, characterized in that: Computer executable instructions or computer programs are stored, and when the computer executable instructions or computer programs are executed by a processor, the robot control method according to any one of claims 1 to 11 is implemented.

Images