Semantic information processing method and device for mobile robot

Abstract

The purpose of this application is to provide a semantic information processing solution for mobile robots. This solution first receives and parses user requests through the mobile robot's system to obtain the semantic information corresponding to the user request, and then performs corresponding operations of adding / querying / deleting semantic information based on the semantic information according to the type of the user request, and then associates and adjusts the semantic information and map data. Compared with the prior art, this application can annotate semantic information during the process of mobile robots perceiving the environment, which is more convenient and will not cause mismatches. This application can optimize the process of adding semantic information, reduce the workload of on-site operation and maintenance personnel, and enhance the accuracy of semantic information.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a semantic information processing technology for a mobile robot. Background Art

[0002] Maps are the basis for mobile robots to locate and navigate. In actual application scenarios, there is a need to add additional semantic information to maps. At present, the process of adding semantic information to maps is: first let the robot perceive the entire environment, obtain the scene map, and then mark the semantic information on the map. However, this process is redundant, and semantic information can only be added after the entire map is obtained. As the mobile robot perceives the environment, the entire map will be adjusted. For example, if the mobile robot finds a map loop, it will optimize the entire map to ensure the correctness of the map. Therefore, the position of the semantic information added during the perception of the environment will mismatch the map, resulting in inaccurate position of the semantic information. Summary of the invention

[0003] One object of the present application is to provide a semantic information processing method and device for a mobile robot.

[0004] According to one aspect of the present application, a semantic information processing method for a mobile robot is provided, wherein the method comprises:

[0005] Receive and analyze user requests to obtain semantic information corresponding to the user requests;

[0006] According to the type of the user request, performing a corresponding operation based on the semantic information;

[0007] The semantic information and map data are associated and adjusted.

[0008] According to another aspect of the present application, a semantic information processing device for a mobile robot is also provided, wherein the device comprises:

[0009] A user request receiving module, used to receive and analyze user requests and obtain semantic information corresponding to the user requests;

[0010] A semantic information execution module, configured to execute a corresponding operation based on the semantic information according to the type of the user request;

[0011] The association and adjustment module is used to associate and adjust the semantic information and map data.

[0012] According to another aspect of the present application, a computing device is also provided, wherein the device includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the semantic information processing method for a mobile robot.

[0013] According to another aspect of the present application, a computer-readable medium is provided, on which computer program instructions are stored. The computer-readable instructions can be executed by a processor to implement the semantic information processing method for a mobile robot.

[0014] In the solution provided by the present application, the user request is first received and parsed by the mobile robot system to obtain the semantic information corresponding to the user request, and then the corresponding add / query / delete semantic information operation is performed based on the semantic information according to the type of the user request, and then the semantic information and map data are associated and adjusted. Compared with the prior art, the present application can annotate semantic information during the process of the mobile robot perceiving the environment, which is more convenient and will not cause mismatches. The present application can optimize the process of adding semantic information, reduce the workload of on-site operation and maintenance personnel, and enhance the accuracy of semantic information. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0016] Figure 1 is a flow chart of a semantic information processing method for a mobile robot according to an embodiment of the present application;

[0017] Figure 2 is a flow chart of adding semantic information according to an embodiment of the present application;

[0018] Figure 3 is a flow chart of querying semantic information according to an embodiment of the present application;

[0019] Figure 4 is a flow chart of deleting semantic information according to an embodiment of the present application;

[0020] Figure 5 is a flow chart of associating and adjusting semantic information and map data according to an embodiment of the present application;

[0021] Figure 6 It is a schematic diagram of a semantic information processing device for a mobile robot according to an embodiment of the present application.

[0022] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0023] The present application is described in further detail below in conjunction with the accompanying drawings.

[0024] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (CPU), input / output interfaces, network interfaces and memories.

[0025] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0026] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be a device that can be computer-readable instructions, data structures, programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0027] The embodiment of the present application provides a semantic information processing method for a mobile robot, which can annotate semantic information in the process of the mobile robot perceiving the environment, which is more convenient and will not cause mismatching. The embodiment of the present application overcomes the shortcomings of redundant processes for adding map semantic information in the prior art solutions, optimizes the process of adding semantic information, reduces the workload of on-site operation and maintenance personnel, and enhances the accuracy of semantic information.

[0028] In actual scenarios, the device executing the method may be a user device, a network device, or a device formed by integrating a user device and a network device through a network. The user device includes but is not limited to terminal devices such as mobile robots, smart phones, tablet computers, and personal computers (PCs), and the network device includes but is not limited to network hosts, single network servers, multiple network server sets, or computer sets based on cloud computing. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0029] Figure 1 It is a flow chart of a semantic information processing method for a mobile robot according to an embodiment of the present application, the method comprising step S101, step S102 and step S103.

[0030] Step S101, receiving and parsing a user request, and obtaining semantic information corresponding to the user request.

[0031] For example, the user request is received by a system of a mobile robot (mobile agent), and the system parses the user request to obtain the semantic information M.

[0032] In some embodiments, the type of the user request includes at least any one of the following: adding corresponding semantic information; querying corresponding semantic information; deleting corresponding semantic information.

[0033] Step S102: performing corresponding operations based on the semantic information according to the type of the user request.

[0034] In some embodiments, Figure 2 As shown, step S102 includes: if the type of the user request includes adding corresponding semantic information, first determine whether the mobile robot is in the process of perceiving the environment. If the mobile robot is in the process of perceiving the environment, mark the semantic information M as a pending association state, and add the semantic information to the pending association device to wait for subsequent processing; otherwise, do not associate the semantic information with the map data. Then, try to add the semantic information to the semantic database to determine whether the semantic information is repeated; if the semantic information is not repeated, successfully add the semantic information to the semantic database; if the semantic information is repeated, the addition fails.

[0035] In some embodiments, Figure 3 As shown, step S102 includes: if the type of the user request includes querying the corresponding semantic information, first try to query the semantic information M from the semantic database. If the semantic information M exists in the semantic database, query the mapping relationship between the semantic information M and the map, recalculate the position of the semantic information, and return the new position of the semantic information; if the semantic information M does not exist in the semantic database, return the result of query failure.

[0036] In some embodiments, Figure 4As shown, step S102 includes: if the type of the user request includes deleting the corresponding semantic information, first query the semantic database for the semantic information M. If the semantic information M exists in the semantic database, delete the semantic information M, delete the mapping relationship between the semantic information M and the map, and return a successful deletion result (for example, the user may be notified through the UI interface that the semantic information M is deleted); if the semantic information does not exist in the semantic database, return a failed deletion result.

[0037] Step S103: Associating and adjusting the semantic information and map data.

[0038] In some embodiments, Figure 5 As shown, the step S103 includes: if the mobile robot detects a loop during the process of perceiving the environment, calculating the new position of the map keyframe, and updating the semantic information stored in the device to be associated; traversing the semantic information in the device to be associated, and if the semantic information in the device to be associated is empty, ending the operation and exiting the step S103; if the semantic information in the device to be associated is not empty, setting the semantic information index to 0.

[0039] In some embodiments, Figure 5 As shown, step S103 also includes: if the semantic information in the device to be associated is not empty, setting the semantic information index to 0 and jumping to step (a). Step (a), trying to retrieve a semantic information from the device to be associated according to the semantic information index. If the corresponding semantic information index does not exist in the device to be associated, then end the operation and exit step S103; otherwise, jump to step (b). Step (b), retrieve the corresponding semantic information M in the device to be associated according to the semantic information index, and query the key frame F closest to the semantic information M in the map; if there is a key frame F that meets the conditions in the map, jump to step (c); otherwise, add the semantic information index and jump to step (a). Step (c), associate the semantic information M and the key frame F, calculate the mapping relationship between the semantic information M and the key frame F, and remove the semantic information M from the device to be associated, and jump to step (a).

[0040] Figure 6 It is a schematic diagram of a semantic information processing device for a mobile robot according to an embodiment of the present application, the device includes a user request receiving module 601, a semantic information execution module 602 and an association and adjustment module 603.

[0041] The user request receiving module 601 receives and analyzes the user request to obtain semantic information corresponding to the user request.

[0042] For example, the user request is received by a system of a mobile robot (mobile agent), and the system parses the user request to obtain the semantic information M.

[0043] In some embodiments, the type of the user request includes at least any one of the following: adding corresponding semantic information; querying corresponding semantic information; deleting corresponding semantic information.

[0044] The semantic information execution module 602 executes corresponding operations based on the semantic information according to the type of the user request.

[0045] In some embodiments, Figure 2 As shown, the semantic information execution module 602 is used to: if the type of the user request includes adding corresponding semantic information, first determine whether the mobile robot is in the process of perceiving the environment. If the mobile robot is in the process of perceiving the environment, mark the semantic information M as a pending association state, and add the semantic information to the pending association device to wait for subsequent processing; otherwise, do not associate the semantic information with the map data. Then, try to add the semantic information to the semantic database to determine whether the semantic information is repeated; if the semantic information is not repeated, successfully add the semantic information to the semantic database; if the semantic information is repeated, the addition fails.

[0046] In some embodiments, Figure 3 As shown, the semantic information execution module 602 is used to: if the type of the user request includes querying the corresponding semantic information, first try to query the semantic information M from the semantic database. If the semantic information M exists in the semantic database, query the mapping relationship between the semantic information M and the map, recalculate the position of the semantic information, and return the new position of the semantic information; if the semantic information M does not exist in the semantic database, return the result of query failure.

[0047] In some embodiments, Figure 4 As shown, the semantic information execution module 602 is used to: if the type of the user request includes deleting the corresponding semantic information, first query the semantic database for the semantic information M. If the semantic information M exists in the semantic database, delete the semantic information M, delete the mapping relationship between the semantic information M and the map, and return a successful deletion result (for example, the user may be notified through the UI interface that the semantic information M is deleted); if the semantic information does not exist in the semantic database, return a failed deletion result.

[0048] The association and adjustment module 603 associates and adjusts the semantic information and the map data.

[0049] In some embodiments, Figure 5As shown, the association and adjustment module 603 is used for: if the mobile robot detects a loop during the process of perceiving the environment, calculating the new position of the map keyframe and updating the semantic information stored in the device to be associated; traversing the semantic information in the device to be associated, and if the semantic information in the device to be associated is empty, terminating the operation; if the semantic information in the device to be associated is not empty, setting the semantic information index to 0.

[0050] In some embodiments, Figure 5 As shown, the association and adjustment module 603 is also used for: if the semantic information in the device to be associated is not empty, setting the semantic information index to 0 and jumping to step (a). Step (a), trying to take out a semantic information from the device to be associated according to the semantic information index. If the corresponding semantic information index does not exist in the device to be associated, the operation is terminated; otherwise, jumping to step (b). Step (b), taking out the corresponding semantic information M in the device to be associated according to the semantic information index, querying the key frame F closest to the semantic information M in the map; if there is a key frame F that meets the conditions in the map, jumping to step (c); otherwise, adding the semantic information index and jumping to step (a). Step (c), associating the semantic information M and the key frame F, calculating the mapping relationship between the semantic information M and the key frame F, and removing the semantic information M from the device to be associated, jumping to step (a).

[0051] In summary, the embodiments of the present application can annotate semantic information in the process of mobile robot sensing the environment, which is more convenient and will not cause mismatch. The embodiments of the present application overcome the shortcomings of redundant process of adding map semantic information in the prior art solution, optimize the process of adding semantic information, reduce the workload of on-site operation and maintenance personnel, and enhance the accuracy of semantic information.

[0052] In addition, a part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. The program instruction for calling the method of the present application may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in a working memory of a computer device that runs according to the program instruction. Here, some embodiments of the present application provide a computing device, which includes a memory for storing computer program instructions and a processor for executing computer program instructions, wherein when the computer program instruction is executed by the processor, the device is triggered to execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0053] In addition, some embodiments of the present application further provide a computer-readable medium on which computer program instructions are stored. The computer-readable instructions can be executed by a processor to implement the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0054] It should be noted that the present application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present application (including related data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0055] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is limited by the attached claims rather than the above description, so it is intended to include all changes that fall within the meaning and scope of the equivalent elements of the claims in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A semantic information processing method for a mobile robot, wherein: The method comprises: Receive and analyze user requests to obtain semantic information corresponding to the user requests; According to the type of the user request, performing corresponding operations based on the semantic information, specifically including: if the type of the user request includes adding corresponding semantic information, determining whether the mobile robot is in the process of perceiving the environment; if the mobile robot is in the process of perceiving the environment, marking the semantic information as a to-be-associated state, and adding the semantic information to the to-be-associated device; otherwise, not associating the semantic information with the map data; Associating and adjusting the semantic information and map data, specifically including: if the mobile robot detects a loop in the process of perceiving the environment, calculating the new position of the map key frame, and updating the semantic information stored in the device to be associated; traversing the semantic information in the device to be associated, and if the semantic information in the device to be associated is empty, ending the operation; if the semantic information in the device to be associated is not empty, setting the semantic information index to 0; specifically, if the semantic information in the device to be associated is not empty, setting the semantic information index to 0, and jumping to step (a); (a) trying to take out a from the device to be associated according to the semantic information index If there is no corresponding semantic information index in the device to be associated, the operation is terminated; otherwise, the process goes to step (b); (b) the corresponding semantic information M in the device to be associated is retrieved according to the semantic information index, and the key frame F closest to the semantic information M in the map is queried; if there is a key frame F that meets the conditions in the map, the process goes to step (c); otherwise, the semantic information index is added and the process goes to step (a); (c) the semantic information M is associated with the key frame F, the mapping relationship between the semantic information M and the key frame F is calculated, and the semantic information M is removed from the device to be associated, and the process goes to step (a).

2. The method according to claim 1, wherein: According to the type of the user request, performing a corresponding operation based on the semantic information further includes: The semantic information is added to the semantic database, and it is determined whether the semantic information is repeated; if the semantic information is not repeated, the semantic information is successfully added to the semantic database; if the semantic information is repeated, the adding fails.

3. The method according to claim 1, wherein: According to the type of the user request, a corresponding operation is performed based on the semantic information, including: If the type of the user request includes querying corresponding semantic information, querying the semantic information from a semantic database; If the semantic information exists in the semantic database, query the mapping relationship between the semantic information and the map, recalculate the position of the semantic information, and return the new position of the semantic information; If the semantic information does not exist in the semantic database, a query failure result is returned.

4. The method according to claim 1, wherein: According to the type of the user request, a corresponding operation is performed based on the semantic information, including: If the type of the user request includes deleting corresponding semantic information, querying the semantic information from a semantic database; If the semantic information exists in the semantic database, the semantic information is deleted, the mapping relationship between the semantic information and the map is deleted, and a result of successful deletion is returned; If the semantic information does not exist in the semantic database, a deletion failure result is returned.

5. A semantic information processing device for a mobile robot, wherein: The device comprises: A user request receiving module, used to receive and analyze user requests and obtain semantic information corresponding to the user requests; A semantic information execution module, configured to execute corresponding operations based on the semantic information according to the type of the user request, specifically comprising: if the type of the user request includes adding corresponding semantic information, determining whether the mobile robot is in the process of perceiving the environment; if the mobile robot is in the process of perceiving the environment, marking the semantic information as a to-be-associated state, and adding the semantic information to a to-be-associated device; otherwise, not associating the semantic information with the map data; The association and adjustment module is used to associate and adjust the semantic information and map data, specifically including: if the mobile robot detects a loop in the process of perceiving the environment, calculating the new position of the map key frame, and updating the semantic information stored in the device to be associated; traversing the semantic information in the device to be associated, if the semantic information in the device to be associated is empty, ending the operation; if the semantic information in the device to be associated is not empty, setting the semantic information index to 0; specifically, if the semantic information in the device to be associated is not empty, setting the semantic information index to 0, and jumping to step (a); (a) trying to obtain the semantic information from the device to be associated according to the semantic information index; Take out a semantic information from the device to be associated. If there is no corresponding semantic information index in the device to be associated, end the operation; otherwise, jump to step (b); (b) take out the corresponding semantic information M in the device to be associated according to the semantic information index, and query the key frame F closest to the semantic information M in the map; if there is a key frame F that meets the conditions in the map, jump to step (c); otherwise, add the semantic information index and jump to step (a); (c) associate the semantic information M with the key frame F, calculate the mapping relationship between the semantic information M and the key frame F, and remove the semantic information M from the device to be associated, and jump to step (a).

6. A computing device, wherein: The device comprises a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1 to 4.

7. A computer readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 4.

Images