Service robot control system and method based on AI visual identification

Abstract

The invention discloses a service robot control system and method based on AI visual identification, and belongs to the technical field of AI intelligent control, and the method comprises the following steps: building a database, obtaining and storing article image information on the Internet, receiving and storing article image information input from the outside, pre-storing an action instruction and an action execution program in the database, and sending the action instruction and the action execution program to the server. Performing preliminary training on the convolutional neural network model through article image information in the database, collecting image information of multiple angles around, generating a three-dimensional simulated diagram by taking the service robot as a center, receiving an action instruction, calling a corresponding action execution program according to the action instruction, and performing action execution on the service robot. And determining the corresponding article closest to the service robot through the three-dimensional simulated diagram, and executing the action execution program, so that the service robot can find the article needing to be interacted in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of AI intelligent control, and in particular to a service robot control system and method based on AI visual recognition. Background Technology

[0002] Currently, AI visual recognition is applied to various scenarios such as object detection, facial recognition, and scene understanding, and is widely used in fields such as security, healthcare, retail, and autonomous driving. Service robots, as an emerging intelligent technology, have a very promising future. Among them, home service robots have already seen good applications, demonstrating effectiveness in caring for people with limited mobility at home, such as the elderly, patients, and the disabled. They can provide functions such as life assistance, health monitoring, emotional companionship, and remote monitoring. Users can use voice commands or control panels to instruct the service robot to retrieve or interact with items, such as opening and closing doors or picking up containers.

[0003] The existing technical solutions mentioned above have the following drawbacks: Since the real environment is more complex and changeable than the laboratory, it is difficult to ensure that all items around the service robot can be displayed. The service robot needs to be able to interact with items normally in complex environments. Summary of the Invention

[0004] To enable service robots to interact with objects normally in complex environments, this application provides a service robot control system and method based on AI visual recognition.

[0005] On the one hand, the service robot control method based on AI visual recognition provided in this application adopts the following technical solution: A service robot control method based on AI visual recognition includes the following steps: Establish a database, acquire and store item image information from the Internet, receive and store item image information input from the outside, select the item graphic area in the item image information, and associate the item image information with the item name information; The database stores action instructions and action execution programs. The action instructions are associated with the action execution programs, and the action execution programs include setting actions for items. A pre-set convolutional neural network model is initially trained using item image information from the database. After receiving an image, the convolutional neural network model identifies all item graphic regions and item name information within the corresponding image. Collect image information from multiple angles around the surroundings and import the image information into a convolutional neural network model to obtain graphic regions of surrounding objects and information on the names of the objects; A 3D simulation map is generated centered on the service robot, and the 3D simulation map is modified in real time based on the surrounding object graphic areas and object name information; After receiving the action instruction, the corresponding action execution program is invoked according to the action instruction. If the action execution program is to set an action for a specific item, the corresponding item closest to the service robot is determined through the 3D simulation map, and the action execution program is executed.

[0006] By adopting the above solution, the system can automatically identify all items around the service robot through AI visual recognition. After the service robot receives the action command, it can quickly identify the items that need to be interacted with and the existing items. This not only ensures that the service robot can quickly identify the target items and complete the command actions normally, but also helps the service robot plan its movement route, avoid obstacles in advance, and perform actions normally in complex environments.

[0007] Preferably, the following steps are also included: Add auxiliary words to the database. Auxiliary words include central direction words and landmark direction words. Central direction words include direction words, and landmark direction words include item names and direction words. A word parsing model is obtained by training a neural network model based on auxiliary words in the database; If the specific item in the action command cannot be found after receiving the action command, wait to receive the command statement. After receiving the command statement, use the word parsing model to determine whether the command statement includes the central direction word and / or the marker direction word and select auxiliary words. If the instruction statement includes a central direction word, then analyze the direction word in the central direction word, take the service robot as the origin, control the service robot to move along the direction indicated by the direction word, and try to execute the action execution program again. If the instruction statement includes a marker direction word, then analyze the item name and direction word in the marker direction word, find the item graphic area in the current 3D simulation map that corresponds to the item name in the marker direction word, and use the found item graphic area as the origin to control the service robot to move closer to the direction indicated by the direction word and try to execute the action execution program again. If the instruction statement includes both a central direction word and a marker direction word, then the direction indicated by the marker direction word is used to correct the direction indicated by the direction word in the central direction word, controlling the service robot to move in the corrected direction and try to execute the action execution program again.

[0008] By adopting the above scheme, if the target item is too small to be seen in the service robot's field of vision, or if there is no target item in the field of vision, causing the service robot to be unable to find the target item, the service robot will wait for the user to issue a prompt command. The system will intelligently analyze the prompt command issued by the user and control the service robot to move closer to the target.

[0009] Preferably, the step "when the specific item in the action instruction cannot be found after receiving the action instruction, then wait to receive the instruction statement" further includes: A preset request statement template is provided, which includes a fixed statement and an item name placeholder. When a request statement needs to be output, the item name to be found is selected and added to the item name placeholder in the request statement template to obtain the request statement. If a specific item cannot be found after receiving an action command, a request statement is output based on the item name of the specific item, and the system waits to receive an auxiliary word.

[0010] By adopting the above scheme, while the service robot is waiting to receive prompts, the service robot will first issue a request statement based on the target information to prompt the user to issue a prompt.

[0011] Preferably, the step "pre-storing action instructions and action execution programs in the database" further includes: A neural network model is trained based on action commands in the database to obtain a command recognition model; When a service robot receives a command statement, the command recognition model filters out the action commands within the command statement.

[0012] By adopting the above solution, since the instructions issued by users may not be completely accurate, the service robot needs to parse the instructions using an intelligent model when receiving them.

[0013] Preferably, the step of "modifying the 3D simulation image in real time based on the surrounding object graphic area and object name information" further includes: When the images were initially acquired, 2D to 3D conversion software was used to generate a 3D model from each image. All 3D models were then imported into a blank 3D model image with the service robot as the center and overlapped with each other. The 3D model in the 3D simulation is divided into occupied spaces and colored according to the surrounding object graphic areas, and the item name information is associated with the corresponding colored mark.

[0014] By adopting the above scheme, the system constructs a three-dimensional model of all the terrain around the service robot when generating the three-dimensional simulation map, which makes it easier for the service robot to construct its movement route.

[0015] On the other hand, the service robot control system based on AI visual recognition provided in this application adopts the following technical solution: A service robot control system based on AI visual recognition includes a data storage module, a model training module, an image acquisition module, a 3D generation module, and an instruction execution module. The data storage module acquires and stores item image information from the Internet, receives and stores item image information input from the outside, and selects the item graphic area in the item image information to associate the item image information with the item name information. The data storage module pre-stores action instructions and action execution programs, and the action instructions are associated with the action execution programs. The action execution programs include setting actions for the item. The model training module presets a convolutional neural network model, calls the item image information stored in the data storage module to perform preliminary training on the convolutional neural network model, and after receiving the image, the convolutional neural network model identifies all item graphic regions and item name information in the corresponding image and transmits the convolutional neural network model to the image acquisition module. The image acquisition module acquires multi-angle image information around the service robot, imports the image information into the convolutional neural network model, obtains the graphic regions of surrounding objects and the name information of the objects, and transmits it to the 3D generation module. The 3D generation module generates a 3D simulation image centered on the service robot, modifies the 3D simulation image in real time based on the surrounding object graphic areas and object name information, and transmits the 3D simulation image to the instruction execution module. The instruction execution module receives action instructions input from the outside. After receiving the action instructions, it calls the corresponding action execution program stored in the data storage module according to the action instructions. If the action execution program is to set an action for a specific item, it determines the corresponding item closest to the service robot through the three-dimensional simulation map and executes the action execution program.

[0016] By adopting the above solution, the system can automatically identify all items around the service robot through AI visual recognition. After the service robot receives the action command, it can quickly identify the items that need to be interacted with and the existing items. This not only ensures that the service robot can quickly identify the target items and complete the command actions normally, but also helps the service robot plan its movement route, avoid obstacles in advance, and perform actions normally in complex environments.

[0017] Preferably, it also includes a verification module, wherein the data storage module pre-stores auxiliary words, including central direction words and marker direction words, the central direction words including direction words, and the marker direction words including item names and direction words; The model training module calls the auxiliary word training neural network model of the data storage module to obtain the word parsing model, and then transmits the word parsing model to the execution verification module; When the instruction execution module cannot find the specific item in the action instruction after receiving the action instruction, it transmits a verification request to the execution verification module. After receiving a verification request, the execution verification module waits to receive an instruction statement. Upon receiving the instruction statement, it uses a word parsing model to determine whether the instruction statement includes a central direction word and / or a marker direction word and selects auxiliary words. If the instruction statement includes a central direction word, it analyzes the direction word in the central direction word, uses the service robot as the origin, controls the service robot to move along the direction indicated by the direction word, and attempts to execute the action execution program again. If the instruction statement includes a marker direction word, it analyzes the item name and direction word in the marker direction word, finds the item graphic region corresponding to the item name in the marker direction word in the current 3D simulation map, uses the found item graphic region as the origin, controls the service robot to move closer to the direction indicated by the direction word, and attempts to execute the action execution program again. If the instruction statement includes both a central direction word and a marker direction word, it uses the direction indicated by the marker direction word to correct the direction indicated by the direction word in the central direction word, controls the service robot to move in the corrected direction, and attempts to execute the action execution program again.

[0018] By adopting the above scheme, if the target item is too small to be seen in the service robot's field of vision, or if there is no target item in the field of vision, causing the service robot to be unable to find the target item, the service robot will wait for the user to issue a prompt command. The system will intelligently analyze the prompt command issued by the user and control the service robot to move closer to the target.

[0019] Preferably, the execution verification module presets a request statement template, which includes a fixed statement and an item name placeholder. When a request statement needs to be output, the item name to be found is selected, and the item name is added to the item name placeholder in the request statement template to obtain the request statement. After receiving the verification request, the execution verification module outputs the request statement according to the item name of the specific item and waits to receive auxiliary words.

[0020] By adopting the above scheme, while the service robot is waiting to receive prompts, the service robot will first issue a request statement based on the target information to prompt the user to issue a prompt.

[0021] Preferably, the model training module calls the action instructions of the data storage module to train the neural network model to obtain the instruction recognition model, and then transmits the instruction recognition model to the instruction execution module; When the instruction execution module receives an instruction statement, the instruction recognition model filters the action instructions in the instruction statement.

[0022] By adopting the above solution, since the instructions issued by users may not be completely accurate, the service robot needs to parse the instructions using an intelligent model when receiving them.

[0023] Preferably, when the 3D generation module first receives image information, it uses 2D to 3D software to generate a 3D model for each image information, imports all 3D models into a blank 3D model image with the service robot as the center and overlaps them, divides the space occupied by the 3D model in the 3D simulation image according to the surrounding object graphic areas and colors and marks them, and associates the item name information with the corresponding color mark.

[0024] By adopting the above scheme, the system constructs a three-dimensional model of all the terrain around the service robot when generating the three-dimensional simulation map, which makes it easier for the service robot to construct its movement route.

[0025] In summary, the present invention has the following beneficial effects: 1. It can not only ensure that service robots can quickly identify target items and complete the command actions normally, but also help service robots plan their movement routes, avoid obstacles in advance, and perform actions normally in complex environments. Attached Figure Description

[0026] Figure 1 This is an overall system block diagram of Embodiment 2 of this application.

[0027] Explanation of reference numerals in the attached figures: 1. Data storage module; 2. Model training module; 3. Image acquisition module; 4. 3D generation module; 5. Instruction execution module; 6. Execution verification module. Detailed Implementation

[0028] Example 1: This application discloses a service robot control method based on AI visual recognition, the specific steps of which are as follows: S100. Establish the database.

[0029] S101. Obtain and store item image information from the Internet, receive and store item image information input from the outside, select the item graphic area in the item image information, and associate the item image information with the item name information.

[0030] S102. Pre-store action instructions and action execution programs in the database. The action instructions are associated with the action execution programs, and the action execution programs include setting actions for items.

[0031] S103. Train a neural network model based on the action instructions in the database to obtain an instruction recognition model. When the service robot receives an instruction statement, the instruction recognition model filters the action instructions in the instruction statement.

[0032] S104. Add auxiliary words to the database. Auxiliary words include central direction words and marker direction words. Central direction words include direction words, and marker direction words include item names and direction words. A preset request statement template is provided. The request statement template includes fixed statements and item name placeholders. When a request statement needs to be output, select the item name to be found, add the item name to the item name placeholder in the request statement template to obtain the request statement.

[0033] S200, Preset convolutional neural network model.

[0034] S201. The convolutional neural network model is initially trained using the item image information in the database. After receiving the image, the convolutional neural network model identifies all the item graphic regions and item name information in the corresponding image.

[0035] S202. Train a neural network model based on auxiliary words in the database to obtain a word parsing model.

[0036] S300: Collects multi-angle image information of the surrounding environment and imports the image information into a convolutional neural network model to obtain the graphic regions of surrounding objects and the names of the objects.

[0037] S301. Generate a 3D simulation map centered on the service robot, and modify the 3D simulation map in real time based on the surrounding object graphic areas and object name information.

[0038] S302. When initially acquiring images, use 2D to 3D conversion software to generate a 3D model from each image information. Import all 3D models into a blank 3D model drawing with the service robot as the center and overlap them.

[0039] S303. Divide the space occupied by the 3D model in the 3D simulation diagram according to the surrounding object graphic area and color mark it, and associate the item name information with the corresponding color mark.

[0040] S400: After receiving the action command, the corresponding action execution program is called according to the action command. If the action execution program is to set an action for a specific item, the corresponding item closest to the service robot is determined through the three-dimensional simulation map, and the action execution program is executed.

[0041] S401. When a specific item in an action command cannot be found after receiving the command, a request statement is output based on the item name of the specific item, and the system waits to receive an auxiliary word. Upon receiving the command statement, the system uses a word parsing model to determine whether the command statement includes a central direction word and / or a marker direction word, and selects the auxiliary word accordingly.

[0042] S402. If the instruction statement includes a central direction word, analyze the direction word in the central direction word, take the service robot as the origin, control the service robot to move along the direction indicated by the direction word, and try to execute the action execution program again.

[0043] S403. If the instruction statement includes a marker direction word, analyze the item name and direction word in the marker direction word, find the item graphic area corresponding to the item name in the marker direction word in the current 3D simulation map, and use the found item graphic area as the origin to control the service robot to move closer to the direction indicated by the direction word and try to execute the action execution program again.

[0044] S404. If the instruction statement includes both a center direction word and a marker direction word, then the direction indicated by the marker direction word is used to correct the direction indicated by the direction word in the center direction word, and the service robot is controlled to move in the corrected direction and try to execute the action execution program again.

[0045] The implementation principle of the service robot control system and method based on AI visual recognition in this application embodiment is as follows: the system automatically identifies all items around the service robot through the AI ​​visual recognition function. After the service robot receives the action command, it can quickly identify the items that need to be interacted with and the existing items. This can not only ensure that the service robot can quickly identify the target items and complete the command action normally, but also help the service robot plan the movement route, avoid obstacles in advance, and perform actions normally in complex environments.

[0046] This application discloses a service robot control system based on AI visual recognition, such as... Figure 1 As shown, it includes a data storage module 1, a model training module 2, an image acquisition module 3, a 3D generation module 4, an instruction execution module 5, and an execution verification module 6.

[0047] like Figure 1 As shown, data storage module 1 acquires and stores item image information from the internet, receives and stores item image information input from external sources, and selects the item graphic area for each item image information, associating the item image information with item name information. Data storage module 1 pre-stores action instructions and action execution programs, with the action instructions and action execution programs linked. The action execution programs include setting actions for the items. Data storage module 1 pre-stores auxiliary words, including center direction words and marker direction words. The center direction words include direction words, and the marker direction words include item names and direction words.

[0048] like Figure 1As shown, the model training module 2 presets a convolutional neural network model and uses the item image information stored in the data storage module 1 to perform initial training on the convolutional neural network model. After receiving an image, the convolutional neural network model identifies all item graphic regions and item name information within the corresponding image and transmits the convolutional neural network model to the image acquisition module 3. The model training module 2 uses the auxiliary words in the data storage module 1 to train the neural network model to obtain a word parsing model and transmits the word parsing model to the execution verification module 6. The model training module 2 uses the action commands in the data storage module 1 to train the neural network model to obtain an instruction recognition model and transmits the instruction recognition model to the instruction execution module 5.

[0049] like Figure 1 As shown, the image acquisition module 3 acquires multi-angle image information around the service robot, imports the image information into the convolutional neural network model, obtains the graphic regions of surrounding objects and the name information of the objects, and transmits it to the 3D generation module 4.

[0050] like Figure 1 As shown, the 3D generation module 4 generates a 3D simulation map centered on the service robot. It modifies the 3D simulation map in real time based on the surrounding object graphic areas and object name information, and then transmits the 3D simulation map to the instruction execution module 5. When the 3D generation module 4 initially receives image information, it uses 2D-to-3D conversion software to generate a 3D model from each image. It imports all 3D models into a blank 3D model map centered on the service robot and overlaps them. Based on the surrounding object graphic areas, it divides the 3D models in the 3D simulation map into occupied spaces and colors them, associating the object name information with the corresponding color marks. When generating the 3D simulation map, the system constructs a 3D model of all the terrain around the service robot, facilitating the service robot's route planning.

[0051] like Figure 1 As shown, the instruction execution module 5 receives instruction statements from the outside. When receiving an instruction statement, the instruction recognition model filters the action instructions within the instruction statement. After receiving the action instruction, it calls the corresponding action execution program stored in the data storage module 1 according to the action instruction. If the action execution program is to set an action for a specific item, it determines the corresponding item closest to the service robot through a 3D simulation and executes the action execution program. If the instruction execution module 5 cannot find the specific item in the action instruction after receiving it, it transmits a verification request to the execution verification module 6. Since the instruction statements issued by the user may not be completely accurate, the service robot needs to parse the instruction statements using an intelligent model when receiving them.

[0052] like Figure 1As shown, after receiving a verification request, the verification module 6 waits to receive a command statement. Upon receiving the command statement, it uses a word analysis model to determine whether the command statement includes a central direction word and / or a marker direction word, and selects auxiliary words. If the command statement includes a central direction word, it analyzes the direction word within the central direction word, uses the service robot as the origin, controls the service robot to move along the direction indicated by the direction word, and attempts to execute the action execution program again. If the command statement includes a marker direction word, it analyzes the item name and direction word within the marker direction word, finds the item graphic region corresponding to the item name in the marker direction word in the current 3D simulation map, uses the found item graphic region as the origin, controls the service robot to move closer to the direction indicated by the direction word, and attempts to execute the action execution program again. If the command statement includes both a central direction word and a marker direction word, it uses the direction indicated by the marker direction word to correct the direction indicated by the direction word in the central direction word, controls the service robot to move in the corrected direction, and attempts to execute the action execution program again. If the target item is too small to be seen in the service robot's field of vision, or if there is no target item in the field of vision, causing the service robot to be unable to find the target item, the service robot will wait for the user to issue a prompt command. The system will intelligently interpret the prompt command issued by the user and control the service robot to move closer to the target.

[0053] The verification module 6 uses a preset request statement template, which includes a fixed statement and item name placeholders. When a request statement needs to be output, the name of the item to be found is selected and added to the item name placeholder in the request statement template to obtain the request statement. After receiving the verification request, the verification module 6 outputs the request statement based on the item name of the specific item and waits to receive auxiliary words. While the service robot is waiting to receive prompts, it will first issue a request statement based on the target information to prompt the user to issue a prompt.

[0054] The embodiments described herein are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape, and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A service robot control method based on AI visual recognition, characterized in that, Includes the following steps: Establish a database, acquire and store item image information from the Internet, receive and store item image information input from the outside, select the item graphic area in the item image information, and associate the item image information with the item name information; The database stores action instructions and action execution programs. The action instructions are associated with the action execution programs, and the action execution programs include setting actions for items. A pre-set convolutional neural network model is initially trained using item image information from the database. After receiving an image, the convolutional neural network model identifies all item graphic regions and item name information within the corresponding image. Collect image information from multiple angles around the surroundings and import the image information into a convolutional neural network model to obtain graphic regions of surrounding objects and information on the names of the objects; A 3D simulation map is generated centered on the service robot, and the 3D simulation map is modified in real time based on the surrounding object graphic areas and object name information; After receiving the action instruction, the corresponding action execution program is invoked according to the action instruction. If the action execution program is to set an action for a specific item, the corresponding item closest to the service robot is determined through the 3D simulation map, and the action execution program is executed.

2. The service robot control method based on AI visual recognition according to claim 1, characterized in that, It also includes the following steps: Add auxiliary words to the database. Auxiliary words include central direction words and landmark direction words. Central direction words include direction words, and landmark direction words include item names and direction words. A word parsing model is obtained by training a neural network model based on auxiliary words in the database; If the specific item in the action command cannot be found after receiving the action command, wait to receive the command statement. After receiving the command statement, use the word parsing model to determine whether the command statement includes the central direction word and / or the marker direction word and select auxiliary words. If the instruction statement includes a central direction word, then analyze the direction word in the central direction word, take the service robot as the origin, control the service robot to move along the direction indicated by the direction word, and try to execute the action execution program again. If the instruction statement includes a marker direction word, then analyze the item name and direction word in the marker direction word, find the item graphic area in the current 3D simulation map that corresponds to the item name in the marker direction word, and use the found item graphic area as the origin to control the service robot to move closer to the direction indicated by the direction word and try to execute the action execution program again. If the instruction statement includes both a central direction word and a marker direction word, then the direction indicated by the marker direction word is used to correct the direction indicated by the direction word in the central direction word, controlling the service robot to move in the corrected direction and try to execute the action execution program again.

3. The service robot control method based on AI visual recognition according to claim 2, characterized in that, The step "If the specific item in the action command cannot be found after receiving the action command, then wait to receive the command statement" also includes: A preset request statement template is provided, which includes a fixed statement and an item name placeholder. When a request statement needs to be output, the item name to be found is selected and added to the item name placeholder in the request statement template to obtain the request statement. If a specific item cannot be found after receiving an action command, a request statement is output based on the item name of the specific item, and the system waits to receive an auxiliary word.

4. The service robot control method based on AI visual recognition according to claim 1, characterized in that, The step "pre-store action instructions and action execution programs in the database" also includes: A neural network model is trained based on action commands in the database to obtain a command recognition model; When a service robot receives a command statement, the command recognition model filters out the action commands within the command statement.

5. The service robot control method based on AI visual recognition according to claim 1, characterized in that, The step of "modifying the 3D simulation map in real time based on the surrounding graphic areas of objects and the information of object names" also includes: When the images were initially acquired, 2D to 3D conversion software was used to generate a 3D model from each image. All 3D models were then imported into a blank 3D model image with the service robot as the center and overlapped with each other. The 3D model in the 3D simulation is divided into occupied spaces and colored according to the surrounding object graphic areas, and the item name information is associated with the corresponding colored mark.

6. A service robot control system based on AI visual recognition, characterized in that: It includes a data storage module (1), a model training module (2), an image acquisition module (3), a 3D generation module (4), and an instruction execution module (5); The data storage module (1) acquires and stores the image information of items on the Internet, receives and stores the image information of items input from the outside, the image information of items is selected by the graphic area of ​​the item, and the image information of items is associated with the name information of the item. The data storage module (1) pre-stores the action instructions and the action execution program. The action instructions are associated with the action execution program. The action execution program includes setting actions for the item. The model training module (2) presets a convolutional neural network model, calls the item image information stored in the data storage module (1) to perform preliminary training on the convolutional neural network model, and after receiving the image, the convolutional neural network model identifies all the item graphic regions and item name information in the corresponding image and transmits the convolutional neural network model to the image acquisition module (3). The image acquisition module (3) acquires multi-angle image information around the service robot, imports the image information into the convolutional neural network model, obtains the surrounding object graphic area and object name information, and transmits it to the three-dimensional generation module (4). The three-dimensional generation module (4) generates a three-dimensional simulation map centered on the service robot, modifies the three-dimensional simulation map in real time according to the surrounding object graphic area and object name information, and transmits the three-dimensional simulation map to the instruction execution module (5). The instruction execution module (5) receives action instructions input from the outside. After receiving the action instructions, it calls the corresponding action execution program stored in the data storage module (1) according to the action instructions. If the action execution program is to set an action for a specific item, it determines the corresponding item closest to the service robot through the three-dimensional simulation map and executes the action execution program.

7. A service robot control system based on AI visual recognition according to claim 6, characterized in that: It also includes a verification module (6), and the data storage module (1) pre-stores auxiliary words, which include central direction words and marker direction words. The central direction words include direction words, and the marker direction words include item names and direction words. The model training module (2) calls the auxiliary word training neural network model of the data storage module (1) to obtain the word parsing model, and transmits the word parsing model to the execution verification module (6); When the instruction execution module (5) cannot find the specific item in the action instruction after receiving the action instruction, it transmits a verification request to the execution verification module (6). After receiving the verification request, the execution verification module (6) waits to receive the instruction statement. After receiving the instruction statement, it uses the word parsing model to determine whether the instruction statement includes a central direction word and / or a marker direction word and selects auxiliary words. If the instruction statement includes a central direction word, it analyzes the direction word in the central direction word, uses the service robot as the origin, controls the service robot to move along the direction indicated by the direction word, and tries to execute the action execution program again. If the instruction statement includes a marker direction word, it analyzes the item name and direction word in the marker direction word, finds the item graphic area corresponding to the item name in the marker direction word in the current three-dimensional simulation map, uses the found item graphic area as the origin, controls the service robot to move closer to the direction indicated by the direction word, and tries to execute the action execution program again. If the instruction statement includes both a central direction word and a marker direction word, it uses the direction indicated by the marker direction word to correct the direction indicated by the direction word in the central direction word, controls the service robot to move in the corrected direction, and tries to execute the action execution program again.

8. A service robot control system based on AI visual recognition according to claim 7, characterized in that: The execution verification module (6) presets a request statement template, which includes a fixed statement and an item name placeholder. When a request statement needs to be output, the item name to be found is selected, and the item name is added to the item name placeholder in the request statement template to obtain the request statement. After receiving the verification request, the execution verification module (6) outputs the request statement according to the item name of the specific item and waits to receive auxiliary words.

9. A service robot control system based on AI visual recognition according to claim 6, characterized in that: The model training module (2) calls the action instructions of the data storage module (1) to train the neural network model to obtain the instruction recognition model, and transmits the instruction recognition model to the instruction execution module (5); When the instruction execution module (5) receives an instruction statement, the instruction recognition model filters the action instructions in the instruction statement.

10. A service robot control system based on AI visual recognition according to claim 6, characterized in that: When the three-dimensional generation module (4) receives image information for the first time, it uses two-dimensional to three-dimensional software to generate a three-dimensional model for each image information. It imports all three-dimensional models into a blank three-dimensional model map with the service robot as the center and overlaps each other. It divides the space occupied by the three-dimensional model in the three-dimensional simulation map according to the surrounding object graphic area and colors and marks it. It associates the item name information with the corresponding color mark.

Images