Target searching method and system for mobile robot

Abstract

The invention provides a mobile robot target searching method and system, and the method comprises the steps: S1, obtaining the description of a target object, carrying out the feature coding of the target object according to the presentation form of the target object, and obtaining a key point matrix; executing a planning navigation algorithm by using the mobile robot, exploring an unknown environment, reading RGBD camera data according to the key point matrix, executing an image segmentation algorithm, constructing a semantic grid map, and determining a sensitive target; using the mobile robot to perform matching operation on the sensitive target and the target article, and determining a to-be-screened target article; screening is carried out on found to-be-screened target objects, and a primary screening result is obtained through a manual intervention mechanism; and according to a primary screening result, carrying out secondary searching on the target article, and searching and determining a final target article in the cached semantic grid map. According to the invention, the technical problems of dependence on a real scene map, high sensing equipment cost, high data storage overhead and high data processing complexity are solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent robots, and in particular to a mobile robot target searching method and system. Background Art

[0002] With the continuous advancement of science and technology, indoor object-finding robots, as an intelligent and innovative technology, are gradually entering people's lives and workplaces. Indoor object-finding robots combine cutting-edge technologies such as computer vision, artificial intelligence, and the Internet of Things. They can autonomously navigate in complex indoor environments and accurately locate and find specific items.

[0003] Object-finding robots based on artificial intelligence algorithms can greatly improve the efficiency and accuracy of object-finding. Through high-precision sensors and intelligent algorithms, they can quickly scan the environment and locate the specific location of the target object, greatly shortening the time of searching and improving efficiency and accuracy. Indoor object-finding robots are widely used in many fields such as home, medical care, and warehousing. In the home, robots can help the elderly and patients with memory disorders find frequently lost items, such as keys and glasses, to improve their quality of life. In medical institutions, robots can help nurses and doctors quickly find the medical equipment and drugs they need, saving valuable time and improving the efficiency and quality of medical services. In large storage centers, robots can track the location of inventory items in real time, optimize storage management processes, reduce labor costs, and reduce error rates.

[0004] Modern object-finding algorithms are mainly based on machine vision to establish the robot's perception of the surrounding situation, combined with mobile navigation and Internet of Things technology to realize the automatic object-finding function of mobile robots. At the same time, voice and natural language processing algorithms can be combined to realize voice control functions. Indoor object-finding robots not only solve the common problem of finding objects in life, improve the efficiency of daily life and work, but also promote the development and application of related technologies. The popularization of this technology will greatly change our lifestyle, enabling robots to interact with home appliances and other smart devices to build a more intelligent and connected living environment.

[0005] The existing invention patent application document with publication number CN113917932A is "A method based on machine vision for finding target goods in a shopping mall". This existing method realizes a method of finding a single item at a time, but does not provide a suitable method for multiple searches or finding multiple targets at a time, which are often application scenarios, and the algorithm requires a real map of the scene, which cannot be applied to unknown scenes.

[0006] The existing invention patent application document "A method for mobile robot to navigate an object in an unknown environment" with publication number CN117268386A adopts SLAM mapping navigation and plans the robot's moving path, and records the target objects found during the movement to achieve the search of the target object. This method requires the use of expensive laser radar based on SLAM mapping, and does not combine target information with mapping information, which increases the cost of data storage and the complexity of data processing.

[0007] In summary, the existing technologies have technical problems such as dependence on real maps of the scene, high cost of sensor equipment, and high data storage overhead and data processing complexity. Summary of the invention

[0008] The technical problem to be solved by the present invention is: how to solve the technical problems in the prior art of dependence on real scene maps, high cost of sensor equipment, and high data storage overhead and data processing complexity.

[0009] The present invention adopts the following technical solutions to solve the above technical problems: A mobile robot target search method comprises:

[0010] S1. Obtain the description of the target object, and perform feature encoding on the target object according to the presentation form of the target object to obtain a key point matrix;

[0011] S2, use the mobile robot to execute the planning and navigation algorithm, explore the unknown environment, read the RGBD camera data according to the key point matrix, execute the image segmentation algorithm, build the semantic grid map, and identify sensitive targets;

[0012] S3. Use a mobile robot to match sensitive targets with target items to determine the target items to be screened;

[0013] S4. Screening and manual intervention mechanisms are performed on the discovered target items to be screened to obtain a screening result;

[0014] S5. Based on the first screening result, a second search is performed on the target item, and the final target item is found and determined in the cached semantic grid map.

[0015] The present invention uses a depth camera and an image semantic segmentation algorithm to build a semantic map, which can be used for autonomous robot navigation and can be used as memory storage to achieve rapid search in the same scene. The present invention is based on key point matching and image text matching methods, which are suitable for accurate matching of target objects (text descriptions or images) and objects discovered by the camera.

[0016] In a more specific technical solution, S1 includes:

[0017] S11, obtaining a text target in the target item description, and encoding text features using a text encoder of the CLIP model according to the text data in the text target to obtain a vector representation of the text data;

[0018] S12. For the image data in the target item description, use the SuperPoint model to extract key points and obtain a key point matrix as the image feature encoding result and the text feature encoding result;

[0019] S13. Save the image feature encoding result and the text feature encoding result.

[0020] In a more specific technical solution, in S12, the representation information of the key point matrix includes: the position of the key point in the image and the key point descriptor.

[0021] In a more specific technical solution, in S12, according to the following logic, the key point matrix is ​​used to represent the position of the key point image and the key point descriptor respectively:

[0022] [[x1,y2,a 1, a 2, a 3, a 4, a 5, a 6, ·····,a n ],[x2,y2,b 1, b 2, b 3, b 4, b 5, b 6, ·····,b n ],[x3,y3,c 1, c 2, c 3, c 4, c 5, c 6, ····

[0023] ·,c n ],·····,[x m ,y m ,z 1, z 2, z 3, z 4, z 5, z 6, ·····,z n ]]

[0024] In the formula, x, y represent the position of the key point coordinates in the image, a, b, c represent different key point descriptors, n represents the length of the descriptor, and m represents the number of key points.

[0025] In a more specific technical solution, S2 includes:

[0026] S21, the mobile robot moves to an unexplored location based on the planning navigation algorithm;

[0027] S22. Pull RGBD data from the depth camera, execute the image segmentation algorithm, obtain the spatial position, shape and size, target type and segmented image of sensitive targets, and use sensitive targets that are not identified by the segmentation model as obstacles to build and store the semantic map.

[0028] In the process of constructing the semantic map of the present invention, a semantic grid map containing information such as item type, location, obstacles and item images is constructed based on an image segmentation algorithm, thereby providing map information for the mobile robot to perform exploration and path planning. In addition, the efficiency of item search is greatly improved by memory search of the semantic map.

[0029] In more specific technical solutions, S3 includes:

[0030] S31, when the segmentation model reads a new sensitive item, the new sensitive item is matched with the target item in terms of category. If the category output by the segmentation model is the same as the sensitive target, a re-matching operation is performed;

[0031] S32. If the target item is presented in the form of text, the image encoder of the CLIP model is used to perform feature encoding on the image of the new item to obtain a target text encoding vector, and the cosine similarity between the image encoding result vector and the target text encoding vector is calculated. When the cosine similarity is greater than a preset threshold, the currently discovered sensitive item is taken as the target item;

[0032] S33, when the target object is presented in the form of an image, use the SuperPoint model to extract features from the segmented image of the target object, input the feature point information of the segmented image and the feature point information of the target image into the SuperGlue model for matching, and output the number of matching key points. When the proportion of the number of key points is greater than a preset ratio threshold, it is determined that the currently discovered sensitive item is the target item;

[0033] S34, if the target object has been found, the mobile robot plans a moving path according to the semantic grid map and the target position to reach the location of the target object;

[0034] S35, updating the information in the semantic grid map, and executing S31 to S34 in a loop until the target item to be screened is found or the work area is explored completely.

[0035] In a more specific technical solution, in S32, the following logic is used to encode the image result vector [a1, a2, a3, a4, ..., an ] and the text encoding result vector [b 1i ,b2,b3,b4,····,b n ], calculate the cosine similarity:

[0036]

[0037] The present invention realizes the matching of target objects with discovered objects by matching image feature points and calculating feature similarity of image-text pairs according to different presentation modes of target objects, thereby improving the accuracy of searching for objects.

[0038] In a more specific technical solution, in S34, if the target object is not found, the mobile robot is controlled to continue exploring the unexplored area according to the semantic grid map.

[0039] In a more specific technical solution, S5 includes:

[0040] S51, determine the category of the target object, and search the semantic grid map to see if there are any objects of the same category that have been found. If so, jump to S52;

[0041] S52, when it is found that there are items of the same category in the semantic grid map, S31 and S32 are executed for the image target and the text target respectively to determine whether there is a found item consistent with the target item in the semantic grid map. If so, the mobile robot is controlled to plan a moving path according to the semantic grid map and the target position to reach the vicinity of the target object;

[0042] S53: If the target object is not found in the semantic grid map, the mobile robot is controlled to continue exploring the unexplored area according to the generated semantic grid map.

[0043] In a more specific technical solution, a mobile robot target search system includes:

[0044] The feature encoding module is used to obtain the description of the target object, and to perform feature encoding on the target object according to its presentation form to obtain a key point matrix;

[0045] The grid map construction module is used to use the mobile robot to execute the planning and navigation algorithm, explore the unknown environment, read the RGBD camera data according to the key point matrix, execute the image segmentation algorithm, build the semantic grid map, and determine the sensitive targets. The grid map construction module is connected with the feature encoding module;

[0046] A matching module is used to use a mobile robot to match sensitive targets with target items and determine target items to be screened. The matching module is connected to a grid map construction module.

[0047] The primary screening module is used to screen and manually intervene in the found target items to be screened, and obtain the primary screening results. The primary screening module is connected to the matching module;

[0048] The secondary screening module is used to conduct a secondary search for the target item based on the primary screening result, find and determine the final target item in the cached semantic grid map, and the secondary screening module is connected to the primary screening module.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] The present invention uses a depth camera and an image semantic segmentation algorithm to build a semantic map, which can be used for autonomous robot navigation and can be used as memory storage to achieve rapid search in the same scene. The present invention is based on key point matching and image text matching methods, which are suitable for accurate matching of target objects (text descriptions or images) and objects discovered by the camera.

[0051] In the process of constructing the semantic map of the present invention, a semantic grid map containing information such as item type, location, obstacles and item images is constructed based on an image segmentation algorithm, thereby providing map information for the mobile robot to perform exploration and path planning. In addition, the efficiency of item search is greatly improved by memory search of the semantic map.

[0052] The present invention realizes the matching of target objects with discovered objects by matching image feature points and calculating feature similarity of image-text pairs according to different presentation modes of target objects, thereby improving the accuracy of searching for objects.

[0053] The present invention solves the technical problems existing in the prior art of dependence on real scene maps, high cost of sensor equipment, and high data storage overhead and data processing complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of basic steps of a mobile robot target search method according to Embodiment 1 of the present invention;

[0055] Figure 2 This is a schematic diagram of specific steps for encoding the characteristics of a target object according to Embodiment 1 of the present invention;

[0056] Figure 3 This is a schematic diagram of data flow processing for the first autonomous search for target objects and construction of semantic maps in Example 1 of the present invention;

[0057] Figure 4 This is a schematic diagram of specific steps for constructing a semantic grid map according to Embodiment 1 of the present invention;

[0058] Figure 5This is a schematic diagram of data flow processing for the second autonomous search for target objects and semantic map construction according to Embodiment 1 of the present invention;

[0059] Figure 6 This is a schematic diagram of the storage format of the semantic grid map according to Embodiment 1 of the present invention;

[0060] Figure 7 A schematic diagram of specific steps for matching target items in Example 1 of the present invention;

[0061] Figure 8 Schematic diagram of specific steps of screening and manual intervention in Example 1 of the present invention;

[0062] Fig. 9 This is a schematic diagram of specific steps for searching for a target object in Example 1 of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Example 1

[0065] like Figure 1 As shown, a mobile robot target search method provided by the present invention includes the following basic steps:

[0066] S1. Obtain the description of the target object and encode its features according to the presentation forms of different target objects;

[0067] like Figure 2 As shown, in this embodiment, step S1 of encoding the characteristics of the target object includes the following specific steps:

[0068] S11, using the text encoder of the CLIP model to feature encode the text in the text target, and the encoded text data is represented by a vector;

[0069] In this embodiment, for a text target, such as "a yellow pen on the table", the text data is feature encoded using a text encoder of the CLIP model, and the encoded text data is represented by a vector, for example: [0.238, 0.366, 0.477, 0.843, 0.266, ...];

[0070] S12, for the image data, using the SuperPoint model to extract key points of the image data;

[0071] In this embodiment, the key point representation is a mathematical matrix, which represents the position of the key point in the image and the descriptor of the key point;

[0072] In this embodiment, the SuperPoint model is used to extract key points from image data. The key point representation is a mathematical matrix that represents the position of the key point in the image and the descriptor of the key point [[x1, y2, a 1, a 2, a 3, a 4, a 5, a 6, ·····,a n ],[x2,y2,b 1, b 2, b 3, b 4, b 5, b 6, ·····,b n ],[x3,y3,c 1, c 2, c 3, c 4, c 5, c 6, ·····,c n ],·····,[x m ,y m ,z 1, z 2, z 3, z 4, z 5, z 6, ·····,z n ]], where x, y represent the position of the key point coordinates in the image, a, b, c represent different key point descriptors, n represents the length of the descriptor, and m represents the number of key points;

[0073] S13, save the feature encoding results of the image and text in the memory for subsequent program reading.

[0074] S2, the mobile robot starts to explore and build a semantic grid map;

[0075] like Figure 3 As shown, in this embodiment, the mobile robot executes a planning and navigation algorithm to explore an unknown environment, reads RGBD camera data, executes an image segmentation algorithm, and constructs a semantic grid map.

[0076] like Figure 4 and Figure 5 As shown, in this embodiment, step S2 of constructing a semantic grid map includes the following specific steps:

[0077] S21, the mobile robot moves to an unexplored location based on the exploration navigation algorithm;

[0078] S22, pull the RGBD data of the depth camera, execute the image segmentation algorithm, obtain the spatial position, shape and size, target type and segmented image of sensitive targets, and regard those that cannot be recognized by the segmentation model as obstacles; based on the above data, build and store the semantic map;

[0079] like Figure 6 As shown, in this embodiment, the RGBD data of the depth camera is pulled, and the image segmentation algorithm is executed to obtain the spatial position, shape and size, target type and segmented image of sensitive targets (such as hair dryers, chargers, mobile phones and other common items in the home environment). Anything that cannot be recognized by the segmentation model is considered an obstacle. Based on the above data, a semantic map with a granularity of 5cm*5cm is constructed. The map is stored in the form of a digital matrix, where the matrix has a total of C+4 channels. The first C channels represent different object categories, and the last 4 represent obstacles, explored areas, and the current and previous positions of the robot.

[0080] S3, after the robot finds the sensitive target, it matches it with the target object;

[0081] like Figure 7 As shown, in this embodiment, step S3 of target item matching includes the following basic steps:

[0082] S31. If the segmentation model reads a new sensitive item, the newly discovered item is matched with the target item in category. If the category output by the segmentation model is the same as the target, the next step of matching is performed;

[0083] S32. If the target item is presented in text, use the CLIP image encoder to perform feature encoding on the newly discovered item image, and calculate the cosine similarity between the result vector of the image encoding and the target text encoding result output by the document generator. When the calculated result is greater than a threshold, the currently discovered sensitive item is the target item.

[0084] In this embodiment, if the target object is presented in text, the CLIP image encoder is used to encode the features of the newly discovered object image, and the cosine similarity between the image encoding result vector and the target text encoding result output by the document encoder is calculated. For example, if the result of the image encoder is [a1, a2, a3, a4, ..., a n ], the result of the text encoder is [b1,b2,b3,b4,····,b n ], the calculation formula of cosine similarity is When the calculated result is greater than the threshold, the currently discovered sensitive item is the target item.

[0085] S33. If the target object is presented in an image, the SuperPoint model is used to extract features from the segmented image, and the extracted feature point information and the feature point information extracted from the target image are input into the SuperGlue model for matching, and the number of key points that can be matched is output. When the proportion of the number of key points that can be matched is greater than a threshold, the currently discovered sensitive object is the target object;

[0086] S34, if the target object has been found, the mobile robot can plan a moving path according to the semantic map and the target position to reach the vicinity of the target object. If it has not been found, the mobile robot will continue to explore the unexplored area according to the generated semantic grid map;

[0087] S35, updating the information in the semantic map, and repeating the above steps S31 to S34 until the target object is found or the work area is explored completely.

[0088] S4, after the exploration of the working area is completed, the discovered targets are screened and manually intervened;

[0089] like Figure 8 As shown, in this embodiment, the step S4 of screening and manual intervention also includes the following basic steps:

[0090] S41, when the working area has been explored and the target object has not been found, the object with the highest matching value between the currently found sensitive object and the target object is taken as the target object;

[0091] S42: The object image is sent to the requester, who is asked to determine whether the object is the target object. If it is, the mobile robot moves to the vicinity of the target object. Otherwise, a prompt indicating that the search task has failed is sent to the requester.

[0092] S5. When searching for the target object for the second time, the target object is searched in the cached semantic map.

[0093] like Fig. 9 As shown, in this embodiment, the step S5 of searching for the target object also includes the following basic steps:

[0094] S51, determining the category of the target object, searching the semantic map for objects of the same category that have been found, and if found, proceeding to the next step of determination;

[0095] S52, if it is found that there are similar objects in the semantic map, the same method as the above steps S31 and S32 is used for the image or text target to determine whether there is a found object consistent with the target object in the semantic map. If so, the mobile robot can plan a moving path according to the semantic map and the target position to reach the vicinity of the target object;

[0096] S53: If the target object is not found in the semantic map, the mobile robot continues to explore the unexplored area according to the generated semantic grid map.

[0097] In summary, the present invention has the following advantages over the prior art:

[0098] The present invention uses a depth camera and an image semantic segmentation algorithm to build a semantic map, which can be used for autonomous robot navigation and can be used as memory storage to achieve rapid search in the same scene. The present invention is based on key point matching and image text matching methods, which are suitable for accurate matching of target objects (text descriptions or images) and objects discovered by the camera.

[0099] In the process of constructing the semantic map of the present invention, a semantic grid map containing information such as item type, location, obstacles and item images is constructed based on an image segmentation algorithm, thereby providing map information for the mobile robot to perform exploration and path planning. In addition, the efficiency of item search is greatly improved by memory search of the semantic map.

[0100] The present invention realizes the matching of target objects with discovered objects by matching image feature points and calculating feature similarity of image-text pairs according to different presentation modes of target objects, thereby improving the accuracy of searching for objects.

[0101] The present invention solves the technical problems existing in the prior art of dependence on real scene maps, high cost of sensor equipment, and high data storage overhead and data processing complexity.

[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A mobile robot target search method, characterized in that: The method comprises: S1. Obtain a description of a target object, and perform feature encoding on the target object according to the presentation form of the target object to obtain a key point matrix; S2. Use the mobile robot to execute the planning and navigation algorithm to explore the unknown environment, read the RGBD camera data according to the key point matrix, execute the image segmentation algorithm, build the semantic grid map, and determine the sensitive targets; S3, using the mobile robot to perform a matching operation on the sensitive target and the target object to determine the target object to be screened; S4, screening and manual intervention mechanisms are performed on the target items to be screened that have been found to obtain a screening result; S5. According to the first screening result, a second search is performed on the target item, and a final target item is found and determined in the cached semantic grid map.

2. A mobile robot target search method according to claim 1, characterized in that: The S1 includes: S11, obtaining a text target in the target item description, and encoding text features using a text encoder of a CLIP model according to text data in the text target to obtain a text data vector representation; S12, for the image data in the target item description, use the SuperPoint model to extract key points to obtain the key point matrix as the image feature encoding result and the text feature encoding result; S13, saving the image feature encoding result and the text feature encoding result.

3. A mobile robot target search method according to claim 2, characterized in that: In S12, the representation information of the key point matrix includes: the position of the key point in the image and the key point descriptor.

4. A mobile robot target search method according to claim 2, characterized in that: In S12, according to the following logic, the key point matrix is ​​used to represent the position of the key point image and the key point descriptor respectively: <h2 style=";text-align:left;direction:ltr">[[x1,y2,a1,a2,a3,a4,a5,a6,·····,a<h2 style=";text-align:left;direction:ltr"> n <h2 style=";text-align:left;direction:ltr"> ],[x2,y2,b1,b2,b3,b4,b5,b6,·····,b<h2 style=";text-align:left;direction:ltr"> n <h2 style=";text-align:left;direction:ltr"> ],[x3,y3,c1,c2,c3,c4,c5,c6,·····,c<h2 style=";text-align:left;direction:ltr"> n <h2 style=";text-align:left;direction:ltr"> ],…,[x<h2 style=";text-align:left;direction:ltr"> m <h2 style=";text-align:left;direction:ltr"> ,y<h2 style=";text-align:left;direction:ltr"> m <h2 style=";text-align:left;direction:ltr"> ,z1,z2,z3,z4,z5,z6,·····,z<h2 style=";text-align:left;direction:ltr"> n <h2 style=";text-align:left;direction:ltr"> ]] Wherein, x, y represent the position of the key point coordinates in the image, a, b, c represent different key point descriptors, n represents the length of the descriptor, and m represents the number of key points.

5. A mobile robot target search method according to claim 1, characterized in that: The S2 includes: S21, the mobile robot moves to an unexplored position based on the planning navigation algorithm; S22. Pull the RGBD data from the depth camera, execute the image segmentation algorithm, obtain the spatial position, shape and size, target type and segmented image of the sensitive target, and use the sensitive target that is not recognized by the segmentation model as an obstacle to build and store a semantic map.

6. A mobile robot target search method according to claim 1, characterized in that: The S3 includes: S31, when the segmentation model reads a new sensitive item, the new sensitive item is matched with the target item in terms of category. If the category output by the segmentation model is the same as that of the sensitive target, a re-matching operation is performed; S32. If the target item is presented in the form of text, use the image encoder of the CLIP model to perform feature encoding on the new item image to obtain a target text encoding vector, calculate the cosine similarity between the image encoding result vector and the target text encoding vector, and when the cosine similarity is greater than a preset threshold, take the currently discovered sensitive item as the target item; S33, when the target item is presented in the form of an image, use the SuperPoint model to extract features from the segmented image of the target item, input the segmented image feature point information and the target image feature point information into the SuperGlue model for matching, output the number of matching key points, and when the proportion of the number of key points is greater than a preset ratio threshold, determine that the currently discovered sensitive item is the target item; S34, if the target object has been found, the mobile robot plans a moving path according to the semantic grid map and the target position to reach the position of the target object; S35, updating the information in the semantic grid map, and looping through S31 to S34 until the target item to be screened is found or the work area is explored completely.

7. A mobile robot target search method according to claim 6, characterized in that: In S32, the image encoding result vector [a1, a2, a3, a4, ..., a n ] and the text encoding result vector [b1,b2,b3,b4,…,b n ], calculate the cosine similarity:

8. A mobile robot target search method according to claim 6, characterized in that: In S34, if the target object is not found, the mobile robot is controlled to continue exploring the unexplored area according to the semantic grid map.

9. A mobile robot target search method according to claim 1, characterized in that: The S5 includes: S51, determining the category of the target object, searching the semantic grid map to see whether there are already found objects of the same category, if so, jumping to S52; S52, when it is found that the same category of items exist in the semantic grid map, S31 and S32 are respectively executed for the image target and the text target to determine whether there is a found item consistent with the target item in the semantic grid map, and if so, the mobile robot is controlled to plan a moving path according to the semantic grid map and the target position to reach the vicinity of the target object; S53: If the target object is not found in the semantic grid map, control the mobile robot to continue exploring the unexplored area according to the generated semantic grid map.

10. A mobile robot target search system, characterized in that: The system comprises: A feature encoding module is used to obtain a description of a target object, and to perform feature encoding on the target object according to the presentation form of the target object to obtain a key point matrix; A grid map construction module is used to use a mobile robot to execute a planning and navigation algorithm to explore an unknown environment, read RGBD camera data according to the key point matrix, execute an image segmentation algorithm, construct a semantic grid map, and determine sensitive targets. The grid map construction module is connected to the feature encoding module; A matching module, used to use the mobile robot to perform a matching operation on the sensitive target and the target object to determine the target object to be screened, the matching module being connected to the grid map construction module; A primary screening module, used to perform screening and manual intervention on the target items to be screened that have been found to obtain a primary screening result, and the primary screening module is connected to the matching module; The secondary screening module is used to conduct a secondary search for the target item according to the primary screening result, and to find and determine the final target item in the cached semantic grid map. The secondary screening module is connected to the primary screening module.

Images