Object searching method and device for robot and robot for cleaning

Abstract

The embodiment of the invention discloses an object searching method and device for a robot and the robot for cleaning. The method comprises the steps that received object searching voice is recognized, object searching feature information of a to-be-searched object is obtained, and the object searching feature information comprises the object category of the to-be-searched object; searching an object according to the object category to obtain a reference object matched with the object category and an object confidence corresponding to the reference object; if it is detected that the object searching feature information further comprises reference feature information except the object category, adjusting the object confidence coefficient according to the reference feature information; and determining a to-be-searched object from the reference objects according to the adjusted object confidence. According to the embodiment of the invention, the confidence degree comprehensively reflects the matching degree of the reference object and the multiple pieces of feature information, so that the accuracy of determining the to-be-searched object from the reference object through the adjusted object confidence degree is higher, and the object searching accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a method and device for robot object finding, and a robot for cleaning. Background Art

[0002] At present, smart home appliances are becoming more and more common. For example, cleaning robots are smart home appliances that can automatically clean the floor. They integrate modern sensing technology, artificial intelligence, automatic control and other technologies, and can replace manual work to clean the floor of the home. However, as users' quality of life improves, users require cleaning robots to have other functions besides cleaning, such as finding objects.

[0003] In the related art, the traditional method of robot object finding is to find objects directly through feature matching. Most of the matching features are relatively simple, resulting in low matching accuracy, which in turn leads to low object finding accuracy. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, the embodiments of the present application provide a method and device for robot searching for objects, and a robot for cleaning, which can improve the accuracy of searching for objects.

[0005] According to one aspect of an embodiment of the present application, a method for robot object finding is provided, comprising: recognizing a received object finding voice to obtain object finding feature information of an object to be found, wherein the object finding feature information includes an object category of the object to be found; searching for objects according to the object category to obtain a reference object matching the object category and an object confidence corresponding to the reference object; if it is detected that the object finding feature information also includes reference feature information other than the object category, adjusting the object confidence according to the reference feature information; and determining the object to be found from the reference objects according to the adjusted object confidence.

[0006] In some embodiments, the object confidence is adjusted according to the reference feature information, including: if the reference feature information includes the color of the object to be found, obtaining a reference picture of the reference object, the reference picture being captured from an image collected during the robot's object searching process; determining the color confidence of the reference object according to the reference picture and the color through a color classification network; and adjusting the object confidence based on the color confidence.

[0007] In some embodiments, the object confidence is adjusted according to the reference feature information, including: if the reference feature information includes the shape of the object to be found, obtaining a reference picture of the reference object, wherein the reference picture is captured from an image collected during the robot's object searching process; determining the shape confidence of the reference object according to the reference picture and the shape through a shape classification network; and adjusting the object confidence based on the shape confidence.

[0008] In some embodiments, the adjusting the object confidence based on the reference feature information includes: if the reference feature information includes a candidate area of ​​the object to be found, determining a target reference object within the candidate area from the reference objects; and increasing the object confidence corresponding to the target reference object.

[0009] In some embodiments, the adjusting the object confidence according to the reference feature information includes: if the reference feature information includes indication information corresponding to the picture of the object to be found, obtaining a reference picture of the reference object, wherein the reference picture is captured from an image collected during the robot's object searching process; and obtaining a picture of the object to be found based on the indication information; calculating the similarity between the picture of the object to be found and the reference picture of the reference object; if the similarity is greater than a preset similarity threshold, increasing the object confidence of the reference object corresponding to the similarity.

[0010] In some embodiments, the recognition of the received object-finding voice to obtain the object-finding feature information of the object to be found includes: converting the object-finding voice into an object-finding text; inputting the object-finding text into a preset text analysis model for recognition to obtain the object-finding feature information of the object to be found.

[0011] In some embodiments, the robot is equipped with a camera device; the object search is performed according to the object category to obtain a reference object that matches the object category and an object confidence level corresponding to the reference object, including: during the process of the robot searching for objects, using the camera device to collect images of the area where the robot is located in real time; target detection is performed on the image to obtain a reference object that matches the object category, and obtain an object confidence level corresponding to the reference object.

[0012] In some embodiments, after determining the object to be found from the reference objects according to the adjusted object confidence, it also includes: determining the position of the object to be found according to the position information of the robot in a preset room map; and prompting the position of the object to be found in the preset room map.

[0013] According to one aspect of an embodiment of the present application, there is provided a device for robot object finding, comprising: a feature acquisition module, configured to recognize a received object finding voice, and obtain object finding feature information of an object to be found, wherein the object finding feature information includes the object category of the object to be found; an object finding module, configured to search for objects according to the object category, and obtain a reference object matching the object category and an object confidence corresponding to the reference object; an adjustment module, configured to adjust the object confidence according to the reference feature information if it is detected that the object finding feature information also includes reference feature information other than the object category; and a determination module, configured to determine the object to be found from the reference objects according to the adjusted object confidence.

[0014] According to one aspect of an embodiment of the present application, a robot for cleaning is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the above-mentioned method for robot finding objects.

[0015] In the technical solution provided in the embodiment of the present application, by recognizing the received object-finding voice, object-finding feature information including the object category of the object to be found is obtained, and then the objects are screened according to the object category to determine the reference object matching the object category and the object confidence corresponding to the reference object. The object confidence is then adjusted by reference feature information other than the object category to determine the object to be found from the reference object according to the adjusted object confidence. In this way, compared to matching the object to be found only by a single feature, the present application first determines the reference object and the object confidence corresponding to the reference object according to the object category, and then further adjusts the object confidence by other reference feature information, combining multiple feature information so that the confidence can comprehensively reflect the degree of matching between the reference object and multiple feature information, thereby making the accuracy of the object to be found determined from the reference object by the adjusted object confidence higher, thereby improving the accuracy of finding objects.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 is a schematic diagram of an implementation environment for robot object finding shown in an exemplary embodiment of the present application;

[0019] Figure 2 is a schematic diagram of the structure of a robot shown in an exemplary embodiment of the present application;

[0020] Figure 3 is a flow chart of a method for robot object finding shown in an exemplary embodiment of the present application;

[0021] Figure 4 is a flow chart of a method for robot object finding shown in another exemplary embodiment of the present application;

[0022] Figure 5 is a flow chart of a method for robot object finding shown in another exemplary embodiment of the present application;

[0023] Figure 6 is a flow chart of a method for robot object finding shown in another exemplary embodiment of the present application;

[0024] Figure 7 is a flow chart of a method for robot object finding shown in another exemplary embodiment of the present application;

[0025] Figure 8 is a block diagram of a device for robot object finding shown in an exemplary embodiment of the present application;

[0026] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing the robot of the embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0028] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0029] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0030] The term "multiple" as used in this application refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0031] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of an implementation environment involved in the present application. The implementation environment includes a house 10 and a robot 11.

[0032] It should be noted that the house 10 may have multiple areas. Each area may be partitioned by a wall, for example, each room is an area. In some embodiments, the house may include multiple room areas such as a bedroom, a living room, a kitchen, and a bathroom.

[0033] The robot 11 is placed in the house 10. Multiple areas in the house 10 can be used as the working space of the robot 11.

[0034] The robot 11 may be an intelligent robot with mobile functions, such as a cleaning robot, a robot dog, a service robot, etc., which is not limited here.

[0035] The cleaning robot can clean and search for items in any area of ​​the house 10 .

[0036] See also Figure 2 , Figure 2 This is a schematic diagram of the robot.

[0037] The robot 11 includes: a robot body 21 and a camera device 22. The robot body 21 communicates with the camera device 22 via a wired or wireless network.

[0038] The robot body 21 is the main body device of the robot. It is a smart device that automatically completes the work of finding objects in the room by relying on artificial intelligence technology.

[0039] The camera device 22 can be arranged on the surface of the cleaning robot body 21. It is used to collect images of the area where the robot is located during the movement.

[0040] The camera device 22 may include one or more cameras to achieve multi-directional image acquisition.

[0041] See also Figure 3 , Figure 3 is a flowchart of a method for robot object finding shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is specifically executed by the robot 11 in the implementation environment. It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0042] like Figure 3 As shown, in an exemplary embodiment, the method for robot object finding includes at least steps S310 to S340, which are described in detail as follows:

[0043] Step S310, recognizing the received object-finding voice to obtain object-finding feature information of the object to be found; wherein the object-finding feature information includes the object category of the object to be found.

[0044] Step S320 , searching for objects according to the object category, obtaining reference objects matching the object category and object confidences corresponding to the reference objects.

[0045] Step S330: If it is detected that the object finding feature information also includes reference feature information other than the object category, the object confidence is adjusted according to the reference feature information.

[0046] Step S340 , determining the object to be found from the reference objects according to the adjusted object confidence.

[0047] In this embodiment, the received object-finding voice is recognized to obtain object-finding feature information including the object category of the object to be found, and then the objects are screened according to the object category to determine the reference object that matches the object category and the object confidence corresponding to the reference object. The object confidence is then adjusted by reference feature information other than the object category to determine the object to be found from the reference objects according to the adjusted object confidence. In this way, compared to matching the object to be found only by a single feature, the present application first determines the reference object and the object confidence corresponding to the reference object according to the object category, and then further adjusts the object confidence by other reference feature information, and combines multiple reference feature information so that the confidence can comprehensively reflect the degree of matching between the reference object and multiple feature information, thereby making the accuracy of the object to be found determined from the reference object by the adjusted object confidence higher, thereby improving the accuracy of the object-finding.

[0048] Furthermore, the received object-finding voice is recognized to obtain object-finding feature information of the object to be found, including: converting the object-finding voice into object-finding text; inputting the object-finding text into a preset text analysis model for recognition to obtain object-finding feature information of the object to be found.

[0049] In this way, the preset text analysis model is used to obtain the object finding feature information of the object to be found from the object finding text converted from the object finding voice, thereby realizing the screening of the object finding feature information in the object finding text, so as to facilitate the object finding based on the object finding feature and improve the accuracy of the object finding.

[0050] It should be noted that the object-finding voice is the voice that the user makes to the robot, which is used to instruct or control the robot to find objects.

[0051] Exemplarily, when the voice uttered by the user contains similar sentences such as "want + object", "want + adjective + object", "find + object", "find + adjective + object", "search + object" or "search + adjective + object", the voice is the voice of finding objects.

[0052] It should be noted that the robot has a voice collection module. The voice collection module is used to collect the user's voice of finding objects. After the robot receives the voice of finding objects collected by the voice collection module, it converts the voice of finding objects into text through a preset voice recognition algorithm. The text of finding objects is the written text corresponding to the voice of finding objects. Among them, the language recognition algorithm can adopt a hybrid model based on a deep neural network, such as a hybrid model formed by a long short-term memory (LSTM) network combined with a connectionist temporal classification (CTC) network.

[0053] The preset text analysis model may be a traditional computational text analysis model, which is used to analyze and process the object search text through text analysis software, machine learning, and natural language processing algorithms.

[0054] Preferably, the preset text analysis model can be a large language model. A large language model is a machine learning model based on deep learning technology, which generates and understands natural language by learning a large amount of text data. In actual application, the large language model can convert the object-finding text into a high-dimensional vector, and capture the deep semantics in the object-finding text from the high-dimensional vector to determine whether the robot needs to perform the task of finding the object, and when the object-finding task needs to be performed, obtain a detailed description of the search target, that is, the object-finding feature information. Compared with traditional computational text analysis models, large language models have a higher accuracy in understanding natural language.

[0055] The preset text analysis model can be used to extract object-finding control information and object-finding feature information from the object-finding voice.

[0056] The object-finding control information includes control information such as "want", "find", "seek", "need", etc. Only after the text analysis module analyzes and determines that the object-finding text contains the object-finding control information, the robot will execute the object-finding task of this application.

[0057] The object search feature information includes one or more of the object category of the object to be found, the color of the object to be found, the shape of the object to be found, the candidate area of ​​the object to be found, and the indication information corresponding to the picture of the object to be found. Among them, the candidate area of ​​the object to be found is used to characterize the area where the object to be found is most likely to exist. The indication information corresponding to the picture of the object to be found is used to indicate the picture of the object to be found. The object category of the object to be found is the characteristic information that must be included in the object search feature information.

[0058] For example, the text converted from the object-finding speech by the preset speech recognition algorithm is "I am looking for the blue socks in the bedroom." The following three object-finding feature information can be extracted from the object-finding speech by the preset text analysis model: "bedroom", "blue", "socks". Among them, "socks" is the object category of the object to be found; "blue" is the color of the object to be found; and "bedroom" is the candidate area of ​​the object to be found.

[0059] Another example: the text converted from the object-finding voice by the preset voice recognition algorithm is "Find the red square box in the next uploaded picture." The following four object-finding feature information can be extracted from the object-finding voice through the preset text analysis model: "next uploaded picture", "red", "square" and "box". Among them, "box" is the object category of the object to be found; "red" is the color of the object to be found; "square" is the shape of the object to be found; "next uploaded picture" is the indication information corresponding to the picture of the object to be found; then the next picture uploaded by the user is the picture of the object to be found.

[0060] Furthermore, object searching is performed according to the object category to obtain a reference object that matches the object category and an object confidence level corresponding to the reference object, including: during the robot's object searching process, using a camera device to capture images of the area where the robot is located in real time; performing target detection on the image to obtain a reference object that matches the object category, and obtaining an object confidence level corresponding to the reference object.

[0061] In this way, by performing target detection on the images collected by the robot in real time, it is possible to determine in real time in the image the reference object that matches the object category and obtain the object confidence of the reference object, so that the object to be found determined according to the object confidence is the best match for the object category, thereby improving the accuracy of finding objects.

[0062] It should be noted that when the robot searches for an object, it searches for the object along a preset path. For example, when the robot is a cleaning robot, the preset path is the cleaning path of the cleaning robot.

[0063] If the object finding feature information does not include the candidate area of ​​the object to be found, the preset path is the cleaning path of all areas of the entire room. If the object finding feature information includes the candidate area of ​​the object to be found, the preset path can be the cleaning path of all areas of the entire room or only the cleaning path of the candidate area.

[0064] The candidate area can include one or more rooms, or it can be specific to the area where the furniture is located, such as: under the sofa or under the TV cabinet.

[0065] When the preset path is the cleaning path of the candidate area, if the object is found successfully, the robot completes the object-finding task, otherwise, the object-finding task is reset and the object-finding path is expanded to the cleaning path of all areas of the room. In this way, objects in the candidate area can be searched first, and the object-finding task is completed when the object is found successfully, otherwise the object-finding area is expanded. Compared with searching in all areas directly, it saves time and improves the efficiency of searching.

[0066] Furthermore, target detection is performed on the image to obtain a reference object that matches the object category, and an object confidence corresponding to the reference object is obtained, including: using a preset target detection algorithm to perform target detection on the image to obtain a reference object that matches the object category, and an object confidence corresponding to the reference object is obtained.

[0067] It should be noted that matching with the object category indicates that the category of the reference object is the same as or similar to the object category of the object to be found.

[0068] After the image is detected using the preset object detection algorithm, the reference object that matches the object category is accompanied by a bounding box and an object confidence. The bounding box is used to frame the reference object. The object confidence is usually a floating point number between 0 and 1, which is used to represent the probability that the detected object belongs to the object category. For example, if the object detection algorithm detects a bottle in an image with an object confidence of 0.95, it means that there is a 95% certainty that there is indeed a bottle in the image.

[0069] The object detection algorithm identifies objects through the image features of objects in the object recognition database. The object recognition database initially covers the image features of common small objects. As the robot is used for a longer time, it will continuously optimize and expand the database through self-learning, so that it can recognize more types of lost objects.

[0070] Specifically, traditional target detection algorithms, target detection algorithms based on various deep learning, etc. can be used, without limitation here.

[0071] Optionally, the object confidence is adjusted according to the reference feature information, including: if the reference feature information includes the color of the object to be found, obtaining a reference picture of the reference object, the reference picture being captured from an image collected during the robot's object searching process; determining the color confidence of the reference object according to the reference picture and color through a color classification network; and adjusting the object confidence based on the color confidence.

[0072] In this way, the color classification network can identify whether the color of the reference object in the reference image is the same as the color of the object to be found, obtain the color confidence of the reference object, and then adjust the object confidence based on the color confidence. This realizes the combination of object category and color features to find objects, which improves the accuracy of finding objects compared to using a single feature to find objects.

[0073] It should be noted that the reference image of the reference object can be obtained by intercepting the bounding box where the reference object is located in the image.

[0074] Furthermore, the color confidence of the reference object is determined according to the reference image and the color through the color classification network, including: inputting the color and the reference image into the color classification network for color classification to obtain the color confidence of the reference object.

[0075] It should be noted that the color confidence is often a floating point number between 0 and 1, which is used to represent the probability that the color of the reference object in the reference image matches the color of the object to be found.

[0076] Further, adjusting the object confidence based on the color confidence includes: weighting the color confidence and the object confidence, and determining the weighted confidence as the adjusted object confidence.

[0077] If the reference feature information does not include the color of the object to be found, the object confidence is controlled to remain unchanged.

[0078] like Figure 4 As shown, in an exemplary embodiment, the method for robot object finding includes at least steps S401 to S409, which are described in detail as follows:

[0079] Step S401, recognize the received object-finding voice to obtain object-finding feature information of the object to be found; wherein the object-finding feature information includes the object category and the color of the object to be found.

[0080] Step S402 , searching for objects according to the object category, obtaining reference objects matching the object category and object confidences corresponding to the reference objects.

[0081] Step S403, determine whether the reference feature information includes the color of the object to be found. If yes, execute step S404. Otherwise, execute step S408.

[0082] Step S404, obtaining a reference image of the reference object; wherein the reference image is captured from images collected during the robot's object-seeking process.

[0083] Step S405: Input the color and the reference image into a color classification network for color classification to obtain the color confidence of the reference object, wherein the color confidence is used to indicate the probability that the color of the reference object in the reference image matches the color of the object to be found.

[0084] Step S406: weighting the color confidence and the object confidence.

[0085] Step S407: Determine the weighted confidence as the adjusted object confidence. Then execute step S409.

[0086] Step S408, controlling the object confidence to remain unchanged.

[0087] Step S409: determining the object to be found from the reference objects according to the adjusted object confidence.

[0088] In this embodiment, by recognizing the received object-finding voice, object-finding feature information including the object category of the object to be found is obtained, and then the objects are screened according to the object category to determine the reference object matching the object category and the object confidence corresponding to the reference object. Then, the color classification network can identify whether the color of the reference object in the reference image is the same as the color of the object to be found, and the color confidence of the reference object is obtained, and then the object confidence is adjusted based on the color confidence. Then, the object to be found is determined from the reference object according to the adjusted object confidence. In this way, compared with matching the object to be found only by a single feature, the present application first determines the reference object and the object confidence corresponding to the reference object according to the object category, and then further adjusts the object confidence by the color of the object to be found, combining the two features of object category and color to find objects, so that the object confidence can comprehensively reflect the matching degree between the category of the reference object and the object category of the object to be found and the color of the reference object and the color of the object to be found, so that the accuracy of the object to be found determined from the reference object by the adjusted object confidence is higher, thereby improving the accuracy of finding objects.

[0089] Optionally, the object confidence is adjusted according to reference feature information, including: if the reference feature information includes the shape of the object to be found, obtaining a reference picture of the reference object, the reference picture being captured from an image collected during the robot's object searching process; determining the shape confidence of the reference object according to the reference picture and shape through a shape classification network; and adjusting the object confidence based on the shape confidence.

[0090] In this way, the shape classification network can identify whether the shape of the reference object in the reference image is of the same type as the shape of the object to be found, obtain the shape confidence of the reference object, and then adjust the object confidence based on the shape confidence. This realizes the combination of object category and shape features to find objects, which improves the accuracy of finding objects compared to using a single feature to find objects.

[0091] Furthermore, the shape confidence of the reference object is determined according to the reference image and the shape through the shape classification network, including: inputting the shape and the reference image into the shape classification network for color classification to obtain the shape confidence of the reference object.

[0092] It should be noted that the shape confidence is often a floating point number between 0 and 1, which is used to indicate the probability that the shape of the reference object in the reference image matches the shape of the object to be found.

[0093] Further, adjusting the object confidence based on the shape confidence includes: weighting the shape confidence and the object confidence, and determining the weighted confidence as the adjusted object confidence.

[0094] If the reference feature information does not include the shape of the object to be found, the object confidence is controlled to remain unchanged.

[0095] like Figure 5 As shown, in an exemplary embodiment, the method for robot object finding includes at least steps S501 to S509, which are described in detail as follows:

[0096] Step S501, recognize the received object-finding voice to obtain object-finding feature information of the object to be found; wherein the object-finding feature information includes the object category and the shape of the object to be found.

[0097] Step S502 , searching for an object according to the object category, obtaining a reference object that matches the object category and an object confidence level corresponding to the reference object.

[0098] Step S503, determine whether the reference feature information includes the shape of the object to be found. If yes, execute step S504. Otherwise, execute step S508.

[0099] Step S504, obtaining a reference image of the reference object; wherein the reference image is captured from images collected during the robot's object-seeking process.

[0100] Step S505: input the shape and the reference image into a shape classification network for shape classification to obtain the shape confidence of the reference object, wherein the shape confidence is used to indicate the probability that the shape of the reference object in the reference image matches the shape of the object to be found.

[0101] Step S506: weighting the shape confidence and the object confidence.

[0102] Step S507: Determine the weighted confidence as the adjusted object confidence. Then execute step S509.

[0103] Step S508, controlling the object confidence to remain unchanged.

[0104] Step S509 : determining the object to be found from the reference objects according to the adjusted object confidence.

[0105] In this embodiment, by recognizing the received object-finding voice, object-finding feature information including the object category of the object to be found is obtained, and then the objects are screened according to the object category to determine the reference object matching the object category and the object confidence corresponding to the reference object. Then, the shape classification network can identify whether the shape of the reference object in the reference image is of the same category as the shape of the object to be found, obtain the shape confidence of the reference object, and then adjust the object confidence based on the shape confidence. Then, the object to be found is determined from the reference object according to the adjusted object confidence. In this way, compared with matching the object to be found only by a single feature, the present application first determines the reference object and the object confidence corresponding to the reference object according to the object category, and then further adjusts the object confidence by the shape of the object to be found, combines the object category and the shape of the two features to find, so that the object confidence can comprehensively reflect the matching degree between the category of the reference object and the object category of the object to be found and the shape of the reference object and the shape of the object to be found, so that the accuracy of the object to be found determined from the reference object by the adjusted object confidence is higher, thereby improving the accuracy of finding objects.

[0106] Optionally, the object confidence is adjusted according to reference feature information, including: if the reference feature information includes the color of the object to be found and the shape of the object to be found, then obtaining a reference picture of the reference object, the reference picture being captured from an image collected during the robot's object searching process; determining the color confidence of the reference object according to the reference picture and color through a color classification network; determining the shape confidence of the reference object according to the reference picture and shape through a shape classification network; and adjusting the object confidence based on the color confidence and the shape confidence.

[0107] Further, adjusting the object confidence based on the color confidence and the shape confidence includes: weighting the color confidence, the shape confidence and the object confidence, and determining the weighted confidence as the adjusted object confidence.

[0108] The weights can be set by the user. If the user does not set them, the weights are the default weights.

[0109] It should be noted that whether the object-finding feature information includes the shape of the object to be found and whether the object-finding feature information includes the color of the object to be found can be judged together. When judging together, if they exist at the same time, they can be weighted based on the obtained shape confidence, color confidence, and object confidence. If they do not exist at the same time, they are weighted separately. It is also possible to directly judge whether the object-finding feature information includes the shape of the object to be found and whether the object-finding feature information includes the color of the object to be found separately to weight them separately.

[0110] Exemplarily, in an exemplary embodiment, a method for robot object finding includes at least the following steps:

[0111] The received object-finding voice is recognized to obtain object-finding feature information of the object to be found, wherein the object-finding feature information includes the object category of the object to be found. In addition, reference feature information other than the object category includes the shape of the object to be found or the color of the object to be found.

[0112] Objects are searched according to the object category to obtain reference objects that match the object category and the object confidence corresponding to the reference objects.

[0113] Determine whether the object-finding feature information also includes the color of the object to be found and the shape of the object to be found. If so, obtain a reference image of the reference object, which is captured from an image collected during the robot's object-finding process. Determine the color confidence of the reference object based on the reference image and color through a color classification network. Determine the shape confidence of the reference object based on the reference image and shape through a shape classification network. Adjust the object confidence based on the color confidence and the shape confidence. Then determine the object to be found from the reference objects based on the adjusted object confidence.

[0114] If the object-finding feature information does not include both the color and shape of the object to be found, determine whether the object-finding feature information also includes the color of the object to be found. If so, obtain a reference image of the reference object, which is captured from an image collected during the robot's object-finding process. Determine the color confidence of the reference object based on the reference image and color through a color classification network. Adjust the object confidence based on the color confidence. Then determine the object to be found from the reference objects based on the adjusted object confidence.

[0115] Otherwise, it is determined whether the reference feature information also includes the shape of the object to be found. If so, a reference image of the reference object is obtained, which is captured from the image collected during the robot's object-finding process. The shape confidence of the reference object is determined based on the reference image and shape through a shape classification network. The object confidence is adjusted based on the shape confidence. The object to be found is then determined from the reference objects based on the adjusted object confidence.

[0116] If the reference feature information includes neither the shape nor the color of the object to be found, the object confidence is controlled to remain unchanged, and the object to be found is determined from the reference objects according to the object confidence.

[0117] In this embodiment, compared to matching the object to be found only by a single feature, the present application first determines the reference object and the object confidence corresponding to the reference object according to the object category, and then further adjusts the object confidence according to the shape of the object to be found and the color of the object to be found. The object category and the three features of shape and color are combined to search for objects, so that the object confidence can comprehensively reflect the degree of match between the reference object and the object to be found from the three dimensions of category, shape and color, thereby making the accuracy of the object to be found determined from the reference object through the adjusted object confidence higher, thereby improving the accuracy of finding objects.

[0118] Optionally, the object confidence is adjusted according to the reference feature information, including: if the reference feature information includes a candidate area of ​​the object to be found, determining a target reference object within the candidate area from the reference objects; and increasing the object confidence corresponding to the target reference object.

[0119] In this way, by increasing the object confidence of the target reference object in the candidate area, the object confidence obtained by the object category is adjusted based on the candidate area where the object to be found is located. The object search is achieved by combining the two features of object category and candidate area, which improves the accuracy of object search compared to using a single feature to search.

[0120] It should be noted that if the reference object is not in the candidate area, the corresponding object confidence remains unchanged.

[0121] In some optional embodiments, the candidate area is a bedroom, and there are three reference objects in the room. They are: sock 1 located in the bedroom, sock 2 located in the bedroom, and sock 3 located in the living room. Then sock 1 located in the bedroom and sock 2 located in the bedroom are both located in the candidate area and are target reference objects. Then the object confidences corresponding to sock 1 located in the bedroom and sock 2 located in the bedroom are increased.

[0122] like Figure 6As shown, in an exemplary embodiment, the method for robot object finding includes at least steps S610 to S650, which are described in detail as follows:

[0123] Step S610, recognizing the received object-finding voice to obtain object-finding feature information of the object to be found; wherein the object-finding feature information includes the object category of the object to be found and the candidate area of ​​the object to be found.

[0124] Step S620 , searching for objects according to the object category, obtaining reference objects matching the object category and object confidences corresponding to the reference objects.

[0125] Step S630: When the reference feature information includes a candidate region of the object to be found, a target reference object located in the candidate region is determined from the reference objects.

[0126] Step S640: increase the object confidence level corresponding to the target reference object, and keep the object confidence levels corresponding to the remaining reference objects unchanged.

[0127] Step S650 , determining the object to be found from the reference objects according to the adjusted object confidence.

[0128] In this embodiment, by recognizing the received object-finding voice, object-finding feature information including the object category of the object to be found is obtained, and then the objects are screened according to the object category to determine the reference object that matches the object category and the object confidence corresponding to the reference object. The object confidence of the target reference object in the candidate area is increased, and the object confidence obtained by the object category is adjusted based on the candidate area where the object to be found is located. Then, the object to be found is determined from the reference objects according to the adjusted object confidence. In this way, compared with matching the object to be found only by a single feature, the object category and the candidate area of ​​the object to be found are combined for object search, so that the object confidence can comprehensively reflect the degree of match between the reference object and the object to be found from the two dimensions of category and area, thereby making the accuracy of the object to be found determined from the reference object by the adjusted object confidence higher, thereby improving the accuracy of object search.

[0129] Optionally, the object confidence is adjusted according to the reference feature information, including: if the reference feature information includes indication information corresponding to the picture of the object to be found, then obtaining a reference picture of the reference object, and obtaining the picture of the object to be found based on the indication information; the reference picture is captured from the image collected during the robot's object searching process; calculating the similarity between the picture of the object to be found and the reference picture of the reference object; if the similarity is greater than a preset similarity threshold, increasing the object confidence of the reference object corresponding to the similarity.

[0130] In this way, by increasing the object confidence of the reference object corresponding to the similarity when the similarity between the image of the object to be found and the reference image of the reference object is greater than a preset similarity threshold, the object confidence obtained by the object category is adjusted based on the reference object where the object to be found is located and the similarity of the object to be found. The object search is achieved by combining the two features of the object category and the image of the object to be found, which improves the accuracy of the object search compared to the use of a single feature.

[0131] In some embodiments, the indication information corresponding to the picture of the object to be found is used to indicate the picture of the object to be found. For example: when the indication information corresponding to the picture of the object to be found is "the next uploaded picture", the picture of the object to be found is the next picture uploaded by the user. For another example: when the indication information corresponding to the picture of the object to be found is "the third picture stored", the picture of the object to be found is the third picture stored by the robot. The specific form is not limited.

[0132] It should be noted that the similarity between the image of the object to be found and the reference image of the reference object is calculated by using a preset re-identification (Re-identification Algorithm) algorithm to calculate the similarity between the image of the object to be found and the reference image of the reference object.

[0133] If the similarity is less than or equal to a preset similarity threshold, the object confidence of the reference object corresponding to the similarity remains unchanged.

[0134] like Figure 7 As shown, in an exemplary embodiment, the method for robot object finding includes at least steps S710 to S780, which are described in detail as follows:

[0135] Step S710, recognize the received object-finding voice to obtain object-finding feature information of the object to be found; wherein the object-finding feature information includes the object category of the object to be found and indication information corresponding to the picture of the object to be found.

[0136] Step S720 , searching for objects according to the object category, obtaining reference objects that match the object category and object confidences corresponding to the reference objects.

[0137] Step S730, when the reference feature information includes indication information corresponding to the image of the object to be found, obtain a reference image of the reference object, and obtain the image of the object to be found based on the indication information. The reference image is captured from an image collected during the robot's object-finding process.

[0138] Step S740, calculating the similarity between the image of the object to be found and the reference image of the reference object.

[0139] Step S750, determine whether each similarity is greater than a preset similarity threshold. If yes, execute step S760; if not, execute step S770.

[0140] Step S760, increasing the object confidence of the reference object corresponding to the similarity. Then executing step S780.

[0141] Step S770 , controlling the object confidence of the reference object corresponding to the similarity to remain unchanged.

[0142] Step S780: determining the object to be found from the reference objects according to the adjusted object confidence.

[0143] In this embodiment, by recognizing the received object-finding voice, object-finding feature information including the object category of the object to be found is obtained, and then the objects are screened according to the object category to determine the reference object matching the object category and the object confidence corresponding to the reference object. When the similarity between the image of the object to be found and the reference image of the reference object is greater than the preset similarity threshold, the object confidence of the reference object corresponding to the similarity is increased, so that the object confidence obtained by the object category is adjusted based on the reference object where the object to be found is located and the similarity of the object to be found. In this way, compared with matching the object to be found only by a single feature, the object category and the image of the object to be found are combined to find the object, so that the object confidence can comprehensively reflect the matching degree between the reference object and the object to be found from the two dimensions of category and image, thereby making the accuracy of the object to be found determined from the reference object by the adjusted object confidence higher, thereby improving the accuracy of finding the object.

[0144] Further, determining the object to be found from the reference objects according to the adjusted object confidences includes: determining the maximum value among the object confidences as the target confidence; and determining the reference object corresponding to the target confidence as the object to be found.

[0145] It should be noted that in order to improve the accuracy of the object search, it can be determined whether the target confidence is greater than the preset reference confidence. If so, the object search is successful, and the reference object corresponding to the target confidence is determined as the object to be found. Otherwise, the object search fails.

[0146] Furthermore, after determining the object to be found from the reference objects according to the adjusted object confidence, it also includes: determining the position of the object to be found according to the position information of the robot in the preset room map; and prompting the position of the object to be found in the preset room map.

[0147] In this way, by determining the position of the object to be found based on the position information of the robot in the preset room map, the position of the object to be found is located, and then the position of the object to be found is prompted based on the positioning, so that the user can quickly find the object to be found based on the prompt, thereby improving the user's search efficiency.

[0148] It should be noted that the location of the object to be found is indicated in the preset room map, that is, the location of the object to be found is marked in the preset room map.

[0149] In some embodiments, if the robot has a screen, it can directly display the room map on the screen and mark the location of the object to be found in the room map. If the robot does not have a screen, the robot can send the room map and the location of the object to be found to a preset display device, such as a mobile phone, computer, TV, tablet or smart watch, so that the display device can display the room map with the location of the object to be found marked.

[0150] It should be noted that the position of the object to be found is determined according to the position information of the robot in the preset room map, that is, the position of the object to be found is determined according to the position information of the robot in the preset room map using a preset distance measurement algorithm.

[0151] Preferably, the preset ranging algorithm may be a monocular vision ranging algorithm.

[0152] The monocular vision ranging algorithm is a technology that estimates the distance between an object and a camera through image information collected by a single camera.

[0153] The monocular vision ranging algorithm is mainly based on the geometric information of the object in the image and the internal and external parameters of the camera to estimate the distance. Common methods include triangulation, depth of field measurement, and structured light measurement. Among them, triangulation is one of the most commonly used methods. Triangulation can estimate the distance based on the size and position information of the object to be found in the image, combined with the internal and external parameters of the camera device, such as focal length, optical center position, etc. Specifically, assume that there is an object to be found with a known width, which is placed at a certain distance from the camera. By shooting the object with a camera device and measuring its pixel width in the image, the formula for the camera focal length can be calculated, and then the distance of the object to be found from the camera device can be estimated, that is, the distance of the object to be found from the robot. After calculating the distance between the object to be found and the robot, the position information of the object to be found can be obtained by combining the direction of the object to be found and the position information of the robot in the preset room map.

[0154] It should be noted that the object search feature information also includes reference feature information other than the object category, for example: one or more of the color of the object to be found, the shape of the object to be found, the candidate area of ​​the object to be found, and the indication information corresponding to the picture of the object to be found.

[0155] It is possible to sequentially determine whether the object-finding feature information also includes the color of the object to be found, the shape of the object to be found, the candidate area of ​​the object to be found, and the indication information corresponding to the picture of the object to be found, and after each determination, adjust the object confidence according to the determination result. Then, the object to be found is determined from the reference objects according to the final adjusted object confidence. The determination order can be set according to the actual situation and is not limited here.

[0156] Specifically, the method for robot object finding includes at least the following steps:

[0157] The received object-finding voice is recognized to obtain object-finding feature information of the object to be found; wherein the object-finding feature information includes the object category of the object to be found and indication information corresponding to the picture of the object to be found.

[0158] Objects are searched according to the object category to obtain reference objects that match the object category and the object confidence corresponding to the reference objects.

[0159] Determine whether the object-finding feature information also includes the color of the object to be found and the shape of the object to be found. If so, obtain a reference image of the reference object, which is captured from an image collected during the robot's object-finding process. Determine the color confidence of the reference object based on the reference image and color through a color classification network. Determine the shape confidence of the reference object based on the reference image and shape through a shape classification network. Adjust the object confidence based on the color confidence and the shape confidence. Then determine the object to be found from the reference objects based on the adjusted object confidence.

[0160] If the object-finding feature information does not include both the color and shape of the object to be found, determine whether the object-finding feature information also includes the color of the object to be found. If so, obtain a reference image of the reference object, which is captured from an image collected during the robot's object-finding process. Determine the color confidence of the reference object based on the reference image and color through a color classification network. Adjust the object confidence based on the color confidence. Then determine the object to be found from the reference objects based on the adjusted object confidence.

[0161] On the contrary, that is, the object-finding feature information does not include the color of the object to be found, then determine whether the reference feature information also includes the shape of the object to be found. If so, obtain a reference image of the reference object, which is captured from the image collected during the robot's object-finding process. Determine the shape confidence of the reference object based on the reference image and shape through the shape classification network. Adjust the object confidence based on the shape confidence. Then determine the object to be found from the reference objects based on the adjusted object confidence.

[0162] If the reference feature information includes neither the shape nor the color of the object to be found, the confidence of the control object remains unchanged.

[0163] Determine whether the reference feature information includes the candidate area of ​​the object to be found. If the reference feature information includes the candidate area of ​​the object to be found, determine the target reference object in the candidate area from the reference objects. And increase the object confidence corresponding to the target reference object. If the reference feature information does not include the candidate area of ​​the object to be found, control the object confidence to remain unchanged.

[0164] Determine whether the reference feature information includes indication information corresponding to the picture of the object to be found. If the reference feature information includes indication information corresponding to the picture of the object to be found, obtain a reference picture of the reference object, and obtain a picture of the object to be found based on the indication information; the reference picture is captured from the image collected during the robot's object-finding process. Calculate the similarity between the picture of the object to be found and the reference picture of the reference object; if the similarity is greater than a preset similarity threshold, increase the object confidence of the reference object corresponding to the similarity, and control the object confidence of other reference objects corresponding to the similarity to remain unchanged. If the reference feature information does not include indication information corresponding to the picture of the object to be found, control the object confidence to remain unchanged.

[0165] The object to be found is determined from the reference objects according to the final object confidence.

[0166] The location of the object to be found is determined based on the robot's location information in the preset room map.

[0167] The location of the object to be found is indicated in the preset room map.

[0168] In this way, compared to matching the object to be found only by a single feature, the present application first determines the reference object and the object confidence corresponding to the reference object according to the object category, and then further adjusts the object confidence by the shape of the object to be found, the color of the object to be found, the candidate area where the object to be found is located, and the picture. The object category and multiple features such as shape, color, area, and picture are combined to search for objects, so that the object confidence can comprehensively reflect the degree of match between the reference object and the object to be found from multiple dimensions such as category, shape, color area, and picture, thereby making the accuracy of the object to be found determined from the reference object through the adjusted object confidence higher, thereby improving the accuracy of finding objects.

[0169] Fig. 9 is a block diagram of a device for robot object finding shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown in the figure is specifically configured in the robot 11. The device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0170] like Figure 8 As shown, the exemplary device for robot object finding includes:

[0171] The feature acquisition module 801 is configured to recognize the received object-finding voice and obtain object-finding feature information of the object to be found, wherein the object-finding feature information includes the object category of the object to be found;

[0172] The object-finding module 802 is configured to search for objects according to object categories, obtain reference objects that match the object categories and object confidences corresponding to the reference objects;

[0173] The adjustment module 803 is configured to adjust the object confidence according to the reference feature information if it is detected that the object finding feature information also includes reference feature information other than the object category;

[0174] The determination module 804 is configured to determine the object to be found from the reference objects according to the adjusted object confidence.

[0175] In an exemplary embodiment, the adjustment module 803 includes:

[0176] The first image acquisition submodule is configured to acquire a reference image of the reference object if the reference feature information includes the color of the object to be found, wherein the reference image is captured from an image collected during the robot's object-finding process.

[0177] The color confidence acquisition submodule is configured to determine the color confidence of the reference object according to the reference image and color through a color classification network.

[0178] The first adjustment submodule is configured to adjust the object confidence based on the color confidence.

[0179] In an exemplary embodiment, the adjustment module 803 includes:

[0180] The second image acquisition submodule is configured to acquire a reference image of the reference object if the reference feature information includes the shape of the object to be found, wherein the reference image is captured from an image collected during the robot's object-finding process.

[0181] The shape confidence acquisition submodule is configured to determine the shape confidence of the reference object according to the reference image and the shape through a shape classification network.

[0182] The second adjustment submodule is configured to adjust the object confidence based on the shape confidence.

[0183] In an exemplary embodiment, the adjustment module 803 includes:

[0184] The target reference object determination submodule is configured to determine the target reference object within the candidate area from the reference objects if the reference feature information includes a candidate area of ​​the object to be found.

[0185] The first adding submodule is configured to increase the object confidence corresponding to the target reference object.

[0186] In an exemplary embodiment, the adjustment module 803 includes:

[0187] The third image acquisition submodule is configured to acquire a reference image of the reference object if the reference feature information includes indication information corresponding to the image of the object to be found, and to acquire the image of the object to be found based on the indication information; the reference image is captured from the image collected during the robot's object-finding process.

[0188] The similarity calculation submodule is configured to calculate the similarity between the image of the object to be found and the reference image of the reference object.

[0189] The second adding submodule is configured to increase the object confidence of the reference object corresponding to the similarity if the similarity is greater than a preset similarity threshold.

[0190] In an exemplary embodiment, the feature acquisition module 801 includes:

[0191] The conversion submodule is configured to convert the object-finding voice into the object-finding text.

[0192] The feature information acquisition submodule is configured to input the object-finding text into a preset text analysis model for recognition, and obtain the object-finding feature information of the object to be found.

[0193] In an exemplary embodiment, the object-finding module 802 includes:

[0194] The acquisition submodule is configured to acquire images of the area where the robot is located in real time through a camera device while the robot is searching for objects.

[0195] The object detection submodule is configured to perform object detection on the image, obtain a reference object that matches the object category, and obtain an object confidence corresponding to the reference object.

[0196] In an exemplary embodiment, the exemplary device for robot object finding further includes:

[0197] The position prompt module is configured to determine the position of the object to be found based on the position information of the robot in a preset room map; and to prompt the position of the object to be found in the preset room map.

[0198] It should be noted that the device for robot finding objects provided in the above embodiment and the method for robot finding objects provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the device for robot finding objects provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0199] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device implements the method for robot searching for objects provided in the above-mentioned embodiments.

[0200] Fig. 9 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig. 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0201] like Fig. 9As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 to the random access memory (RAM) 903, such as executing the method described in the above embodiment. In the RAM 903, various programs and data required for system operation are also stored. The CPU 901, ROM 902 and RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0202] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0203] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present application are executed.

[0204] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0205] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0206] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0207] Another aspect of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for robot object finding as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0208] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for robot object finding provided in each of the above embodiments.

[0209] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. A method for robot to find objects, characterized in that: include: Recognize the received object-finding voice to obtain object-finding feature information of the object to be found, wherein the object-finding feature information includes the object category of the object to be found; Searching for objects according to the object category to obtain reference objects matching the object category and object confidences corresponding to the reference objects; If it is detected that the object-finding feature information also includes reference feature information other than the object category, adjusting the object confidence level according to the reference feature information; The object to be found is determined from the reference objects according to the adjusted object confidence.

2. The method according to claim 1, characterized in that The adjusting the object confidence according to the reference feature information includes: If the reference feature information includes the color of the object to be found, a reference image of the reference object is obtained, where the reference image is captured from an image collected during the robot's object-finding process; Determining the color confidence of the reference object according to the reference image and the color through a color classification network; The object confidence is adjusted based on the color confidence.

3. The method according to claim 1, characterized in that The adjusting the object confidence according to the reference feature information includes: If the reference feature information includes the shape of the object to be found, a reference image of the reference object is obtained, where the reference image is captured from an image collected during the robot's object-finding process; Determining a shape confidence of the reference object according to the reference image and the shape through a shape classification network; The object confidence is adjusted based on the shape confidence.

4. The method according to claim 1, characterized in that: The adjusting the object confidence according to the reference feature information includes: If the reference feature information includes a candidate region of the object to be found, determining a target reference object within the candidate region from the reference objects; The object confidence corresponding to the target reference object is increased.

5. The method according to claim 1, characterized in that The adjusting the object confidence according to the reference feature information includes: If the reference feature information includes indication information corresponding to the picture of the object to be found, then obtaining a reference picture of the reference object, the reference picture being captured from an image collected during the robot's object-finding process, and obtaining the picture of the object to be found based on the indication information; Calculating the similarity between the image of the object to be found and the reference image of the reference object; If the similarity is greater than a preset similarity threshold, the object confidence of the reference object corresponding to the similarity is increased.

6. The method according to claim 1, characterized in that The step of recognizing the received object-finding voice to obtain the object-finding feature information of the object to be found includes: Converting the object-finding voice into object-finding text; The object-finding text is input into a preset text analysis model for recognition to obtain object-finding feature information of the object to be found.

7. The method according to claim 1, characterized in that: The robot is equipped with a camera device; searching for objects according to the object category to obtain reference objects matching the object category and object confidences corresponding to the reference objects include: When the robot is searching for objects, the camera device collects images of the area where the robot is located in real time; Performing object detection on the image to obtain a reference object that matches the object category, and obtaining an object confidence level corresponding to the reference object.

8. The method according to any one of claims 1 to 7, characterized in that After determining the object to be found from the reference objects according to the adjusted object confidence, the method further includes: Determine the position of the object to be found according to the position information of the robot in a preset room map; The location of the object to be found is indicated in a preset room map.

9. A device for robot to find objects, characterized in that: include: A feature acquisition module is configured to recognize the received object-finding voice and obtain object-finding feature information of the object to be found, wherein the object-finding feature information includes the object category of the object to be found; An object-finding module configured to search for objects according to the object category, and obtain a reference object matching the object category and an object confidence level corresponding to the reference object; an adjustment module, configured to adjust the object confidence according to the reference feature information if it is detected that the object-finding feature information also includes reference feature information other than the object category; A determination module is configured to determine the object to be found from the reference objects according to the adjusted object confidence.

10. A robot for cleaning, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method for robot object hunting as described in any one of claims 1 to 8.

Images