Image sample construction method and apparatus

Abstract

Embodiments of the present specification provide an image sample construction method and device, wherein the image sample construction method is applied to a robot, and includes receiving a target object adding instruction sent by a user for an initial cleaning map; in response to the target object adding instruction, showing the user the initial cleaning map; receiving an object position of the target object determined by the user on the initial cleaning map and attribute information of the target object; performing image acquisition on the target object based on the object position of the target object, and constructing an image sample based on the image of the target object and the attribute information of the target object.

Description

Technical Field

[0001] This specification relates to the field of robotics, and in particular to a method for constructing image samples. One or more embodiments of this specification also relate to an image sample construction apparatus, a robot, and a computer-readable storage medium. Background Technology

[0002] Currently, deep learning models and other image recognition algorithms are widely used in the field of image recognition. Robotic vacuum cleaners, by acquiring images with cameras and then using algorithms deployed within the robot to identify target objects in the images, can perceive the complete surrounding environment, greatly improving the intelligence of the robotic vacuum cleaner.

[0003] However, training deep learning models requires collecting a large number of samples (such as image data) as a dataset. When the dataset lacks samples of specific targets, the model's recognition rate of specific targets will drop significantly, which will affect the robot's perception of the surrounding environment when performing cleaning work. Therefore, how to efficiently collect images of target objects in a targeted manner and build image samples of the model from the collected images of target objects is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a method for constructing image samples. One or more embodiments of this specification also relate to an image sample construction apparatus, a robot, and a computer-readable storage medium to address technical deficiencies in the prior art.

[0005] According to a first aspect of the embodiments of this specification, an image sample construction method is provided, applied to a robot, comprising:

[0006] Receive user instructions to add target objects to the initial cleaning map;

[0007] In response to the instruction to add the target object, the initial cleaning map is displayed to the user;

[0008] Receive the object location and attribute information of the target object as determined by the user in the initial cleaning map;

[0009] The target object is image acquired based on its location, and an image sample is constructed based on the image and attribute information of the target object.

[0010] According to a second aspect of the embodiments of this specification, an image sample construction method is provided, comprising:

[0011] The instruction receiving module is configured to receive instructions from the user to add targets to the initial cleaning map;

[0012] The instruction response module is configured to display the initial cleaning map to the user in response to the instruction to add the target object.

[0013] The location determination module is configured to receive the object location and attribute information of the target object as determined by the user on the initial cleaning map;

[0014] The image sample construction module is configured to acquire images of the target object based on the object location of the target object, and construct image samples based on the image of the target object and the attribute information of the target object.

[0015] According to a third aspect of the embodiments of this specification, a robot is provided, comprising:

[0016] A mechanical body, wherein a memory and a processor are provided on the mechanical body;

[0017] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the image sample construction method.

[0018] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the image sample construction method.

[0019] This specification provides an embodiment of an image sample construction method and apparatus. The image sample construction method is applied to a robot and includes receiving a target object addition instruction sent by a user for an initial cleaning map; responding to the target object addition instruction, displaying the initial cleaning map to the user; receiving the object location and attribute information of the target object determined by the user on the initial cleaning map; acquiring an image of the target object based on the object location; and constructing an image sample based on the image of the target object and the attribute information of the target object.

[0020] Specifically, the image sample construction method can selectively collect images of target objects based on the locations of target objects added by the user in the initial cleaning map, and construct image samples based on the target object's image and corresponding attribute information added by the user in the initial cleaning map. This method is highly targeted and has a high efficiency in constructing image samples. Attached Figure Description

[0021] Figure 1This is a flowchart illustrating an image sample construction method provided in one embodiment of this specification;

[0022] Figure 2 This is a schematic diagram illustrating the identification of avoidance objects in an image sample construction method provided in one embodiment of this specification;

[0023] Figure 3 This is a schematic diagram of the structure of the initial clean map in an image sample construction method provided in one embodiment of this specification;

[0024] Figure 4 This is a schematic diagram illustrating the determination of the object location in an initial clean map within an image sample construction method provided in one embodiment of this specification;

[0025] Figure 5 This is a schematic diagram illustrating the determination of target object attribute information in an initial clean map in an image sample construction method provided by an embodiment of this specification;

[0026] Figure 6 This is a schematic diagram illustrating the image acquisition of a target object in an image sample construction method provided in one embodiment of this specification;

[0027] Figure 7 This is a schematic diagram of the structure of an image sample construction device provided in one embodiment of this specification;

[0028] Figure 8 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0029] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0030] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0031] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0032] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0033] 2D: Also known as planar graphics, 2D graphics only contain the horizontal X-axis and the vertical Y-axis.

[0034] 3D: also called three-dimensional graphics, three-dimensional refers to the three dimensions in a Cartesian coordinate system: X (horizontal), Y (vertical), and Z (vertical).

[0035] In existing technologies, because robotic vacuum cleaners operate on the ground with a very unique perspective, the image data in datasets collected from commonly used normal perspectives is not very complete. Therefore, when a deep learning model trained on this incomplete image data is applied to a robotic vacuum cleaner, the robot's perception of the surrounding environment and objects will be very poor. As a result, the constructed cleaning map may be inaccurate and incomplete, leading to low cleaning efficiency when the robotic vacuum cleaner performs cleaning tasks based on this cleaning map.

[0036] To improve the generalization performance of deep learning models, devices with a specific robot vacuum cleaner's height and field of view can be used to collect images in users' real homes. However, this requires users to use additional devices with the same robot vacuum cleaner's height and field of view, significantly increasing the collection cost. Furthermore, the complex placement of items (such as furniture, appliances, and clothing) in real homes makes collecting images of specific targets from a variety of items time-consuming and inefficient. Adding category labels to the target images after collection also presents significant challenges to acquiring image sample datasets.

[0037] This specification provides a method for constructing image samples. One or more embodiments of this specification also relate to an image sample construction apparatus, a robot, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0038] See Figure 1 , Figure 1 A flowchart of an image sample construction method according to an embodiment of this specification is shown, wherein the method is applied to a robot and specifically includes the following steps.

[0039] Step 102: Receive the user's instruction to add target objects to the initial cleaning map.

[0040] The image sample construction method provided in the embodiments of this specification can be applied to any scenario in which a robot cleans a work area, such as a scenario in which a robot sweeps a work area, a scenario in which a robot disinfects a work area, or a scenario in which a robot vacuums a work area. This specification does not limit this in any way. Therefore, the type of robot is also different depending on the application scenario. For example, in a sweeping scenario, the robot can be a sweeping robot; in a disinfection scenario, the robot can be a disinfection robot; and in a vacuuming scenario, the robot can be a vacuuming robot.

[0041] For ease of understanding, the embodiments in this specification all use the application of the image sample construction method in a scenario where a robot is used to clean a work area, with a robotic vacuum cleaner as an example for detailed explanation.

[0042] In practical applications, receiving a target object addition instruction sent by the user for the initial cleaning map can be understood as receiving a target object addition instruction generated and sent by the user through operation of the robot's display screen panel; it can also be understood as receiving a target object addition instruction generated and sent by the user through operation of a user terminal (such as a smartphone) that has a communication connection with the robot.

[0043] For example, in one scenario, the robot has a display screen panel, and the user can generate a target object addition instruction by clicking or touching the initial cleaning map on the robot's display screen panel; in another scenario, the robot establishes a communication connection with a user terminal (such as a smartphone) and sends the initial cleaning map to the user terminal's application, and the user can generate a target object addition instruction by clicking or touching the initial cleaning map in the user terminal's application.

[0044] Specifically, the initial cleaning map can be understood as a cleaning map created by the robot for a specific work area to be cleaned. The specific creation method is as follows:

[0045] Before receiving the user's instruction to add target objects to the initial cleaning map, the process also includes:

[0046] During the cleaning process of the work area, an initial cleaning map is created for the work area based on an image recognition model and a distance detection device.

[0047] The work area can be any area to be cleaned, such as a residential room, a factory site, etc.; the image recognition model can be understood as a pre-trained deep learning model; and the distance detection device can be understood as the robot's camera calibration system or other distance sensors, including but not limited to radar, infrared, collision plates, etc.

[0048] Taking a room as an example, during the process of the robot performing a cleaning task on the room, an initial cleaning map is created for the room based on an image recognition model and a distance detection device. The robot can then perform a cleaning task on the room again based on this initial cleaning map.

[0049] In the embodiments described in this specification, the deep learning-based image recognition model and distance detection device can quickly and accurately create an initial cleaning map for the work area, enabling the robot to clean the work area again based on the initial cleaning map, thereby improving cleaning efficiency.

[0050] In specific implementation, during the cleaning task of the work area, creating an initial cleaning map for the work area based on an image recognition model and a distance detection device includes:

[0051] During the cleaning process of the work area, at least one image of the work area is captured.

[0052] Each image is sequentially input into the image recognition model to obtain the object to be avoided in each image and the attribute information of the object to be avoided;

[0053] The distance between the robot and the object to be avoided is determined by a distance detection device;

[0054] An initial cleaning map is created for the work area based on the object to be avoided, the attribute information of the object to be avoided, and the distance between the robot and the object to be avoided.

[0055] The objects to be avoided can be understood as items in the work area. For example, if the work area is a room, then the objects to be avoided can be the walls, furniture, appliances, and household items in the room. When cleaning the work area, the robot needs to avoid these items to prevent damage.

[0056] Specifically, during the cleaning process, the robot captures multiple images of the work area using its installed cameras. Each image is then input into a pre-trained image recognition model to obtain the objects to be avoided and their attribute information. This attribute information includes, but is not limited to, the object's category, name, size, and height. Simultaneously, the robot uses a distance detection device to determine the distance between itself and the objects to be avoided. Based on the objects, their attribute information, and the distance between the robot and the objects, an initial cleaning map is created for the work area.

[0057] For example, if the robot is a robotic vacuum cleaner and its working area is a user's home environment, then during the cleaning process, the robotic vacuum cleaner will use its sensors (such as radar, infrared, bumpers, cameras, etc.) to detect data about the home environment and create an initial cleaning map. Specifically, during the cleaning process, the robotic vacuum cleaner will use a camera mounted on it and calibrate the camera, and incorporate image recognition algorithms such as deep learning models (i.e., image recognition models) to collect images of the user's home environment. It will then use these image recognition algorithms to identify objects in the collected images. Simultaneously, the robotic vacuum cleaner will use its camera calibration system or other sensors to measure the distance from objects to the robot. This process creates a semantic map of the user's home environment (i.e., the initial cleaning map). This map not only includes the external dimensions of the user's home environment but also the results of the robot's image recognition of objects within the environment (i.e., semantic information).

[0058] See Figure 2 , Figure 2 A schematic diagram illustrating the identification of avoidance objects in an image sample construction method according to an embodiment of this specification is shown.

[0059] Taking slippers as an example, when a robot vacuum cleaner is performing a cleaning task in a user's home environment, it captures an image containing slippers. This image is then input into an image recognition model, which outputs "slippers." Simultaneously, it uses a camera calibration system or other sensors to measure the distance between the slippers and the robot vacuum cleaner. For instance, through recognition and ranging algorithms, the robot vacuum cleaner can identify that the captured image contains slippers and measure the distance between the slippers and the robot vacuum cleaner as 40 centimeters.

[0060] See Figure 3 , Figure 3A schematic diagram of the structure of an initial clean map in an image sample construction method according to an embodiment of this specification is shown.

[0061] Combination Figure 2 When the robot vacuum cleaner completes the cleaning task of the user's home environment, it will create a complete home environment map and generate a detailed semantic map. The user can then view the semantic map through the application on the user terminal that is connected to the robot vacuum cleaner, or through the display screen panel on the robot vacuum cleaner.

[0062] like Figure 3 In the semantic map, when the robot vacuum cleaner identifies a slipper as an object to avoid, it places a shoe icon at the corresponding location; when it identifies a sofa, it places a sofa icon; when it identifies a trash can, it places a trash can icon; when it identifies an electrical wire, it places an electrical wire icon, and so on. Furthermore, the voice map can also use images of actual objects to avoid instead of the corresponding icons; the specific settings depend on the application, and this application does not impose any limitations on this.

[0063] In practical applications, there are no limitations on the types of objects to be identified and avoided; they can be diverse, as long as they are collected and labeled in the dataset used by the image recognition model during pre-training. For example, they can be everyday items such as slippers and towels, furniture and appliances such as sofas, coffee tables, and televisions, or even living beings such as people and pets.

[0064] Step 104: In response to the target object addition instruction, display the initial cleaning map to the user.

[0065] Specifically, when the robot receives a target addition instruction sent by the user for the initial cleaning map, it responds to the instruction and displays the constructed initial cleaning map to the user. At this time, the initial cleaning map is in an editable state, and the user adds target objects to the initial cleaning map based on it.

[0066] In practical applications, the target addition command sent by the user to the initial cleaning map received by the robot can be understood as receiving a target addition command generated by the user clicking the target addition icon on the initial cleaning map on the robot's display screen panel. After receiving the target addition command, the robot displays the pre-built initial cleaning map to the user on the display screen panel. Alternatively, it can be understood as receiving a target addition command generated by the user clicking the target addition icon on the initial cleaning map on a terminal application that has a communication connection with the robot. After receiving the target addition command, the robot displays the pre-built initial cleaning map to the user on the display screen panel.

[0067] Step 106: Receive the object location and attribute information of the target object as determined by the user on the initial cleaning map.

[0068] In practical applications, after the initial cleaning map is displayed to the user, if the user feels that the robot vacuum has missed recognizing a certain object to avoid (also known as a target object), or wants the robot vacuum to actively recognize a certain target object (also known as a target object), the target object can be added to the initial cleaning map and a category label can be added to the target object.

[0069] The attribute information of the target object includes, but is not limited to, the target object's name, category, volume, height, etc. In this embodiment of the specification, taking the category as an example, receiving the attribute information of the target object determined by the user in the initial cleaning map can be understood as receiving the category information of the target object added by the user at the target object's location on the initial cleaning map.

[0070] Specifically, receiving the object location of the target object determined by the user in the initial cleaning map can be understood as receiving the object location of the target object specified by the user on the initial cleaning map displayed on the robot's display screen panel, or receiving the object location of the target object specified by the user on the initial cleaning map displayed on the terminal application that establishes a communication connection with the robot.

[0071] In specific implementation, receiving the object location and attribute information of the target object determined by the user in the initial cleaning map includes:

[0072] Receive the user's calibration operation on the initial cleaning map, and determine the object location of the target object in the initial cleaning map; and

[0073] Receive the attribute information set by the user for the target object at the object location through input or selection.

[0074] The calibration operation can be understood as the user drawing a frame (such as drawing a circle, an outline, or a square) on the initial cleaning map; the input or selection operation can be understood as the user inputting text on the initial cleaning map, or the user performing an operation on the initial cleaning map through a category list or category icon.

[0075] In practical applications, receiving the user's calibration operation on the initial cleaning map and determining the object's location on the initial cleaning map can be understood as receiving the user's drawing of a frame at a specific location on the initial cleaning map, and using the framed location as the object's location on the initial cleaning map; receiving the attribute information set by the user for the target object at the object location through input or selection can be understood as receiving the category of the target object entered by the user in text form at the target object's location on the initial cleaning map, or receiving the category of the target object selected by the user at the target object's location on the initial cleaning map by selecting from a category list or a pop-up category chart.

[0076] Specifically, if we want to enrich the dataset of the image recognition model based on the collected images of the target object, so as to retrain the image recognition model and improve its accuracy, we not only need to collect images of the target object, but also add category labels to the images of the target object, so as to construct image training samples for the image recognition model based on the images of the target object and the category of the target object.

[0077] In the embodiments of this specification, the user can accurately determine the location of the target object to be collected in the initial cleaning map by drawing a frame. Subsequently, the robot can be accurately moved to the target object based on the object's location to achieve accurate image collection of the target object. Furthermore, the user can perform category labeling on the target object based on the user's input and selection operations, which reduces the difficulty of subsequent data labeling. Subsequently, image samples of the model dataset can be directly constructed based on the image of the target object and its corresponding category.

[0078] See Figure 4 , Figure 4 This diagram illustrates the determination of the object location of a target object in an initial clean map in an image sample construction method according to an embodiment of this specification.

[0079] Combination Figure 3 , Figure 3 For the initial cleaning map of the work area, in Figure 3Based on the initial cleaning map, if a user wants to add a target object, such as a dining table, at a specific location on the initial cleaning map, the user can first draw a box at that specific location on the initial cleaning map to determine the position of the dining table in the initial cleaning map, so as to add the target object in the initial cleaning map. Here, the specific location can be understood as the map position of the dining table determined by the user in the initial cleaning map.

[0080] In practical applications, the initial cleaning map can be in 2D or 3D format, depending on the actual needs. This manual does not impose any restrictions on this.

[0081] See Figure 5 , Figure 5 This diagram illustrates the determination of attribute information of a target object in an initial clean map in an image sample construction method according to an embodiment of this specification.

[0082] Combination Figure 4 For example, if a user wants to add a target object, such as a dining table, to a specific location on the initial cleaning map, the user can first draw a box at that location to define the table's position. Then, based on that position, the user can determine the table's category. Specifically, the user can input the category via text, or the initial cleaning map interface can display a list or chart of categories for the user to select. Alternatively, the user can first define the table's category, for example, by inputting the category via text, or by displaying a list or chart of categories for the user to select, and then draw a box at that specific location on the initial cleaning map to define the table's position. This allows the user to add target objects to the initial cleaning map.

[0083] In practical applications, the initial cleaning map can be in 2D or 3D format, depending on the actual needs. This manual does not impose any restrictions on this.

[0084] In the embodiments described in this specification, the user can accurately determine the location of the target object to be collected in the initial cleaning map by drawing a frame. Subsequently, the robot can be accurately moved to the target object based on the object's location to achieve accurate image collection of the target object. Furthermore, the target object's image can be labeled with corresponding category information, so that the target object's image and the corresponding category label can be used to construct training samples for the image recognition model. This enriches the image recognition model's dataset, enhances the recognition accuracy of the image recognition model, and ultimately improves the working efficiency of the robot using the image recognition model.

[0085] Specifically, after constructing image samples of the target object based on the object location of the target object, the method further includes:

[0086] The initial cleaning map is updated based on the image of the target object and the attribute information of the target object.

[0087] In the embodiments of this specification, after adding target objects to the initial cleaning map, the initial cleaning map can be updated based on the image and attribute information of the target objects, so that the initial cleaning map contains more item information of the work area. Subsequently, when performing cleaning tasks on the work area based on the initial cleaning map, the robot can more accurately avoid items, improve work efficiency and enhance user experience.

[0088] Step 108: Acquire an image of the target object based on its location, and construct an image sample based on the image and attribute information of the target object.

[0089] Specifically, after determining the location of the target object, the robot will move to the vicinity of the target object location to acquire an image of the target object.

[0090] In practical applications, if a user adds a new target object to the initial cleaning map through the terminal application, the terminal application will synchronize the relevant information of the target object (i.e., the object's location and attribute information) to the robot. In this way, the robot can locate and move to the vicinity of the target object based on the initial cleaning map to collect images (i.e., take pictures of the target object).

[0091] Specifically, the step of acquiring an image of the target object based on its location includes:

[0092] The movement is based on the object position of the target object, and an image of the target object is acquired upon reaching that position; or

[0093] During the cleaning task based on the initial map, if the location of the target object is identified, an image of the target object is acquired.

[0094] In practical applications, after receiving the location of a target object, the robot can directly move to that location to capture an image of the target object. Alternatively, it can capture an image of the target object during the next cleaning task on the work area, when cleaning near the target object's location, thus saving on image acquisition steps and improving work efficiency. Furthermore, the robot can capture images of the target object from different distances and angles, as long as its camera view covers all or part of the target object.

[0095] See Figure 6 , Figure 6 A schematic diagram of image acquisition of a target object is shown in an image sample construction method according to an embodiment of this specification.

[0096] Combination Figure 5 After determining the location of the target object, the robot moves to its vicinity and captures images of the target object from different distances and angles. The robot's camera can capture all or part of the target object's image. The robot then wirelessly transmits the images of the target object captured near its location to an image recognition model. This allows the image recognition model to be retrained based on the target object's image and attribute information, thereby increasing the model's accuracy.

[0097] In the embodiments of this specification, the image sample construction method can add the object location of the target object based on the initial clean map, and then directly realize the image acquisition of the target object based on the object location of the target object. It is highly targeted and the image acquisition efficiency of the target object is high.

[0098] In another embodiment of this specification, after constructing the image sample based on the image of the target object and the attribute information of the target object, the method further includes:

[0099] The image recognition model is trained based on the image samples.

[0100] In practical applications, image recognition models are used in robots. When a robot performs cleaning tasks in a work area, the image recognition model will identify and avoid the items in the work area in real time to prevent the robot from colliding with the items. Furthermore, the identification of items in the work area can also improve the robot's work efficiency. Therefore, the accuracy of the image recognition model is very important for the robot. The accuracy of the image recognition model requires a large and rich dataset for training. The dataset includes multiple item images and the item category label corresponding to each item image.

[0101] Therefore, during the process of a robot performing a cleaning task in a work area, an initial cleaning map can be built first based on an initial image recognition model. Then, the robot can be guided to recognize images of more diverse items using the initial cleaning map. Finally, more item images and image categories can be added to the dataset used to train the image recognition model, allowing for more accurate training. When the robot performs a cleaning task in another unfamiliar work area based on the image recognition model, it can create a more accurate initial cleaning map for that unfamiliar work area and perform the cleaning task in that unfamiliar work area more quickly.

[0102] In the embodiments of this specification, when the image sample construction method collects target objects through the initial cleaning map, the collected target objects are all those that the user determines need to identify, which is highly targeted and the collection efficiency of target objects is very high. Moreover, the users all have collection devices (i.e., robots) at home, so there is no need for people to carry collection devices to their homes, saving manpower and resources. At the same time, the user performs category labeling on the collected target objects during the collection process, which reduces the difficulty of subsequent data sample labeling and improves the user experience.

[0103] Corresponding to the above method embodiments, this specification also provides embodiments of an image sample construction apparatus. Figure 7 A schematic diagram of an image sample construction apparatus according to one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0104] The instruction receiving module 702 is configured to receive instructions from the user to add target objects for the initial cleaning map;

[0105] The instruction response module 704 is configured to display the initial cleaning map to the user in response to the instruction to add the target object.

[0106] The location determination module 706 is configured to receive the object location and attribute information of the target object determined by the user on the initial cleaning map;

[0107] The image sample construction module 708 is configured to acquire images of the target object based on the object location of the target object, and construct image samples based on the image of the target object and the attribute information of the target object.

[0108] Optionally, the device further includes:

[0109] The map creation module is configured to create an initial cleaning map for the work area based on an image recognition model and a distance detection device during the cleaning process.

[0110] Optionally, the map creation module is further configured to:

[0111] During the cleaning process of the work area, at least one image of the work area is captured.

[0112] Each image is sequentially input into the image recognition model to obtain the object to be avoided in each image and the attribute information of the object to be avoided;

[0113] The distance between the robot and the object to be avoided is determined by a distance detection device;

[0114] An initial cleaning map is created for the work area based on the object to be avoided, the attribute information of the object to be avoided, and the distance between the robot and the object to be avoided.

[0115] Optionally, the position determination module 706 is further configured to:

[0116] Receive the user's calibration operation on the initial cleaning map, and determine the object location of the target object in the initial cleaning map; and

[0117] Receive the attribute information set by the user for the target object at the object location through input or selection.

[0118] Optionally, the instruction receiving module 702 is further configured to:

[0119] Receive instructions from users via a terminal application to add target objects to the initial cleaning map.

[0120] Optionally, the instruction response module 704 is further configured to:

[0121] In response to the instruction to add the target object, the initial cleaning map is displayed to the user through the terminal application.

[0122] Optionally, the position determination module 706 is further configured to:

[0123] Receive the user's calibration operation on the initial cleaning map via the terminal application, determine and send the object location of the target object in the initial cleaning map; and

[0124] The system receives attribute information of the target object from the user, who determines or selects the target object at the object location via the terminal application.

[0125] Optionally, the image sample construction module 708 is further configured to:

[0126] The movement is based on the object position of the target object, and an image of the target object is acquired upon reaching that position; or

[0127] During the cleaning task based on the initial map, if the location of the target object is identified, an image of the target object is acquired.

[0128] Optionally, the device further includes:

[0129] The model training module is configured to train the image recognition model based on the image samples.

[0130] In the embodiments of this specification, the image sample construction device can selectively acquire images of target objects based on the location of the target objects added by the user on the initial cleaning map, and construct image samples based on the target object's image and corresponding attribute information added by the user on the initial cleaning map. This method is highly targeted and has a high efficiency in constructing image samples.

[0131] The above is an illustrative scheme of an image sample construction apparatus according to this embodiment. It should be noted that the technical solution of this image sample construction apparatus and the technical solution of the image sample construction method described above belong to the same concept. For details not described in detail in the technical solution of the image sample construction apparatus, please refer to the description of the technical solution of the image sample construction method described above.

[0132] Figure 8 A structural block diagram of a computing device 800 according to one embodiment of this specification is shown. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0133] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0134] In one embodiment of this specification, the above-described components of the computing device 800 and Figure 8 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 8 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0135] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 800 can also be a mobile or stationary server.

[0136] The processor 820 is configured to execute the following computer-executable instructions, wherein the processor executes the computer-executable instructions to implement the steps of the image sample construction method.

[0137] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the image sample construction method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the image sample construction method described above.

[0138] An embodiment of this specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the image sample construction method.

[0139] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the image sample construction method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the image sample construction method described above.

[0140] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0141] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0142] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0143] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0144] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for constructing image samples, characterized in that, Applications in robots, including: The robot receives a target object addition instruction sent by the user for the initial cleaning map. The target object is an object that the robot avoids during the cleaning process of the work area. The target object addition instruction is used to acquire images of the target object. In response to the instruction to add the target object, the initial cleaning map is displayed to the user; Receive the object location and attribute information of the target object as determined by the user in the initial cleaning map; The movement is based on the object location of the target object. When the movement reaches the object location of the target object, an image of the target object is captured. Alternatively, when the object location of the target object is identified during the cleaning task based on the initial cleaning map, an image of the target object is captured, and an image sample is constructed based on the image of the target object and the attribute information of the target object.

2. The image sample construction method according to claim 1, characterized in that, Before receiving the user's instruction to add target objects to the initial cleaning map, the process also includes: During the cleaning process of the work area, an initial cleaning map is created for the work area based on an image recognition model and a distance detection device.

3. The image sample construction method according to claim 2, characterized in that, During the cleaning task of the work area, an initial cleaning map is created for the work area based on an image recognition model and a distance detection device, including: During the cleaning process of the work area, at least one image of the work area is captured. Each image is sequentially input into the image recognition model to obtain the object to be avoided in each image and the attribute information of the object to be avoided; The distance between the robot and the object to be avoided is determined by a distance detection device; An initial cleaning map is created for the work area based on the object to be avoided, the attribute information of the object to be avoided, and the distance between the robot and the object to be avoided.

4. The image sample construction method according to claim 1, characterized in that, Receiving the object location and attribute information of the target object determined by the user in the initial cleaning map includes: Receive the user's calibration operation on the initial cleaning map, and determine the object location of the target object in the initial cleaning map; and Receive the attribute information set by the user for the target object at the object location through input or selection.

5. The image sample construction method according to claim 1, characterized in that, The process of receiving user instructions to add target objects to the initial cleaning map includes: Receive instructions from users via a terminal application to add target objects to the initial cleaning map.

6. The image sample construction method according to claim 5, characterized in that, The step of displaying the initial cleaning map to the user in response to the target object addition instruction includes: In response to the instruction to add the target object, the initial cleaning map is displayed to the user through the terminal application.

7. The image sample construction method according to claim 6, characterized in that, Receiving the object location and attribute information of the target object determined by the user in the initial cleaning map includes: Receive the user's calibration operation on the initial cleaning map via the terminal application, determine and send the object location of the target object in the initial cleaning map; and The system receives attribute information of the target object from the user, who determines or selects the target object at the object location via the terminal application.

8. The image sample construction method according to claim 6, characterized in that, After constructing the image sample based on the image of the target object and the attribute information of the target object, the method further includes: An image recognition model is trained based on the image samples.

9. An image sample construction apparatus, characterized in that, include: The instruction receiving module is configured to receive a target object addition instruction sent by the user for the initial cleaning map, wherein the target object is an object that the robot avoids during the cleaning task of the work area, and the target object addition instruction is to acquire an image of the target object. The instruction response module is configured to display the initial cleaning map to the user in response to the instruction to add the target object. The location determination module is configured to receive the object location and attribute information of the target object as determined by the user on the initial cleaning map; The image sample construction module is configured to move based on the object location of the target object, and to acquire an image of the target object when it moves to the object location of the target object, or to acquire an image of the target object when the object location of the target object is identified during the cleaning task based on the initial cleaning map, and to construct an image sample based on the image of the target object and the attribute information of the target object.

10. A robot, characterized in that, include: A mechanical body, wherein a memory and a processor are provided on the mechanical body; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-8.

Images