A scene recognition method for a cleaning robot and a cleaning robot

Abstract

The present invention relates to a scene recognition method for a cleaning robot and the cleaning robot, wherein the scene recognition method is characterized by comprising the following steps: step 1, pre-storing the category names of N types of common target objects in a home scene in the cleaning robot; step 2, the cleaning robot collects scene images, performs target object detection on the scene images, and finally determines the probability Z of the current environment being a specific home scene through a linear combination between the detected target objects and a weight matrix; step 3, introducing a home scene matrix H; step 4, taking i=arg max(Z), where i is the maximum score a in Z i The index value of the home scene matrix H is obtained by taking the L corresponding to the i-th index position. i Compared with the prior art, the advantage of the present invention is that it can accurately and quickly identify different specific home scenes.

Description

Technical Field

[0001] The invention relates to a scene recognition method of a cleaning robot and the cleaning robot. Background Art

[0002] With the development and progress of society, people have higher requirements for the quality of life. This is reflected in the home scene where more and more families are no longer satisfied with the traditional home decoration style. More families choose open scenes such as open kitchens and open bedrooms, reflecting the pursuit of a better life. As for traditional cleaning robots, when cleaning the home environment, different cleaning modes are often selected according to different spaces. For open scenes, traditional cleaning robots often use virtual walls to divide the space, and do not perform targeted space division. In addition, traditional cleaning robot scene recognition algorithms often have high requirements for changes in light and shadow in the external environment, and cannot achieve good recognition effects for various home environments. Summary of the invention

[0003] The first technical problem to be solved by the present invention is to provide a scene recognition method for a cleaning robot that can accurately and quickly recognize different specific home scenes in response to the above-mentioned prior art.

[0004] The second technical problem to be solved by the present invention is to provide a cleaning robot that can accurately and quickly identify different spaces in an open scene environment in response to the above-mentioned prior art.

[0005] The technical solution adopted by the present invention to solve the above-mentioned first technical problem is: a scene recognition method of a cleaning robot, characterized by comprising the following steps:

[0006] Step 1: Pre-store the category names of N common target objects in home scenes in the cleaning robot;

[0007] Step 2: The cleaning robot collects scene images, detects target objects in the scene images, and finally determines the probability Z of the current environment being a specific home scene through the linear combination of the detected target objects and the weight matrix.

[0008] Z=O1×W1+O2×W2+…O k ×W k

[0009] Where k is the number of target objects detected in the scene image;

[0010] O is the confidence of the target object detected in the scene image, which is specifically expressed as:

[0011] O i =[p1, p2, ... pN ]

[0012] p is the confidence that the target object detected in the scene image is predicted to be a certain type of target object, N is the number of target object categories, and p i The confidence level of identifying the target object as the i-th type target object;

[0013] W is a weight matrix, and the specific value of the weight matrix reflects the degree of association between the target object detected by the scene image and the corresponding specific home scene, which is specifically expressed as: In this matrix, q ij The predicted association degree of the i-th target object to the j-th specific home scene is obtained through prior knowledge and pre-stored in the configuration file of the cleaning robot. J represents the number of common specific home scenes in the home scene.

[0014] The result obtained by using the calculation formula of Z is finally expressed as follows:

[0015] Z=[a1,a2…,a J ]

[0016] In the above formula, a1, a2…, a J Indicates the score of each specific home scene. The higher the score, the higher the possibility that the scene image obtained through target object detection corresponds to a specific home scene;

[0017] Step 3: Introduce the home scene matrix H, which is specifically expressed as follows:

[0018] H=[L1,L2…,L J ]

[0019] L1, L2…, L J For a common specific home scene;

[0020] Step 4: Take i=argmax(Z), where i is the maximum score a in Z i The index value of the home scene matrix H is obtained by taking the L corresponding to the i-th index position. i This is the specific home scene that is finally identified.

[0021] Normally, in common home scenes, the above method can be used to obtain the specific scene in the current spatial position. In real life, due to the continuous improvement of living standards, more and more families choose open space methods to enhance communication and exchanges between people, such as common open kitchens, open bedrooms, etc. When using the above method for target detection, for open scenes such as when the kitchen is connected to the living room, the target detected by the cleaning robot will usually change with the cleaning route. For example, when the camera is facing the living room, the detected target will make the robot judge that it is currently in the living room, and then select the cleaning mode of the living room. In fact, the position at this time is still in the kitchen. This scenario will cause certain problems for the cleaning of the intelligent cleaning robot. As an improvement, when the cleaning robot is in an open space and can identify two specific home scenes, space segmentation is performed in the following way:

[0022] Step a: The cleaning robot obtains a coordinate scene vector (X) based on its current position coordinates in space and the specific home scene recognized at the current position. i ,Y i ,Z i ), where (X i ,Y i ) represents the spatial position information, Z i Representing a specific home scene, the spatial scene vector obtained by the cleaning robot during the cleaning process is marked as:

[0023] T={(X1,Y1,Z1),(X2,Y2,Z2),…(X n ,Y n ,Z n )};

[0024] Step b: Initialize the space dividing line equation of the cleaning robot as w(X i ,Y i )+b=0;

[0025] Step c: The cleaning robot defines the geometric interval of the space dividing line with respect to the space scene vector as D i =Z i (w×(X i ,Y i )+b);

[0026] Step d: Use linear optimization method to obtain the minimum geometric interval min (D i ) corresponding to w and b;

[0027] Step e, substitute w and b obtained in step d into the space dividing line formula w(X,Y)+b to obtain the final space dividing line formula w(X,Y)+b; use the position where the result of the space dividing line formula is 0 as the space dividing line of the open space, and use this space dividing line to divide the open space into two different specific home scenes.

[0028] A cleaning time is set in advance. After the preset cleaning time, the spatial scene vector is obtained by continuous accumulation. When the preset cleaning time is over, the final spatial dividing line formula w(X,Y)+b is calculated through steps a to e, so that the spatial dividing line formula can be continuously corrected to obtain the optimal w and b values.

[0029] Preferably, the pre-set cleaning time is 15 to 25 minutes.

[0030] The technical solution adopted by the present invention to solve the above-mentioned second technical problem is: a cleaning robot includes a body, a cleaning component is provided on the body, and a controller for controlling the working state of the cleaning component is provided in the body, characterized in that: the controller adopts the above-mentioned scene recognition method to identify the current working environment.

[0031] The controller is equipped with cleaning strategies corresponding to different types of specific home scenes. Under each cleaning strategy, the controller controls the cleaning component to work under different working conditions. When the controller identifies the specific home scene in which the cleaning robot is currently located, it controls the cleaning component to work under the corresponding working conditions according to the preset corresponding cleaning strategy.

[0032] Compared with the prior art, the advantages of the present invention are: by identifying the target object in the scene image, multiplying the confidence of the target object with the weight matrix, and obtaining the score of each specific home scene. The higher the score, the higher the possibility that the scene image obtained by target object detection corresponds to a specific home scene. Then, by introducing the home scene matrix, different specific home scenes can be accurately and quickly identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of a scene recognition method for a cleaning robot in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present invention is described in detail below with reference to the accompanying drawings.

[0035] like Figure 1 As shown, the present invention provides a scene recognition method for a cleaning robot, which comprises the following steps:

[0036] Step 1: Pre-store the category names of N common target objects in home scenes in the cleaning robot;

[0037] Target objects include range hoods, stoves, sofas, TVs, beds, dining tables and chairs, coffee tables, mirrors, toilets, books, etc.

[0038] Step 2: The cleaning robot collects scene images, detects target objects in the scene images, and finally determines the probability Z that the current environment is a specific home scene through the linear combination between the detected target objects and the weight matrix:

[0039] Z=O1×W1+O2×W2+…O k ×W k

[0040] Where k is the number of target objects detected in the scene image;

[0041] O is the confidence of the target object detected in the scene image, which is specifically expressed as:

[0042] O i =[p1, p2, ... p N ]

[0043] p is the confidence that the target object detected in the scene image is predicted to be a certain type of target object, N is the number of target object categories, and p i The confidence level of identifying the target object as the i-th type target object;

[0044] W is a weight matrix, and the specific value of the weight matrix reflects the degree of association between the target object detected by the scene image and the corresponding specific home scene, which is specifically expressed as: In this matrix, q ij The predicted association degree of the i-th target object to the j-th specific home scene is obtained through prior knowledge and pre-stored in the configuration file of the cleaning robot. J represents the number of common specific home scenes in the home scene.

[0045] For example, in a scene image, three target objects are detected, namely, a coffee table, a TV, and a sofa. Then the product of the confidence of the three target objects and the weight matrix is ​​calculated. When the target object is a coffee table, if p2 represents the confidence of the coffee table, then p2 ≥ 0.95, p1, p3, ... p N All approach 0, at this time O1=[0001, 0.99, … 0.001]; p1+p2+p3+…+p N =1; when the target object is a TV, if p1 represents the confidence of the TV, then p1 = ≥ 0.95, p2, p3, ... p N All approach 0, at this time O2=[0.99, 0.001, …0.001], p1+p2+p3+…+p N=1; when the target object is a sofa, if p3 represents the confidence of the sofa, then p3 = ≥ 0.95, p1, p2, ... p N All approach 0, O3 = [0.001, 0.001, 0.99, ... 0.001]; p1 + p2 + p3 + ... + p N =1, the weight matrices W1, W2, and W3 corresponding to the coffee table, TV, and sofa are also pre-stored in the configuration file of the cleaning robot; the result obtained by using the calculation formula of Z is finally expressed as follows:

[0046] Z=[a1,a2…,a J ]

[0047] In the above formula, a1, a2…, a J Indicates the score of each specific home scene. The higher the score, the higher the possibility that the scene image obtained through target object detection corresponds to a specific home scene;

[0048] Step 3: Introduce the home scene matrix H, which is specifically expressed as follows:

[0049] H=[L1,L2…,L J ]

[0050] L1, L2…, L J For a common specific home scene;

[0051] Step 4: Take i=argmax(Z), where i is the maximum score a in Z i The index value of the home scene matrix H is obtained by taking the L corresponding to the i-th index position. i This is the specific home scene that is finally identified.

[0052] When the cleaning robot is in an open space, as the camera of the cleaning robot changes its position, for example, in an open kitchen, there is no partition between the kitchen and the living room, which are completely open and connected. When the cleaning robot is in the kitchen and the camera faces the range hood, it will recognize the kitchen environment. However, when the camera of the cleaning robot faces the living room, the scene image may show target objects such as sofas, TVs, and coffee tables. At this time, the robot may be considered to be in the living room, but in fact, the cleaning robot is still in the kitchen. In this case, when the cleaning robot can recognize two different specific home scenes in a short period of time, the space is segmented in the following way:

[0053] Step a: Preset the cleaning time. During the preset cleaning time, the cleaning robot continuously accumulates the spatial scene vector (X i ,Y i ,Zi ), where (X i ,Y i ) represents the spatial position information, Z i Representing a specific home scene, the spatial scene vector obtained by the cleaning robot within the preset cleaning time is marked as:

[0054] T={(X1,Y1,Z1),(X2,Y2,Z2),…(X n ,Y n ,Z n )};

[0055] n represents the number of spatial scene vector markers acquired by the cleaning robot within the preset cleaning time;

[0056] Step b: Initialize the space dividing line equation of the cleaning robot as w(X i ,Y i )+b=0;

[0057] Step c: The cleaning robot defines the geometric interval of the space dividing line with respect to the space scene vector as D i =Z i (w×(X i ,Y i )+b);

[0058] Step d: Use linear optimization method to obtain the minimum geometric interval min (D i ) corresponding to w and b;

[0059] Step e, substituting w and b obtained in step d into the space dividing line formula w(X,Y)+b to obtain the final space dividing line formula w(X,Y)+b; using the position where the result of the space dividing line formula is 0 as the space dividing line of the open space, using the space dividing line to divide the open space into two different specific home scenes;

[0060] Substitute the actual coordinates of the cleaning robot into the spatial dividing line formula w(X,Y)+b. If the result is greater than 0, it means that the currently displayed scene is the first specific home scene. If the result is less than 0, it means that the currently displayed scene is the second specific home scene. The first specific home scene and the second specific home scene are divided according to actual conditions.

[0061] The above-mentioned preset cleaning time is 15 to 25 minutes, preferably 20 minutes.

[0062] This embodiment also provides a cleaning robot, including a body, a cleaning component is provided on the body, and a controller for controlling the working state of the cleaning component is provided in the body, and the controller uses the above-mentioned scene recognition method to identify the current working environment. The controller is provided with cleaning strategies corresponding to different types of specific home scenes. Under each cleaning strategy, the controller controls the cleaning component to work under different working conditions. When the controller identifies the specific home scene in which the cleaning robot is currently located, it controls the cleaning component to work under the corresponding working conditions according to the preset corresponding cleaning strategy.

Claims

1. A scene recognition method for a cleaning robot, characterized in that The steps include: Step 1: Pre-store the category names of N common target objects in home scenes in the cleaning robot; Step 2: The cleaning robot collects scene images, detects target objects in the scene images, and finally determines the probability Z of the current environment being a specific home scene through the linear combination of the detected target objects and the weight matrix. Z=O1×W1+O2×W2+…O k ×W k Where k is the number of target objects detected in the scene image; O is the confidence of the target object detected in the scene image, which is specifically expressed as: O i =[p1, p2,…p N ] p is the confidence that the target object detected in the scene image is predicted to be a certain type of target object, N is the number of target object categories, and p i The confidence level of identifying the target object as the i-th type target object; W is a weight matrix, and the specific value of the weight matrix reflects the degree of association between the target object detected by the scene image and the corresponding specific home scene, which is specifically expressed as: In this matrix, q ij The predicted association degree of the i-th target object to the j-th specific home scene is obtained through prior knowledge and pre-stored in the configuration file of the cleaning robot. J represents the number of common specific home scenes in the home scene. The result obtained by using the calculation formula of Z is finally expressed as follows: <h2 style=";text-align:left;direction:ltr">Z=[a1, a2…,a<h2 style=";text-align:left;direction:ltr"> J <h2 style=";text-align:left;direction:ltr"> ] In the above formula, a1, a2…, a J Indicates the score of each specific home scene. The higher the score, the higher the possibility that the scene image obtained through target object detection corresponds to a specific home scene; Step 3: Introduce the home scene matrix H, which is specifically expressed as follows: H=[L1,L2…,L J ] L1, L2…, L J For a common specific home scene; Step 4: Take i=argmax(Z), where i is the maximum score a in Z i The index value of the home scene matrix H is obtained by taking the L corresponding to the i-th index position. i This is the specific home scene that is finally identified.

2. The scene recognition method of the cleaning robot according to claim 1, characterized in that: When the cleaning robot is in an open space and can identify two specific home scenes, the space is segmented in the following way: Step a: The cleaning robot obtains a coordinate scene vector (X) based on its current position coordinates in space and the specific home scene recognized at the current position. i ,Y i ,Z i ), where (X i ,Y i ) represents the spatial position information, Z i Representing a specific home scene, the spatial scene vector obtained by the cleaning robot during the cleaning process is marked as: T={(X1,Y1,Z1),(X2,Y2,Z2),…(X n ,Y n ,Z n )}? Step b: Initialize the space dividing line equation of the cleaning robot as w(X i ,Y i )+b=0; Step c: The cleaning robot defines the geometric interval of the space dividing line with respect to the space scene vector as D i =Z i (w×(X i ,Y i )+b); Step d: Use linear optimization method to obtain the minimum geometric interval min (D i ) corresponding to w and b; Step e, substitute w and b obtained in step d into the space dividing line formula w(X,Y)+b to obtain the final space dividing line formula w(X,Y)+b; use the position where the result of the space dividing line formula is 0 as the space dividing line of the open space, and use this space dividing line to divide the open space into two different specific home scenes.

3. The scene recognition method of the cleaning robot according to claim 2, characterized in that: A cleaning time is pre-set. After the preset cleaning time, the spatial scene vector is obtained by continuous accumulation. When the preset cleaning time is over, the final spatial dividing line formula w(X,Y)+b is calculated through steps a to e.

4. The scene recognition method of the cleaning robot according to claim 3, characterized in that: The pre-set cleaning time is 15 to 25 minutes.

5. A cleaning robot, comprising a body, a cleaning component disposed on the body, and a controller for controlling the working state of the cleaning component disposed in the body, characterized in that: The controller uses the scene recognition method as described in claim 1 or 2 or 3 or 4 to identify the current working environment.

6. The cleaning robot according to claim 5, characterized in that: The controller is equipped with cleaning strategies corresponding to different types of specific home scenes. Under each cleaning strategy, the controller controls the cleaning component to work under different working conditions. When the controller identifies the specific home scene in which the cleaning robot is currently located, it controls the cleaning component to work under the corresponding working conditions according to the preset corresponding cleaning strategy.

Images