Self - moving device camera occlusion processing method, self - moving device and storage medium

Abstract

The present invention discloses a method for processing camera occlusion of a self-mobile device, a self-mobile device, and a storage medium. The method includes: obtaining a first picture captured by a camera of the self-mobile device; determining whether there is an occlusion phenomenon in the corresponding camera according to the first picture; and performing an occlusion processing operation after determining that there is an occlusion phenomenon in the camera. The present invention improves the detection accuracy of the occlusion phenomenon and shortens the detection time of the occlusion phenomenon, thereby improving the processing efficiency of the occluder.

Description

Technical Field

[0001] The present invention relates to the technical field of self - moving devices, and particularly to a method for processing camera occlusion of a self - moving device, a self - moving device, and a storage medium. Background Art

[0002] Currently, self - moving devices (such as yard robots) are widely used in life scenarios to perform various functions in an automated manner, including snow removal, lawn mowing, and leaf blowing. In the above - mentioned self - moving devices, cameras are installed on their bodies. However, in complex processing environments, the cameras of self - moving devices may be occluded, for example, by snow, ice, water accumulation, water vapor, or foreign objects (such as mud). At this time, if the above - mentioned occlusion situation cannot be detected in time, the detection function of the camera may be affected, and problems such as collisions or navigation and positioning deviations may occur in the self - moving device. In the prior art, self - moving devices generally determine the occlusion situation through fusion positioning or based on an artificial intelligence model, and both of the above methods have the problem of low determination efficiency. Summary of the Invention

[0003] Based on this, it is necessary to provide a method for processing camera occlusion of a self - moving device, a self - moving device, and a storage medium for the above - mentioned technical problems, so as to solve problems such as low efficiency in determining the occlusion situation in the prior art.

[0004] A method for processing camera occlusion of a self - moving device, the method includes:

[0005] Obtain a first picture taken by a camera of the self - moving device;

[0006] Determine whether there is an occlusion phenomenon in the camera corresponding to the first picture;

[0007] After determining that the camera has an occlusion phenomenon, perform an occlusion processing operation.

[0008] A self - moving device includes a camera and a control module communicatively connected to the camera, and the control module is used to execute the steps of the above - mentioned method for processing camera occlusion of a self - moving device.

[0009] A computer - readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of the above - mentioned method for processing camera occlusion of a self - moving device.

[0010] The method for processing the occlusion of the camera of a self - moving device provided by the present invention includes: obtaining a first picture captured by the camera of the self - moving device; determining whether there is an occlusion phenomenon in the corresponding camera according to the first picture; and performing an occlusion processing operation after determining that there is an occlusion phenomenon in the camera. In the present invention, after obtaining the first picture captured by the camera of the self - moving device, it is determined whether there is an occlusion phenomenon in the corresponding camera through the first picture (such as the first information entropy value or the target edge value of the first picture, etc.), thereby improving the detection accuracy of the occlusion phenomenon and shortening the detection time of the occlusion phenomenon, and thus improving the processing efficiency of the occluding object. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0012] Figure 1 It is a schematic flowchart of the method for processing the occlusion of the camera of a self - moving device in an embodiment of the present invention.

[0013] Figure 2 It is a schematic flowchart of step S200 of the method for processing the occlusion of the camera of a self - moving device in an embodiment of the present invention.

[0014] Figure 3 It is a schematic flowchart of step S200 of the method for processing the occlusion of the camera of a self - moving device in another embodiment of the present invention.

[0015] Figure 4 It is a schematic flowchart of step S400 of the method for processing the occlusion of the camera of a self - moving device in an embodiment of the present invention.

[0016] Figure 5 It is a schematic flowchart of step S500 of the method for processing the occlusion of the camera of a self - moving device in an embodiment of the present invention.

[0017] Figure 6 It is a schematic flowchart of step S500 of the method for processing the occlusion of the camera of a self - moving device in another embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0019] As Figure 1 shown, a method for processing occlusion of a camera of a self - moving device provided by an embodiment of the present invention includes steps S100 to S300:

[0020] S100, obtain a first picture captured by the camera of the self - moving device. Understandably, the self - moving device may be a courtyard robot such as a snow - sweeping robot or a weeding robot. In one embodiment, the self - moving device is a snow - sweeping robot, and cameras are respectively arranged at different positions of the self - moving device. In this embodiment, the first pictures captured by each camera can be obtained respectively, so as to detect each camera respectively. The user can set the conditions for obtaining the first picture captured by the camera of the self - moving device according to the actual situation. For example: the first picture can be obtained in real time, can be obtained at intervals of a preset acquisition duration, can also be obtained after being triggered by some preset conditions (such as the time points of starting pedestrian detection or abnormal navigation positioning or starting obstacle detection or collision, etc.), and can also be obtained by the user issuing a preset acquisition instruction. The first picture can be a picture captured by the camera at the current moment, and the first picture can also be a picture captured and stored by the camera at a certain moment before the current moment.

[0021] S200, determine whether there is an occlusion phenomenon for the camera corresponding to the first picture. Understandably, the self - moving device may include one or more cameras. In this embodiment, according to the first picture, it can be determined whether there is an occlusion phenomenon for the camera that captured the first picture. That is, the first pictures corresponding to each camera can be obtained respectively, so as to determine whether there is an occlusion phenomenon for the camera that captured the first picture according to the obtained first picture.

[0022] As Figure 2 shown, in one embodiment, in step S200, the determining whether there is an occlusion phenomenon for the camera corresponding to the first picture includes steps S210 to S240:

[0023] S210, determine a first information entropy value corresponding to the first picture. Understandably, information entropy can reflect the information complexity of a picture. For a picture captured when there is an occlusion phenomenon for the camera, the amount of information it contains is less, and the corresponding information entropy is smaller. The first information entropy value can be obtained in the following way: First, count the number of occurrences of each pixel value in the first picture, and calculate the occurrence probability of the pixel points corresponding to this pixel value according to the above - mentioned number, and then determine the first information entropy value according to the above - mentioned occurrence probability.

[0024] In one embodiment, the calculation formula of the first information entropy value is as follows:

[0025]

[0026] Among them, H is the first information entropy value corresponding to the first picture; k is the pixel value; p k is the occurrence probability of the pixel points with the pixel value of K in the first picture.

[0027] In one embodiment, in step S110, the determining the first information entropy value corresponding to the first picture includes:

[0028] S211, dividing the first picture into a preset number of image blocks, and determining the second information entropy value corresponding to each of the image blocks. It can be understood that the preset number can be set according to the actual situation (such as the size of the first picture). In one embodiment, the first picture can be evenly divided into image blocks with a size of 10*10 pixels. In another embodiment, the first picture can also be divided into a preset number of image blocks with unequal shapes according to information such as the color, similarity, or depth of field of each scene in the first picture.

[0029] S212, determining the average value of all the second information entropy values as the first information entropy value of the first picture. It can be understood that after dividing the first picture into a preset number of image blocks and determining the second information entropy value corresponding to each of the image blocks, determining the average value of all the second information entropy values as the first information entropy value of the first picture can obtain a more accurate result of the first information entropy value of the first picture, and at the same time avoid the situation that the first information entropy value fluctuates greatly at different times.

[0030] S220. Determine the scene information corresponding to the current position of the self - moving device, and obtain the preset information entropy range corresponding to the scene information. Understandably, the preset information entropy range corresponds to the scene information and can be dynamically set according to different scene information corresponding to the current position of the self - moving device. The scene information includes, but is not limited to, one or more of time information, weather information, and environmental information, etc. For example, the scene information may include: an outdoor scene (with sundries), a scene with a lot of snow (but the sundries are covered by the snow) and dim light, which is a simple scene; an outdoor scene (with sundries), a daytime and sunny (no snow covering the sundries and bright light) scene, which is a complex scene; a night (dim light), courtyard (many sundries) and little snow (the snow does not cover the sundries) scene, which is a complex scene; a night, courtyard (more sundries), little snow (the snow does not cover the sundries) and bright light scene, which is a complex scene; a camera facing a single - color wall (single color), which is a simple scene. In the case of the above complex scenes, the preset information entropy range can be set in a relatively high information entropy range. In the case of simple scenes, the preset information entropy range can be set in a relatively low information entropy range. The preset information entropy range can be first determined by means of experiments under different scene information, and then continuously optimized according to the feedback during use. The scene information can be determined by sensors (such as photosensitive sensors, laser sensors or vision sensors, etc.) and a networking module. Among them, the sensor can determine the current environmental brightness and whether there are sundries and snow, etc., and the networking module can determine the current time and weather, etc. In one embodiment, the preset information entropy range is between a first threshold and a second threshold, and the second threshold is greater than the first threshold.

[0031] S230. When the first information entropy value is within the preset information entropy range, determine that there is an occlusion phenomenon for the camera corresponding to the first picture. Understandably, in step S220, the preset information entropy range corresponding to the scene information is determined. In this embodiment, by comparing the first information entropy value with the preset information entropy range, if the first information entropy value is within the preset information entropy range, it can be determined that there is an occlusion phenomenon for the camera corresponding to the first picture.

[0032] In one embodiment, after step S220, that is, after determining the scene information corresponding to the current position of the self - moving device and obtaining the preset information entropy range corresponding to the scene information, it further includes:

[0033] S240. When the first information entropy value is greater than the third threshold, it is determined that the camera corresponding to the first picture has no occlusion phenomenon, where the third threshold is greater than the second threshold. Understandably, the third threshold can also be dynamically set according to different scene information corresponding to the current position of the self-mobile device. If the first information entropy value is greater than the third threshold, it can be determined that the camera corresponding to the first picture has no occlusion phenomenon. If the first information entropy value is between the second threshold and the third threshold, the camera corresponding to the first picture may have occlusion, but the occlusion may not be obvious and there is no need to process the above occlusion, so as to save the power consumption to a certain extent on the premise of ensuring the normal recognition work of the camera of the self-mobile device.

[0034] As Figure 3 shown, in one embodiment, step S200, determining whether the camera corresponding to the first picture has an occlusion phenomenon according to the first picture, includes steps S250 to S260:

[0035] S250. Perform edge detection based on the Sobel operator or the Canny operator to obtain the target edge value of the first picture. The target edge value refers to the edge information corresponding to the picture edge of the first picture (the picture edge is white and the non-picture edge is black). Therefore, if there is an occluded area in the first picture, the larger the occluded area, the higher the degree of occlusion of the border of the first picture (the less white corresponding to the picture edge), and the less the target edge information contained in the first picture. When the first picture is completely occluded, its target edge information is 0 (the corresponding position of the picture edge of the first picture is completely occluded and changes from white to black).

[0036] In one embodiment, when performing edge detection on the first picture based on the Sobel operator, the target edge value of the first picture can be obtained by counting the number of pixel points with a pixel value of 1 corresponding to the picture edge in the first picture (the pixel points with a pixel value of 1 corresponding to the picture edge are white pixel points; the black pixel points have a pixel value of 0).

[0037] In one embodiment, for edge detection based on the Canny operator, the first image can be first converted into a grayscale image, and then the values of all pixel points in the grayscale image (at this time, the pixel points corresponding to the edges of the picture with a pixel value of 255 are white pixel points; the black pixel points have a pixel value of 0) are added up and divided by the total number of pixel points, so as to obtain the target edge value of the first picture; it is also possible to count the number of pixel points with a pixel value of 255 in the grayscale image and divide it by the total number of pixel points to obtain the proportion of pixel points with a pixel value of 255 as the target edge value of the first picture. When performing edge detection based on the Canny operator, the threshold during the detection of the Canny operator can also be adjusted to adapt to the change of ambient light.

[0038] S260. When the target edge value is within the preset edge value range, it is determined that there is an occlusion phenomenon for the camera corresponding to the first picture. It can be understood that the preset edge value range can be first determined by experiments under different scenario information, and then continuously optimized according to the feedback during use. By comparing the target edge value with the preset edge value range, if the target edge value is within the preset edge value range, it can be determined that there is an occlusion phenomenon for the camera corresponding to the first picture.

[0039] In one embodiment, after step S250, that is, after performing edge detection based on the Sobel operator or the Canny operator to obtain the target edge value of the first picture, the following steps are further included:

[0040] S270. When the target edge value exceeds the preset edge value range, it is determined that there is no occlusion phenomenon for the camera corresponding to the first picture. Herein, the non-occlusion phenomenon in the present invention means that the camera is completely unoccluded and thus does not need to be processed, or the camera may have a certain occlusion, but the occlusion may not be obvious and there is no need to process the above occlusion, so as to save the power consumption to a certain extent while ensuring the normal recognition work of the camera of the self-mobile device. The occlusion phenomenon in the present invention means that the camera has a large degree of occlusion or is completely occluded, so the occluding object needs to be processed, otherwise it will affect the continued operation of the self-mobile device.

[0041] In one embodiment, when the ambient light is strong (such as when the car headlight or strong flashlight shines directly on the camera), the first picture can also be optimized by the method of histogram equalization, so as to avoid being affected by strong light when determining whether there is an occlusion phenomenon for the camera corresponding to the first picture.

[0042] S300. After determining that there is an occlusion phenomenon in the camera, perform an occlusion processing operation. Understandably, after determining that there is an occlusion phenomenon in the camera, the occlusion processing operation can be performed immediately. Among them, the occlusion operation processing can include heating and jittering the occluder to automatically remove the occluder, or prompting or alarming a preset processor to prompt the preset processor to process the occluder. This is not limited here.

[0043] In one embodiment, in step S300, the performing the occlusion processing operation includes steps S400 and S500:

[0044] S400. Determine the occlusion type corresponding to the occlusion phenomenon. Among them, the occlusion type can be used to characterize whether the occluder can be removed, or / and the method of processing the occlusion phenomenon.

[0045] As Figure 4 shown, in one embodiment, in step S400, the determining the occlusion type corresponding to the occlusion phenomenon includes steps S401 and S402:

[0046] S401. Identify the first feature information of the occluder in the first picture through a preset image recognition model. Understandably, the preset image recognition model can be set according to the actual situation. For example, it can be trained by a neural network model according to different image samples. Each image sample includes at least one feature information and the feature identifier corresponding to this feature information. After the neural network model sequentially identifies each image sample to obtain the corresponding feature recognition result, if the feature recognition result does not match the feature identifier of the image sample corresponding to it, then adjust the model parameters of the neural network model to iteratively train the neural network model until the feature recognition result obtained by the iteratively trained neural network model for the image sample matches the feature identifier of the image sample corresponding to this feature recognition, indicating that the neural network model has been trained, and the trained neural network model is the above-mentioned preset image recognition model. That is, the preset image recognition model can accurately identify the first feature information corresponding to the occluder in the first picture. In one embodiment, the first feature information includes but is not limited to the occluder edge information, the occluder blurred gray scale information, and the occluder irregular distribution information, etc.

[0047] S402. Determine the type of the obstacle according to the first feature information, and determine the occlusion type according to the type of the obstacle. The occlusion type includes a heating removal type, a shaking removal type, and a non-removable type. Understandably, determining the occlusion type can facilitate the removal of the obstacle according to the occlusion type. The types of the obstacle include, but are not limited to, one or more of water vapor, water droplets, snow, ice, etc. In one embodiment, the step of determining the type of the obstacle according to the first feature information and determining the occlusion type according to the type of the obstacle includes:

[0048] When determining that the type of the obstacle is ice and snow (snow or ice) according to the edge information of the obstacle, determine that the occlusion type is the heating removal type and / or the shaking removal type. Among them, the heating removal type can remove the obstacle by heating; the shaking removal type can remove the obstacle by shaking. That is, in this embodiment, it is considered that the ice and snow type of obstacle can be removed by heating and / or shaking. In this embodiment, the edge information of the obstacle corresponding to the ice and snow is regarded as the first feature information. Therefore, the edge information of the obstacle corresponding to the first picture can be obtained first (the edge information of the obstacle can be the edge shape identified by image recognition technology). Then, according to the edge shape corresponding to the above-mentioned edge information of the obstacle, it can be identified whether it is ice and snow. The specific identification method can be to determine a number of standard edge shape images according to the edge shape of the actual ice and snow. Then, the edge information of the identified obstacle is matched with the standard edge shape image. If the match is successful, the type of the obstacle is regarded as ice and snow; if the match fails, the type of the obstacle is regarded as not ice and snow.

[0049] When determining that the type of the obstacle is mirror water vapor according to the blurred gray-scale information of the obstacle, determine that the occlusion type is the heating removal type. That is, in this embodiment, it is considered that the mirror water vapor type of obstacle can be removed by heating. The mirror water vapor makes the camera blurred. Therefore, the blurred gray-scale information of the obstacle can be determined first. The blurred gray-scale information of the obstacle can be obtained by first performing gray-scale processing on the first picture, then obtaining the average value of the gray-scale values of the first picture, and then comparing the average value with a preset value to obtain the blurred gray-scale information of the obstacle (that is, the first feature information). That is, if the average value is greater than the preset value, it is considered to be mirror water vapor; if the average value is less than the preset value, it is considered not to be mirror water vapor.

[0050] When determining that the type of the occluder is a water droplet according to the irregular distribution information of the occluder, determine that the occlusion type is a heating removal type and / or a jitter removal type. That is, in this embodiment, it is considered that occluders of the water droplet type can be removed by heating and / or jittering. In this embodiment, the irregular distribution information of the occluder corresponding to the water droplet is regarded as the first feature information. Therefore, the irregular distribution information of the occluder corresponding to the occluder in the first picture can be obtained first (the edge information of the occluder can be used to extract the image area corresponding to the occluder in the first picture through image recognition technology). Furthermore, based on the extracted edge information of the occluder, it can be identified whether it is a water droplet. The specific identification method can be to determine a number of standard irregular distribution images through the irregular distribution state of actual water droplets. Then, the irregular distribution information of the identified occluder is matched with the standard irregular distribution images. If the match is successful, it is considered that the type of the occluder is a water droplet; if the match fails, it is considered that the type of the occluder is not a water droplet. In addition, the jitter mentioned in the present invention can refer to a series of actions or combined actions such as the self-mobile device walking or rotating.

[0051] S500. Perform an occlusion processing operation on the camera according to the occlusion type. It can be understood that after determining that there is an occlusion phenomenon in the camera, by determining the occlusion type corresponding to the occlusion phenomenon and performing an occlusion processing operation on the camera according to the occlusion type, on the premise of ensuring the normal recognition work of the camera in the self-mobile device, an accurate occlusion processing operation can also be performed on the occluder according to the occlusion type, reducing the power consumption, thereby improving the processing efficiency of the occluder.

[0052] As Figure 5 shown, in one embodiment, in step S500, the performing an occlusion processing operation on the camera according to the occlusion type includes steps S501 to S504:

[0053] S501. When determining that the occlusion type is a heating removal type, control a heating module installed on the camera to heat the occluder. It can be understood that a heating module is installed on the camera, and the heating module can be used to heat the lens of the camera, thereby removing the occluder on the lens of the camera. The heating module can be a hot air device that heats the camera by blowing hot air to the camera; the heating module can also be a heating device that heats the camera by directly transferring heat to the camera by being installed on or near the lens of the camera.

[0054] S502. After continuously heating for the first preset duration, obtain a second picture captured by the camera. Understandably, the second picture is the picture captured by the camera after continuously heating for the first preset duration. The first preset duration can be set according to actual situations. For example, it can be set according to the type of the occluder. The first preset duration set when the type of the occluder is ice and snow can be greater than the first preset duration set when the type of the occluder is water vapor.

[0055] S503. Determine whether the occluder has been cleared according to the second picture.

[0056] In one embodiment, the method in step S200 can be used to determine whether there is an occlusion phenomenon for the corresponding camera according to the second picture. If it is determined that there is no occlusion phenomenon, it can be determined that the occluder has been cleared.

[0057] In one embodiment, the second feature information of the occluder in the second picture can also be recognized by the preset image recognition model. The second feature information includes occluder edge information, occluder blurred gray-scale information, and occluder irregular distribution information, etc. Then, determine whether the occluder has been cleared according to the above-mentioned second feature information. For example, when the second feature information indicates that there is no longer an occluder on the camera, it can be determined that the occluder has been cleared. Understandably, when it is determined by the second feature information that the occluder has been reduced to a preset feature threshold, it can also be considered that the occluder will no longer affect the normal recognition work of the camera of the self-mobile device. At this time, it can also be determined that the occluder has been cleared.

[0058] S504. If it is confirmed that the occluder has been cleared, control the heating module to stop heating and prompt information indicating that the occluder has been cleared.

[0059] In the above embodiments of the present invention, first, after obtaining the first picture captured by the camera of the self-mobile device, determine whether there is an occlusion phenomenon for the corresponding camera through the first picture (such as the first information entropy value or the target edge value of the first picture, etc.), thereby improving the detection accuracy of the occlusion phenomenon and shortening the detection time of the occlusion phenomenon, and thus improving the processing efficiency of the occluder.

[0060] In one embodiment, step S500, the performing an occlusion processing operation on the camera according to the occlusion type, further includes:

[0061] S505. When it is determined that the occlusion type is the non-removable type, prompt the preset processing party with information indicating that the occluder cannot be removed. That is, when the occlusion type is the non-removable type, information indicating that the occluder cannot be removed can be sent to the preset processing party or an alarm can be directly triggered, so that the preset processing party can perform manual processing on the occluder based on the above prompt, etc.

[0062] As Figure 6 shown, in one embodiment, after step S503, that is, after determining whether the occluder has been removed based on the second picture, steps S506 to S512 are further included:

[0063] S506. If it is confirmed that the occluder has not been removed, identify the second feature information of the occluder in the second picture through the preset image recognition model. In this embodiment, control the heating module installed on the camera to continuously heat the occluder for a first preset duration to determine whether the occluder has decreased.

[0064] S507. Determine whether the occluder has decreased based on the second feature information and the first feature information. Understandably, after comparing the size of the second feature information with the size of the first feature information, if the size of the second feature information is smaller than the size of the first feature information (or the size of the second feature information is smaller than the size of the first feature information by a certain size difference), it can be determined that the occluder has decreased.

[0065] S508. After determining that the occluder has decreased, control the heating module to continue heating the occluder. Understandably, the occluder may be an object that cannot be removed or is difficult to remove by heating. Determine whether the occluder has decreased through the method in step S470. After determining that the occluder has decreased, then control the heating module to continue heating the occluder, which can avoid the waste of power caused by the heating module heating the camera ineffectively for a long time.

[0066] S509. After continuously heating for a second preset duration, obtain the third picture taken by the camera. Understandably, the third picture is the picture taken by the camera after continuously heating for the second preset duration. The second preset duration can be set according to the actual situation, for example, it can be set according to the type of the occluder. In this embodiment, the first preset duration is used to preheat the occluder to further determine whether the occluder is of the heat-removable type, and only short-time heating of the occluder is required. And the second preset duration is used to remove the occluder. Therefore, in one embodiment, the first preset duration is less than the second preset duration.

[0067] S510. Determine whether the occluder has been cleared according to the third picture.

[0068] In one embodiment, the method in step S200 can be used to determine whether there is an occlusion on the corresponding camera according to the third picture. If it is determined that there is no occlusion, it can be determined that the occluder has been cleared.

[0069] In one embodiment, the third feature information of the occluder in the third picture can also be recognized by the preset image recognition model. The third feature information includes occluder edge information, occluder blurred gray-scale information, and occluder irregular distribution information, etc. Determine whether the occluder has been cleared according to the third feature information. For example, when the third feature information indicates that there is no longer an occluder on the camera, it can be determined that the occluder has been cleared. Understandably, when it is determined through the third feature information that the occluder has been reduced to a preset feature threshold, it can also be considered that the occluder will no longer affect the normal recognition work of the camera of the self-mobile device. At this time, it can also be determined that the occluder has been cleared.

[0070] S511. If it is confirmed that the occluder has been cleared, control the heating module to stop heating and prompt the information that the occluder has been cleared. Furthermore, the self-mobile device and its camera can be controlled to work normally.

[0071] S512. If it is confirmed that the occluder has not been cleared, after determining that the occluder has been reduced again, control the heating module to continue heating the occluder until it is confirmed that the occluder has been cleared, then control the heating module to stop heating and prompt the information that the occluder has been cleared. In this embodiment, after controlling the heating module to continue heating the occluder, the method in steps S509 to S510 can be used to confirm whether the occluder has been cleared. If the occluder has not been completely cleared after heating and clearing for a preset number of rounds, an abnormal occluder clearing information can be prompted to a preset processing party, indicating that the occluder may be a mixed occluder, that is, the occluder may simultaneously include an occluder of the heating and clearing type and an occluder of the non-clearing type. Prompting the preset processing party with the abnormal occluder clearing information can be directly sending the abnormal occluder clearing information to the preset processing party or directly issuing an alarm to prompt the preset processing party to perform manual processing.

[0072] In one embodiment, the heating module may include a plurality of sub-modules distributed in different regions of the camera; the camera may determine the region where the occluder is located based on the first feature information of the occluder; when controlling the heating module to heat the occluder, the occluder can be heated in a targeted manner by controlling the module in the region where the occluder is located, thereby improving the accuracy of heating and the heating efficiency of the heating module at the same time.

[0073] In one embodiment, in step S509, before continuously heating for a second preset duration, steps S513 and S514 are included:

[0074] S513, obtain the difference information between the first feature information and the second feature information. It can be understood that the difference information may be the ratio or the numerical value by which the size of the second feature information is reduced relative to the size of the first feature information.

[0075] S514, determine the second preset duration according to the first preset duration and the difference information. It can be understood that the difference information reflects the degree of clearing of the occluder (i.e., the reduced ratio or numerical value) after heating the occluder by controlling the heating module installed on the camera for the first preset duration. According to the above-mentioned clearing degree, the second preset duration can be determined to clear the occluder through heating for the second preset duration, thereby avoiding waste of power caused by overheating for too long a time or being unable to completely clear the occluder due to heating for too short a time, and improving the clearing efficiency.

[0076] In one embodiment, in step S500, when performing the occlusion processing operation on the camera according to the occlusion type, steps S515 to S523 are further included:

[0077] S515, when determining that the occlusion type is the jitter clearing type, control the self-moving device to reciprocate according to the initial jitter mode. It can be understood that when determining that the occlusion type is the jitter clearing type (such as water droplets or snow accumulation), the occluder can also be cleared by controlling the self-moving device to reciprocate according to the initial jitter mode. Among them, the reciprocating speed and reciprocating frequency corresponding to the initial jitter mode are relatively low compared with the high-speed jitter mode described later. In one embodiment, when the occlusion type of the occluder includes both the jitter clearing type and the heating clearing type at the same time, the self-moving device can be controlled to reciprocate according to the initial jitter mode and the heating module installed on the camera can be controlled to heat the occluder at the same time, thereby accelerating the time for clearing the occluder.

[0078] S516. After the third preset duration of continuous reciprocating motion, obtain a fourth picture captured by the camera. Understandably, the fourth picture is the picture captured by the camera after the third preset duration of continuous reciprocating motion. The third preset duration can be set according to the actual situation.

[0079] S517. Determine whether the occluder has been cleared based on the fourth picture.

[0080] In one embodiment, the method in step S200 can be followed to determine whether there is an occlusion phenomenon for the corresponding camera based on the fourth picture; if it is determined that there is no occlusion phenomenon, it can be determined that the occluder has been cleared.

[0081] In one embodiment, the fourth feature information of the occluder in the fourth picture can also be recognized by the preset image recognition model. The fourth feature information includes occluder edge information, occluder blurred gray-scale information, and occluder irregular distribution information, etc.; determine whether the occluder has been cleared based on the fourth feature information. For example, when the fourth feature information indicates that there is no longer an occluder on the camera, it can be determined that the occluder has been cleared. Understandably, when it is determined through the fourth feature information that the occluder has been reduced to a preset feature threshold, it can also be considered that the occluder will no longer affect the normal recognition work of the camera of the self-mobile device, and it can be determined that the occluder has been cleared.

[0082] S518. If it is confirmed that the occluder has been cleared, control the self-mobile device to stop moving and prompt information indicating that the occluder has been cleared. Furthermore, the self-mobile device and its camera can be controlled to work normally.

[0083] S519. If it is confirmed that the occluder has not been cleared, control the self-mobile device to reciprocate in a high-speed jitter mode. The reciprocating speed and reciprocating frequency of the high-speed jitter mode are both greater than those of the initial jitter mode. Understandably, if the occluder cannot be completely cleared through the initial jitter mode, the self-mobile device can be controlled to reciprocate in a high-speed jitter mode with a higher reciprocating speed to clear the occluder.

[0084] S520. After the fourth preset duration of continuous reciprocating motion, obtain the fifth picture captured by the camera. Understandably, the fifth picture is the picture captured by the camera after the fourth preset duration of continuous reciprocating motion. The fourth preset duration can be set according to the actual situation. In one embodiment, if it is confirmed that the occluder has not been cleared, the degree of clearing of the occluder can be determined by comparing the fourth picture with the first picture through continuous reciprocating motion for the fourth preset duration, so as to determine the fourth preset duration, thereby avoiding waste of power caused by reciprocating motion for too long a time or the occluder not being completely cleared due to reciprocating motion for too short a time.

[0085] S521. Determine whether the occluder has been cleared according to the fifth picture.

[0086] In one embodiment, the method in step S200 can be followed to determine whether there is an occlusion phenomenon of the corresponding camera according to the fifth picture; if it is determined that there is no occlusion phenomenon, it can be determined that the occluder has been cleared.

[0087] In one embodiment, the fifth feature information of the occluder in the fifth picture can also be recognized by the preset image recognition model. The fifth feature information includes occluder edge information, occluder blurred gray-scale information, and occluder irregular distribution information, etc.; determine whether the occluder has been cleared according to the fifth feature information. For example, when the fifth feature information indicates that there is no longer an occluder on the camera, it can be determined that the occluder has been cleared; understandably, when it is determined through the fifth feature information that the occluder has been reduced to the preset feature threshold, it can also be considered that the occluder will no longer affect the normal recognition work of the camera of the self-moving device, and it can be determined that the occluder has been cleared.

[0088] S522. If it is confirmed that the occluder has been cleared, control the self-moving device to stop moving and prompt the information that the occluder has been cleared. Furthermore, the self-moving device and its camera can be controlled to work normally.

[0089] S523. If it is confirmed that the occluder has not been cleared, prompt the information that the occluder clearing is abnormal. Understandably, if it is confirmed that the occluder has not been cleared, the information that the occluder clearing is abnormal can be prompted to the preset processing party, indicating that the occluder may contain some objects that cannot be cleared by heating. Prompting the information that the occluder clearing is abnormal to the preset processing party can be directly sending the information that the occluder clearing is abnormal to the preset processing party or directly giving an alarm to prompt the preset processing party to perform manual processing.

[0090] Understandably, in this embodiment, the occluder of the jitter clearing type is cleared only by two movements with different jitter patterns. If it cannot be completely cleared after two times, an abnormal occluder clearing message is prompted, so that excessive power consumption can be avoided; in the present invention, it is also possible to set jitter clearing with more jitter times (the number of jitter times is set according to requirements, and the jitter clearing method can refer to the above embodiment), which is not limited herein to achieve further automatic clearing of the occluder.

[0091] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0092] In one embodiment, the present invention further provides a self - moving device, including a camera and a control module communicatively connected to the camera. The control module is used to execute the steps of the above - mentioned method for processing occlusion of the camera of the self - moving device. The self - moving device can be a courtyard robot such as a snow - sweeping robot or a weeding robot. In one embodiment, a front - facing camera can be arranged at the front of the self - moving device, and side cameras can be respectively arranged on both sides of the body of the self - moving device. Among them, at least one heating module is arranged in each camera.

[0093] The execution functions of the control module correspond one - to - one with the method for processing occlusion of the camera of the self - moving device in the above embodiment. For specific limitations on the control module, reference can be made to the limitations on the method for processing occlusion of the camera of the self - moving device in the above text, which will not be elaborated herein. Each sub - module in the above - mentioned control module can be implemented in whole or in part by software, hardware, and their combination. The above - mentioned each sub - module can be embedded in or independent of the processor in the control module in the form of hardware, or stored in the memory in the control module in the form of software, so that the processor can call and execute the operations corresponding to each of the above - mentioned sub - modules.

[0094] In one embodiment, the present invention further provides one or more readable storage media storing computer - readable instructions. The readable storage media provided in this embodiment include non - volatile readable storage media and volatile readable storage media; computer - readable instructions are stored on the readable storage media. When the computer - readable instructions are executed by one or more processors, one or more processors implement the steps of the method for processing occlusion of the camera of the self - moving device in the above embodiment.

[0095] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above-described embodiments can be completed by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above various methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or an external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0096] Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0097] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for processing the occlusion of a camera of a self - moving device, characterized in that, The method includes: Obtaining a first picture taken by a camera of a self - moving device; Determining whether there is an occlusion phenomenon for the corresponding camera according to the first picture; After determining that there is an occlusion phenomenon for the camera, performing an occlusion processing operation; The performing the occlusion processing operation includes: Determining an occlusion type corresponding to the occlusion phenomenon; the occlusion type includes a heating - clearing type, a jitter - clearing type, and an unclearing type; Performing an occlusion processing operation on the camera according to the occlusion type; The performing the occlusion processing operation on the camera according to the occlusion type includes: When determining that the occlusion type is the heating - clearing type, controlling a heating module installed on the camera to heat an occluder; After continuously heating for a first preset duration, obtaining a second picture taken by the camera; Determining whether the occluder has been cleared according to the second picture; If it is confirmed that the occluder has been cleared, controlling the heating module to stop heating and prompting information indicating that the occluder has been cleared; After determining whether the occluder has been cleared according to the second picture, it further includes: If it is confirmed that the occluder has not been cleared, identifying second feature information of the occluder in the second picture through a preset image recognition model; Determining whether the occluder has decreased according to a comparison of the size of the second feature information and the size of first feature information of the occluder in the first picture; After determining that the occluder has decreased, controlling the heating module to continue heating the occluder; After continuously heating for a second preset duration, obtaining a third picture taken by the camera; Determining whether the occluder has been cleared according to the third picture; If it is confirmed that the occluder has been cleared, controlling the heating module to stop heating and prompting information indicating that the occluder has been cleared; If it is confirmed that the occluder has not been cleared, after determining that the occluder has decreased again, controlling the heating module to continue heating the occluder until it is confirmed that the occluder has been cleared, then controlling the heating module to stop heating and prompting information indicating that the occluder has been cleared; The determining the occlusion type corresponding to the occlusion phenomenon includes: Identifying first feature information of the occluder in the first picture through a preset image recognition model; Determining the occluder type according to the first feature information and determining the occlusion type according to the occluder type; The first feature information includes occluder edge information, occluder blurred gray - scale information, and occluder irregular distribution information; the determining the occluder type according to the first feature information and determining the occlusion type according to the occluder type includes: When determining that the occluder type is ice and snow according to the occluder edge information, determining that the occlusion type is the heating - clearing type and / or the jitter - clearing type; When determining that the occluder type is mirror water vapor according to the occluder blurred gray - scale information, determining that the occlusion type is the heating - clearing type; When determining that the occluder type is water droplets according to the occluder irregular distribution information, determining that the occlusion type is the heating - clearing type and / or the jitter - clearing type; Before continuously heating for the second preset duration, it includes: Obtaining the difference information between the first feature information and the second feature information; Determining the second preset duration according to the first preset duration and the difference information.

2. The method for processing the occlusion of the camera of the self - moving device according to claim 1, wherein, The determining whether there is an occlusion phenomenon of the camera corresponding to the first picture includes: Determining the first information entropy value corresponding to the first picture; Determining the scene information corresponding to the current position of the self - moving device, and obtaining the preset information entropy range corresponding to the scene information; When the first information entropy value is within the preset information entropy range, determining that there is an occlusion phenomenon of the camera corresponding to the first picture; When the first information entropy value exceeds the preset information entropy range, determining that there is no occlusion phenomenon of the camera corresponding to the first picture.

3. The method for processing the occlusion of the camera of the self - moving device according to claim 2, wherein, The determining the first information entropy value corresponding to the first picture includes: Dividing the first picture into a preset number of image blocks, and determining the second information entropy values respectively corresponding to each of the image blocks; Determining the average value of all the second information entropy values as the first information entropy value of the first picture.

4. The method for processing the occlusion of the camera of the self - moving device according to claim 1, wherein, The determining whether there is an occlusion phenomenon of the camera corresponding to the first picture includes: Performing edge detection based on the Sobel operator or the Canny operator to obtain the target edge value of the first picture; When the target edge value is within the preset edge value range, determining that there is an occlusion phenomenon of the camera corresponding to the first picture; When the target edge value exceeds the preset edge value range, determining that there is no occlusion phenomenon of the camera corresponding to the first picture.

5. The method for processing the occlusion of the camera of the self - moving device according to claim 1, characterized in that, The performing occlusion processing operations on the camera according to the occlusion type further includes: When determining that the occlusion type is the jitter - clearing type, controlling the self - moving device to reciprocate according to the initial jitter mode; After continuously reciprocating for the third preset duration, obtaining the fourth picture taken by the camera; Determining whether the occluder has been cleared according to the fourth picture; If it is confirmed that the occluder has been cleared, controlling the self - moving device to stop moving, and prompting information indicating that the occluder has been cleared; If it is confirmed that the occluder has not been cleared, controlling the self - moving device to reciprocate according to the high - speed jitter mode; After continuously reciprocating for the fourth preset duration, obtaining the fifth picture taken by the camera; Determining whether the occluder has been cleared according to the fifth picture; If it is confirmed that the occluder has been cleared, controlling the self - moving device to stop moving, and prompting information indicating that the occluder has been cleared; If it is confirmed that the occluder has not been cleared, prompting information indicating abnormal occlusion clearing.

6. A self - moving device, characterized in that, It includes a camera and a control module communicatively connected to the camera, and the control module is used to execute the steps of the method for processing occlusion of the camera of the self - moving device according to any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it realizes the steps of the method for processing occlusion of the camera of the self - moving device according to any one of claims 1 to 5.

Images