Obstruction state determination method and apparatus, and electronic device

Abstract

The embodiments of the present application belong to the technical field of machine vision. Provided are an obstruction state determination method and apparatus, and an electronic device. The method comprises: during a process of controlling a robot to move, acquiring first environmental information detected by a first visual sensor of the robot, and determining an initial obstruction state of the first visual sensor on the basis of the first environmental information; when the initial obstruction state of the first visual sensor is that there is an obstruction, acquiring second environmental information detected by a second visual sensor; on the basis of the second environmental information, determining whether the robot is in a bright environment; and if the robot is in a bright environment, determining that a target obstruction state of the first visual sensor is that there is an obstruction. The obstruction state determination method provided in the embodiments of the present application can improve the accuracy of determination for the obstruction state of a visual sensor.

Description

Occlusion state determination method and device and electronic equipment TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and in particular to an occlusion state determination method and device and electronic equipment. BACKGROUND

[0002] A vision sensor is a sensing device used to perceive the external environment of a robot, which is composed of a camera, an image sensor, a fill light and other hardware components, and can capture optical information in a scene and convert it into a digital image or video signal.

[0003] In related technologies, a traditional image or deep learning method is used to detect whether a vision sensor of a robot is occluded. However, images collected by the vision sensor in a dark environment are easily misjudged as an occlusion state, thereby interfering with the detection result and reducing the accuracy of occlusion state determination of the vision sensor. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide an occlusion state determination method and device and electronic equipment, which can improve the accuracy of occlusion state determination of a vision sensor.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application provides an occlusion state determination method, which comprises: in the process of controlling the movement of a robot, acquiring first environment information detected by a first vision sensor of the robot, and determining an initial occlusion state of the first vision sensor according to the first environment information; when the initial occlusion state of the first vision sensor is occlusion, acquiring second environment information detected by a second vision sensor; judging whether the robot is in a bright light environment according to the second environment information; and if the robot is in a bright light environment, determining that a target occlusion state of the first vision sensor is occlusion.

[0006] To achieve the above purpose, a second aspect of the embodiments of the present application provides an occlusion state determination device, which comprises: an initial occlusion state module, configured to acquire first environment information detected by a first vision sensor of a robot in the process of controlling the movement of the robot, and determine an initial occlusion state of the first vision sensor according to the first environment information; a second environment information module, configured to acquire second environment information detected by a second vision sensor when the initial occlusion state of the first vision sensor is occlusion; an environment brightness determination module, configured to judge whether the robot is in a bright light environment according to the second environment information; and an occlusion determination module, configured to determine that a target occlusion state of the first vision sensor is occlusion if the robot is in a bright light environment.

[0007] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect of the embodiments when executing the computer program.

[0008] To achieve the above object, a fourth aspect of the embodiments of the present application provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect of the embodiments.

[0009] The shielding state determination method, device and electronic device provided by the present application have the following beneficial effects: in the process of controlling the movement of the robot, the first environment information detected by the first visual sensor of the robot is acquired, the initial shielding state of the first visual sensor is determined according to the first environment information, when the initial shielding state of the first visual sensor is existing shielding, the second environment information detected by the second visual sensor is acquired, whether the robot is in a bright light environment is determined according to the second environment information, if the robot is in the bright light environment, the target shielding state of the first visual sensor is determined as existing shielding, the information of the first visual sensor and the second visual sensor is combined, the initial shielding state of the first visual sensor is determined according to the first environment information, and whether the robot is in the bright light environment is determined according to the second environment information, so as to avoid the influence of factors such as dark light environment to cause misjudgment of the shielding state, the target shielding state is determined by comprehensive analysis, and the accuracy of the shielding state determination of the visual sensor can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] Fig. 1 is one optional flowchart of the shielding state determination method provided by the embodiments of the present application.

[0011] Fig. 2 is a schematic diagram of the visual sensor on the robot provided by the embodiments of the present application.

[0012] Fig. 3 is another optional flowchart of the shielding state determination method provided by the embodiments of the present application.

[0013] Fig. 4 is a flowchart of step 103 in Fig. 1.

[0014] Fig. 5 is another optional flowchart of the shielding state determination method provided by the embodiments of the present application.

[0015] Fig. 6 is another optional flowchart of the shielding state determination method provided by the embodiments of the present application.

[0016] Fig. 7 is another optional flowchart of the shielding state determination method provided by the embodiments of the present application.

[0017] FIG. 8 is another optional flowchart of the method for determining the occlusion state according to an embodiment of the present application.

[0018] FIG. 9 is another optional flowchart of the method for determining the occlusion state according to an embodiment of the present application.

[0019] FIG. 10 is another optional flowchart of the method for determining the occlusion state according to an embodiment of the present application.

[0020] FIG. 11 is a flowchart of step 902 in FIG. 10.

[0021] FIG. 12 is another optional flowchart of the method for determining the occlusion state according to an embodiment of the present application.

[0022] FIG. 13 is a flowchart of step 1102 in FIG. 12.

[0023] FIG. 14 is another optional flowchart of the method for determining the occlusion state according to an embodiment of the present application.

[0024] FIG. 15 is a flowchart of step 1302 in FIG. 14.

[0025] FIG. 16 is another optional flowchart of the method for determining the occlusion state according to an embodiment of the present application.

[0026] FIG. 17 is a schematic diagram of functional modules of the device for determining the occlusion state according to an embodiment of the present application.

[0027] FIG. 18 is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0029] It should be noted that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing the embodiments of the present application only and is not intended to limit the present application.

[0031] A visual sensor is a sensing device for perceiving the external environment of a robot, which is composed of a camera, an image sensor, a fill light and other hardware components, and can capture optical information in a scene and convert it into a digital image or video signal.

[0032] In the related art, a traditional image or deep learning method is used to detect whether the visual sensor of the robot is blocked, however, the image collected by the visual sensor in a dark environment is easy to be misjudged as a blocked state, thereby interfering with the detection result and reducing the accuracy of the determination of the blocked state of the visual sensor.

[0033] Based on this, the embodiments of the present application provide a blocked state determination method, device and electronic equipment, which can improve the accuracy of the determination of the blocked state of the visual sensor.

[0034] The blocked state determination method, device and electronic equipment provided by the embodiments of the present application are specifically described by the following embodiments. First, the blocked state determination method in the embodiments of the present application is described. The blocked state determination method in the embodiments of the present application can be described by the following embodiments.

[0035] FIG. 1 is an optional flowchart of the blocked state determination method provided by the embodiments of the present application. The method in FIG. 1 can include, but is not limited to, steps 101 to 104. It can be understood that the order of steps 101 to 104 in FIG. 1 is not specifically limited in the embodiments, and the order of steps can be adjusted or some steps can be reduced, added or increased according to actual needs.

[0036] Step 101, in the process of controlling the movement of the robot, first environment information detected by a first visual sensor of the robot is acquired, and an initial blocked state of the first visual sensor is determined according to the first environment information.

[0037] Step 101 is described in detail as follows.

[0038] It can be understood that the robot is an automatic device capable of performing specific tasks, which is composed of a body, a perception module, an execution module, a control module and the like, and is applied to various industries, services, gardens and the like, such as a mowing robot. In the present application, the perception module of the robot at least includes a first visual sensor and a second visual sensor, which are used to acquire environmental information (including but not limited to image or video data) and can be monocular camera, multi-view camera, infrared sensor, laser radar and the like. In some embodiments, the types of the first visual sensor and the second visual sensor are the same, for example, one of monocular camera, multi-view camera, infrared sensor, laser radar. In other embodiments, the types of the first visual sensor and the second visual sensor are different, for example, the first visual sensor is a monocular camera and the second visual sensor is a multi-view camera.

[0039] The first visual sensor and the second visual sensor are installed at different positions of the robot and have different sensing areas. For example, referring to FIG. 2, the robot is equipped with a visual sensor module, which is used to detect or perceive environmental information to achieve obstacle avoidance and positioning. The visual sensing module includes a front side visual sensor located in front of the robot, a left side visual sensor located on the left side of the robot and a right side visual sensor located on the right side of the robot. The front side visual sensor is used for positioning, obstacle avoidance and boundary detection, and the left side or right side visual sensor is used for boundary detection. The first visual sensor can be any one or more of the front side visual sensor, the left side visual sensor and the right side visual sensor configured on the robot, and the second visual sensor is any two or more of the visual sensors other than the first visual sensor.

[0040] For example, the first environmental information can be image or video stream information collected by the first visual sensor, including but not limited to brightness, contrast, edge feature, color, shape, texture, depth and the like in the image or video stream. The occlusion state includes the presence of an occlusion or the absence of an occlusion. The initial occlusion state refers to the occlusion state of the first visual sensor determined according to the first environmental information detected by the first visual sensor. The first environmental information can be detected by an occlusion detection algorithm, for example, if the brightness of the image or video stream is lower than a preset value or the blurring degree of the image or video stream is lower than a preset standard, it is considered that there is an occlusion object, and then the initial occlusion state of the first visual sensor is determined as the presence of an occlusion. If the brightness of the image or video stream is higher than the preset value or the blurring degree of the image or video stream is lower than the preset standard, the initial occlusion state is determined as the absence of an occlusion.

[0041] In some embodiments, the occlusion detection algorithm includes denoising, filtering, enhancing contrast, color correction, etc. processing on the image or video stream in the first environment information, and then extracting the brightness, contrast, edge, corner, texture, color distribution, etc. that can reflect the brightness, shape, position and color, etc. attribute features of the objects in the field of view of the first sensor from the pre-processed image or video stream. Then based on the extracted attribute features, an occlusion model considering the attributes of the objects in the field of view is established to determine whether the sensor is occluded. The occlusion model is trained using known occluded and non-occluded image or video stream data, and during the training process, the model learns how to determine whether the sensor is occluded based on the extracted features. After training, the accuracy and generalization ability of the model are verified using a validation data set, and the trained occlusion model is deployed to actual applications for real-time detection of whether the first visual sensor is occluded. During real-time detection, the model continuously analyzes the first environment information and determines the initial occlusion state of the first visual sensor based on the extracted features and the occlusion model.

[0042] It can be understood that during the movement of the robot, for example, during navigation, obstacle avoidance, etc., the first environment information detected by the first visual sensor is obtained, and the perspective of the robot and the observed environment information will change accordingly. Dynamic changes help to more accurately determine whether the visual sensor is occluded. If the sensor is occluded, the relative position and angle between the occlusion and the sensor will change as the robot moves, which will cause the observed environment information to change, and more comprehensive first environment information will be obtained. By analyzing the first environment information, it can be determined whether the first visual sensor is occluded to obtain the initial occlusion state. If the robot is in a stationary state, relying solely on a single perspective to obtain the first environment information to determine the initial occlusion state of the first visual sensor may lead to misjudgment, because some static objects or environmental features may produce similar observation effects with the occlusion, leading to misjudgment of the existence of occlusion. However, when the robot is in a moving state, the possibility of misjudgment can be reduced by comparing multiple perspectives and observation results.

[0043] Step 102, when the initial occlusion state of the first visual sensor is occluded, the second environment information detected by the second visual sensor is obtained.

[0044] The following describes step 102 in detail.

[0045] It can be understood that when the first environment information acquired by the first visual sensor represents that the initial occlusion state is that there is occlusion, the second environment information detected by the second visual sensor is acquired, where the second visual sensor is different from the first visual sensor, and thus the second environment information is different from the first environment information. Taking the robot in FIG. 2 as an example, when the first visual sensor is the left visual sensor on the left side of the robot, the second visual sensor can be the front visual sensor and / or the right visual sensor, and at this time, the first environment information is the environment information on the left side of the robot, and the second environment information is the environment information on the front side and / or the right side of the robot. When the first visual sensor is the front visual sensor of the robot, the second visual sensor can be the left visual sensor and / or the right visual sensor, and at this time, the first environment information is the environment information on the front side of the robot, and the second environment information is the environment information on the left side and / or the right side of the robot. When the first visual sensor is the right visual sensor of the robot, the second visual sensor can be the left visual sensor and / or the front visual sensor, and at this time, the first environment information is the environment information on the right side of the robot, and the second environment information is the environment information on the left side and / or the front side of the robot.

[0046] Referring to FIG. 3, in some embodiments, when the second visual sensor is configured with the second light compensation lamp, the following steps 201 to 202 can be included before the second environment information detected by the second visual sensor is acquired.

[0047] Step 201: It is determined whether the second light compensation lamp is in an open state.

[0048] Step 202: If the second light compensation lamp is in the open state, the second light compensation lamp is controlled to be closed.

[0049] The steps 201 to 202 are described in detail as follows.

[0050] For example, the second light compensation lamp is an auxiliary lighting device equipped on the robot, which is used in cooperation with the second visual sensor to provide additional illumination for the second visual sensor, so as to enhance the visual acquisition capability of the second visual sensor in a weak light environment. When the ambient light is insufficient, the second light compensation lamp can actively emit light to illuminate the visual field of the second visual sensor, so as to ensure that the second visual sensor can acquire clear image or video information. The second light compensation lamp can adopt an LED lamp or other high-brightness light source, and the illumination intensity can be flexibly controlled. The second light compensation lamp and the second visual sensor can be integrated on the same module or component to form a closely coordinated visual system. The light direction and illumination range of the second light compensation lamp can be optimized according to specific application requirements. The light compensation range of the second light compensation lamp is located outside the sensing range of the first visual sensor, so as to avoid interference of the light compensation operation of the second light compensation lamp on the first visual sensor.

[0051] In some embodiments, determining whether the second light is in an open state can be achieved by setting a reference image in the system, which is a clear image captured by the second visual sensor when the second light is normally turned on. During the operation of the robot operating system, the second visual sensor will continuously capture new images, and compare these newly captured images with the reference image. If the newly captured images are significantly different from the reference image (e.g., blurred due to insufficient light), it can be determined that the second light is not in an open state; if the newly captured images are similar to the reference image, it can be determined that the second light is in an open state.

[0052] In some embodiments, determining whether the second light is in an open state can be achieved by directly detecting the working state of the second light to determine whether it is turned on, for example, by setting a current or voltage sensor in the circuit of the second light to detect the working current or voltage of the second light. If the detected current or voltage is within the normal working range, it can be determined that the second light is in an open state; if the detected current or voltage is not within the normal working range, it can be determined that the second light is not in an open state.

[0053] It can be understood that if the second light is in an open state, the second light is controlled to be turned off. If the second light is in an open state, it will affect the second environmental information detected by the second visual sensor and affect the judgment of the surrounding environment by the second visual sensor. When the second visual sensor detects sufficient brightness, a signal is sent to the control module of the robot, and the control module then sends a signal to turn off the second light to be in a closed state, and the second environmental information detected by the second visual sensor is obtained in the closed state of the second light.

[0054] The above steps 201 to 202 further consider the influence of the second light configured on the second visual sensor on the occlusion state judgment. Before obtaining the second environmental information detected by the second visual sensor, the system will first determine whether the second light is in an open state, and if the second light is in an open state, the system will control it to be turned off to avoid the interference of the second light on the second environmental information. This method solves the problem that the second light may interfere with the second environmental information when it is in an open state, thereby affecting the accuracy of the occlusion state judgment. By turning off the second light, the interference of the second light on the second environmental information can be reduced, the accuracy and reliability of the occlusion state judgment can be improved, and the autonomous navigation ability and obstacle avoidance performance of the robot can be further improved, achieving more accurate and reliable movement control in complex environments.

[0055] Step 103, determining whether the robot is in a bright light environment according to the second environmental information.

[0056] The step 103 is described in detail as follows.

[0057] It can be understood that the second environment information is image or video stream information collected by the second visual sensor, including but not limited to brightness, contrast, edge feature, color, shape, texture, depth and other features in the image or video stream. By obtaining the brightness information in the second environment information, the brightness of the environment where the robot is located is determined, including a bright environment and a dark environment. When the first visual sensor or / and the second visual sensor can capture sufficient details or image quality that meets the requirements, it can be considered as a bright environment, and when the brightness of the environment is reduced to a certain extent, so that the first visual sensor or / and the second visual sensor cannot capture sufficient details or the image quality is significantly reduced, it can be considered as a dark environment.

[0058] Referring to FIG. 4, in some embodiments, determining whether the robot is in a bright environment according to the second environment information can include the following steps 301 to 303.

[0059] Step 301: obtaining the ambient light brightness according to the second environment information.

[0060] Step 302: if the ambient light brightness is greater than or equal to a preset brightness value, it is determined that the robot is in a bright environment.

[0061] Step 303: if the ambient light brightness is less than the preset brightness value, it is determined that the robot is in a dark environment.

[0062] The steps 301 to 303 are described in detail as follows.

[0063] For example, the ambient light brightness refers to the brightness value of the brightness level of the environment around the robot represented by the second environment information. According to the second environment information, the ambient light brightness can be obtained by first preprocessing the image in the second environment information, such as denoising and enhancing contrast, calculating the average brightness or brightness histogram of the image, and obtaining the ambient light brightness. The ambient light brightness reflects the overall brightness level of the environment where the robot is located, and subsequently by comparing the ambient light brightness with the preset brightness value, it can be determined whether the robot is in a bright environment or a dark environment.

[0064] It can be understood that the preset brightness value is a brightness threshold set to distinguish between bright and dark environments. In related application scenarios, relevant lighting standards or specifications can be obtained, such as lighting standards for office, hospital, and other environments. According to these standards, a suitable brightness value can be selected as the preset brightness value. It can also be tested in the actual environment to record the brightness level in different environments, and an intermediate value that can better distinguish between bright and dark environments can be selected as the preset brightness value. At the same time, adaptive adjustment can be used to allow the system to automatically learn and adjust the preset brightness value according to the environment, dynamically adapting to different environmental changes, such as through machine learning technology, the best preset brightness value can be obtained by analyzing a large amount of environmental data. If the ambient light brightness is greater than or equal to the preset brightness value, it is determined that the robot is in a bright environment, and if the ambient light brightness is less than the preset brightness value, it is determined that the robot is in a dark environment.

[0065] For example, when the first visual sensor is the left visual sensor on the left side of the robot and the initial occlusion state is occlusion, the second environment information can be obtained through the front visual sensor and / or the right visual sensor, and it can be determined whether the robot is in a dark environment or a bright environment according to the second environment information. When the first visual sensor is the right visual sensor and the initial occlusion state is occlusion, the second environment information can be obtained through the left visual sensor and / or the front visual sensor, and the brightness of the environment in which the robot is located can be determined.

[0066] Through steps 301 to 303, the ambient light brightness is obtained according to the second environment information, and it is determined whether the robot is in a bright environment or a dark environment according to the ambient light brightness and the preset brightness value, thereby solving the problem of how to accurately determine the light environment in which the robot is located according to the environment information. In this way, the lighting conditions of the environment in which the robot is located can be accurately determined, and accurate environment information is provided for subsequent occlusion state determination.

[0067] In step 104, if the robot is in a bright environment, the target occlusion state of the first visual sensor is determined to be occlusion.

[0068] Step 104 is described in detail below.

[0069] It can be understood that during the movement of the robot, the initial occlusion state is determined to be occlusion according to the first environment information detected by the first visual sensor, and the robot is determined to be in a bright environment according to the second environment information detected by the second visual sensor, thereby avoiding the initial occlusion state being misjudged as occlusion due to a dark environment, and more accurately determining the target occlusion state of the first visual sensor to be occlusion.

[0070] Through steps 101 to 104, in the process of controlling the robot to move, the first environment information detected by the first vision sensor of the robot is acquired, the initial shielding state of the first vision sensor is determined according to the first environment information, when the initial shielding state of the first vision sensor is that there is shielding, the second environment information detected by the second vision sensor is further acquired, whether the robot is in a bright light environment is determined according to the second environment information, and if the robot is in the bright light environment, the target shielding state of the first vision sensor is determined to be that there is shielding. In combination with the information of the first vision sensor and the second vision sensor, the initial shielding state of the first vision sensor is determined according to the first environment information, and whether the robot is in the bright light environment is further determined in combination with the second environment information, so as to avoid the influence of factors such as the dark light environment to cause misjudgment of the shielding state, the target shielding state is comprehensively analyzed and determined, the problem of how to accurately determine whether the first vision sensor is shielded in the process of moving the robot is solved, the accuracy of the shielding state determination of the vision sensor is improved, and the safety and reliability of the robot are increased.

[0071] Referring to FIG. 5, in some embodiments, if the first vision sensor is configured with the first light supplement lamp, the shielding state determination method can further include the following steps 401 to 403:

[0072] Step 401: If the robot is in a dark light environment, it is determined whether the first light supplement lamp is in a closed state.

[0073] Step 402: If the first light supplement lamp is in a closed state, the first light supplement lamp is controlled to be turned on, and the third environment information detected by the first vision sensor is acquired.

[0074] Step 403: The target shielding state of the first vision sensor is determined according to the third environment information.

[0075] The steps 401 to 403 are described in detail as follows.

[0076] It can be understood that if the first vision sensor is configured with the first light supplement lamp, the first light supplement lamp is a lighting device for providing auxiliary lighting for the first vision sensor, which provides an additional light source for the first vision sensor in the case of insufficient ambient light, so as to ensure that the first vision sensor can acquire clear and reliable image information. The light supplement range of the first light supplement lamp is located outside the sensing range of the second vision sensor, so as to avoid the interference of the light supplement operation of the first light supplement lamp on the second vision sensor.

[0077] In some embodiments, determining whether the first light is in an open state can be achieved by setting a reference image in the system, which is a clear image captured by the first visual sensor when the first light is normally turned on. During the operation of the robot operating system, the first visual sensor will continuously capture new images, and compare these newly captured images with the reference image. If the newly captured images are significantly different from the reference image (e.g. blurred due to insufficient light), it can be determined that the first light is not in an open state; if the newly captured images are similar to the reference image, it can be determined that the first light is in an open state.

[0078] In some embodiments, determining whether the first light is in an open state can be achieved by directly detecting the working state of the first light to determine whether it is turned on or not, for example, by setting a current or voltage sensor in the circuit of the first light to detect the working current or voltage of the first light. If the detected current or voltage is within the normal working range, it can be determined that the first light is in an open state; if the detected current or voltage is not within the normal working range, it can be determined that the first light is not in an open state.

[0079] For example, the third environmental information is the image or video stream information collected by the first visual sensor with the assistance of the illumination of the first light. The illumination of the first light can effectively eliminate the influence of the dark environment on the occlusion detection of the first visual sensor. When the second visual sensor detects that the robot is in a dark environment, the light of the first visual sensor is turned on to eliminate the influence of the dark environment on the occlusion detection of the first visual sensor. The target occlusion state of the first visual sensor is determined according to the third environmental information obtained in the lighted state.

[0080] Through the above steps 401 to 403, in a dark environment, directly using the first visual sensor for occlusion detection may be affected by insufficient environmental light, resulting in inaccurate results. By actively turning on the first light, auxiliary illumination can be provided for the first visual sensor, which can more accurately determine the occlusion state of the first visual sensor in a dark environment, eliminating the influence of insufficient environmental light, thereby obtaining higher quality image information and improving the accuracy and reliability of the occlusion state determination.

[0081] Referring to FIG. 6, in some embodiments, the occlusion state determination method can further include the following steps 501 to 503:

[0082] Step 501: If the robot is in a dark environment, control the robot to rotate to obtain the fourth environmental information detected by the second visual sensor.

[0083] At step 502, it is determined whether the robot is in a bright light environment according to the fourth environment information.

[0084] At step 503, if the robot is in a bright light environment, it is determined that the target occlusion state of the first vision sensor is occluded.

[0085] The steps 501-503 are described in detail as follows.

[0086] In some embodiments, the controlling the robot to rotate can include: controlling the robot to rotate in place or to rotate in an expanded search range so that the second vision sensor rotates towards the direction in which the first vision sensor is located. If the robot is blocked during the rotation, the robot is controlled to rotate in the opposite direction or to retreat in the direction of the block and then rotate again so that the second vision sensor rotates towards the direction in which the first vision sensor is located. When it is detected that the robot is in a dark light environment, the robot can confirm whether the direction in which the first vision sensor is located is also a dark light environment by rotating the second vision sensor, which can improve the accuracy of the determination of the occlusion state of the first vision sensor.

[0087] In some embodiments, if the second vision sensor is provided with a second light supplement lamp, the second light supplement lamp needs to be turned off before the fourth environment information detected by the second vision sensor is acquired, so as to avoid the interference of the second light supplement lamp on the fourth environment information.

[0088] It can be understood that if it is determined based on the second environment information that the robot is in a dark light environment, the target occlusion state of the first vision sensor is not immediately determined, but the robot is controlled to rotate so that the second vision sensor collects picture or video stream information towards the direction in which the first vision sensor is located. During the rotation of the robot, the fourth environment information is acquired again by the second vision sensor, and the brightness of the ambient light obtained according to the fourth environment information is used to determine again whether the robot is in a bright light environment. If the result of this determination shows that the robot is in a bright light environment, it can be determined that the target occlusion state of the first vision sensor is occluded.

[0089] Since in a dark light environment, even if there is no actual occlusion, the vision sensor may be misjudged as being occluded due to insufficient light. Therefore, based on the steps 501-503, by controlling the robot to rotate and making the second vision sensor detect whether the environment detected by the first vision sensor is a dark light environment, it can be determined whether the first vision sensor is occluded by foreign matter, which can avoid the misjudgment of occlusion when the second vision sensor detects that the environment is a dark light environment and the first vision sensor detects that the environment is a bright light environment, and improve the accuracy of the determination of the occlusion state.

[0090] Referring to FIG. 7, in some embodiments, the occlusion state determination method can further include the following steps 601 to 603.

[0091] In step 601, if the robot is in a dark light environment, the robot is controlled to rotate, and the fifth environment information detected by the first vision sensor is obtained.

[0092] In step 602, whether the robot is in a bright light environment is determined according to the fifth environment information.

[0093] In step 603, if the robot is in a bright light environment, it is determined that the target occlusion state of the first vision sensor is occlusion.

[0094] The steps 601 to 603 are described in detail as follows.

[0095] In some embodiments, the robot is controlled to rotate so that the first vision sensor rotates in a direction away from the second vision sensor, which can avoid obtaining the environment information detected by the second vision sensor before, thereby improving the accuracy of the judgment of the first vision sensor.

[0096] It can be understood that when the second vision sensor detects that the robot is in a dark light environment based on the second environment information, the robot is controlled to rotate, and the fifth environment information detected by the first vision sensor at this time is obtained. According to the fifth environment information, it is determined again whether there is a dark light environment. If it is determined that there is a bright light environment, the misjudgment of occlusion caused by the dark light environment can be excluded, and it is determined that the target occlusion state of the first vision sensor is occlusion.

[0097] Through the steps 601 to 603, when the second vision sensor detects a dark light environment, the robot is controlled to rotate, and the fifth environment information detected by the first vision sensor in different directions is obtained. According to the obtained fifth environment information, it is determined again whether it is a dark light environment. If it is not a dark light environment, the target occlusion state of the first vision sensor is determined. This implementation solves the problem that the occlusion state cannot be accurately determined in a dark light environment, so that the robot can accurately determine the occlusion state in a dark light environment, thereby improving the adaptability and intelligence of the robot. In addition, by controlling the rotation of the robot to find a bright light environment, the dependence on a single vision sensor can be reduced, and the robustness of the system can be improved.

[0098] Referring to FIG. 8, in some embodiments, after it is determined that the target occlusion state of the first vision sensor is occlusion, the following steps 701 to 702 can be further included.

[0099] In step 701, whether the first vision sensor is located at the first position of the robot is determined.

[0100] Step 702, if the first visual sensor is located at the first position of the robot, controlling the robot to continue moving.

[0101] The steps 701-702 are described in detail as follows.

[0102] It can be understood that the first position is not a key position on the robot, for example, a position on the side or the back of the robot, and the robot can still move normally if the first visual sensor located at the first position is blocked.

[0103] For example, referring to FIG. 2, the first position refers to a position on the left or right side of the robot, and if the first visual sensor is located at the first position of the robot, that is, the first visual sensor that is determined to be blocked is the left visual sensor or the right visual sensor, since the front visual sensor (i.e., the second visual sensor) has the functions of positioning, obstacle avoidance, and boundary recognition, the robot can still work normally under the condition that the front visual sensor (i.e., the second visual sensor) can work normally by relying on the environmental information detected by the front visual sensor when the first visual sensor cannot work normally. Therefore, the robot can be controlled to continue moving.

[0104] Through the steps 701-702, it is determined whether the visual sensor that is blocked is located on the left or right side of the robot, and if so, the front visual sensor in front can still work and continue to move, which solves the problem that the robot can continue to effectively navigate according to the information of other sensors when the first visual sensor is blocked, avoids the entire system from stopping working due to a single visual sensor being blocked, and improves the working efficiency of the robot and the user experience.

[0105] Referring to FIG. 9, in some embodiments, the method for determining the blocking state can further include the following steps 801-803.

[0106] Step 801, if the first visual sensor is not located at the first position of the robot and the first visual sensor is located at the second position of the robot, it is determined whether the robot can receive a satellite positioning signal.

[0107] Step 802, if the satellite positioning signal can be received, the robot is controlled to move to the charging station.

[0108] Step 803, if the satellite positioning signal cannot be received, the robot is controlled to stop and report an error.

[0109] The steps 801-803 are described in detail as follows.

[0110] It can be understood that the second position is a key position on the robot, for example, a certain position in front of the robot. If the first visual sensor located at the second position is blocked, the robot cannot rely on the environmental information detected by the first visual sensor to move, and other sensors need to be used to control the movement of the robot.

[0111] For example, referring to FIG. 2, the second position refers to a certain position on the front side of the robot. If the front side visual sensor (i.e., the first visual sensor) located on the front side of the robot is blocked, it is determined whether the robot can receive satellite positioning signals. If the robot can normally receive satellite positioning signals, it means that although the front side visual sensor (i.e., the first visual sensor) is blocked, the robot can still navigate through positioning information, so the robot can be controlled to automatically move to the charging station and clean the optical window of the front side visual sensor (i.e., the first visual sensor). If the robot cannot receive satellite positioning signals, it means that the entire system is in a serious positioning failure state. In this case, continuing to move the robot can cause greater safety risks or further damage to the robot. Therefore, the system controls the robot to stop and report an error to remind the user or maintenance personnel to further check and handle.

[0112] Through the above steps 801 to 803, if the first visual sensor is not located at the first position of the robot and is located at the second position of the robot, it is determined whether the robot can receive satellite positioning signals to determine the next action strategy of the robot to cope with the situation that the front side visual sensor is blocked, to ensure the safe operation of the robot as much as possible, and to reduce the risk of robot out of control or damage due to the problem of the first visual sensor, and to improve the overall performance and user experience of the robot.

[0113] In some embodiments, after determining that the target blocking state of the first visual sensor is blocked, the method further comprises: controlling the robot to continue moving, and simultaneously controlling the cleaning assembly to clean the optical window of the first visual sensor during the movement.

[0114] It can be understood that after determining that the first visual sensor is blocked, the robot is controlled to continue moving, and the cleaning assembly is controlled to clean the optical window of the first visual sensor. Even if the first visual sensor is blocked, if other sensors can ensure the normal operation of the robot, the robot can continue to move, and the cleaning assembly can be started to clean the blocked first visual sensor to restore its normal working state as soon as possible. This way can minimize downtime while repairing affected sensors, improving the availability and efficiency of the system.

[0115] In some embodiments, after determining that the target occlusion state of the first vision sensor is occluded, the method further comprises: controlling the robot to stop moving, and controlling the cleaning assembly to clean the optical window of the first vision sensor.

[0116] It can be understood that if it is determined that the first vision sensor is occluded and the movement of the robot cannot be supported by other sensor data, it is necessary to stop for processing. After stopping, the cleaning assembly is started to clean the occluded first vision sensor in order to restore its normal working state as soon as possible. This way is more cautious, although it will cause a short stop, but it can ensure that the robot will not move incorrectly due to insufficient data before the first vision sensor is restored.

[0117] Referring to FIG. 10, in some embodiments, after determining that the target occlusion state of the first vision sensor is occluded, the method can further comprise steps 901 to 902.

[0118] Step 901: Obtain the occlusion ratio of the first vision sensor.

[0119] Step 902: Determine the occlusion detection time of the first vision sensor according to the occlusion ratio.

[0120] The steps 901 to 902 are described in detail below.

[0121] It can be understood that the occlusion ratio refers to the ratio of the area occluded in the vision sensor to the total area. Each frame of image obtained is analyzed to identify the occlusion, and the number of pixels or area of the occlusion in the image is calculated. According to the resolution and image size of the vision sensor, the pixels or area of the occlusion are converted into the proportion of the entire picture, which is the occlusion ratio of the first vision sensor.

[0122] For example, the occlusion detection time refers to the cumulative time of inputting the data of the vision sensor into the occlusion detection algorithm or the time required for occlusion target detection. When the occlusion ratio is different, the occlusion detection time is also different. For example, when the occlusion ratio is 30%, the occlusion detection time is set to 5s, and when the occlusion ratio is 50%, the occlusion detection time is set to 2s.

[0123] Through the above steps 901 to 902, the occlusion ratio of the first vision sensor is obtained, and the occlusion detection time of the first vision sensor is determined according to the occlusion ratio. The occlusion detection time can be flexibly adjusted according to the occlusion ratio, the identification accuracy of small occlusions is improved, and the misidentification problem of small occlusions is avoided.

[0124] Referring to FIG. 11, in some embodiments, determining the occlusion detection time of the first vision sensor according to the occlusion ratio can comprise steps 1001 to 1002:

[0125] Step 1001, if the shielding ratio is greater than or equal to the preset ratio, the shielding detection time of the first visual sensor is determined as the first time.

[0126] Step 1002, if the shielding ratio is less than the preset ratio, the shielding detection time of the first visual sensor is determined as the second time.

[0127] It can be understood that, wherein the first time is less than the second time. If the shielding ratio is greater than or equal to the preset ratio, since the shielding with the shielding ratio greater than or equal to the preset ratio is easy to identify, it is necessary to reduce the detection time of the shielding to improve the efficiency of the shielding detection; if the calculated shielding ratio is less than the preset ratio, since the shielding with the shielding ratio less than the preset ratio is not easy to identify, it is necessary to increase the detection time of the shielding to accumulate more environmental information to detect the shielding and improve the accuracy of the shielding detection.

[0128] For example, if the preset ratio is set to 50%, if the shielding ratio is 30%, the shielding detection time is set to 10s at this time, if the first visual sensor obtains environmental information every 500ms, a total of 20 times of environmental information is obtained, and the shielding is judged based on the 20 times of obtained environmental information; if the shielding ratio is 80%, the shielding detection ratio is set to 2s at this time, if the first visual sensor obtains environmental information every 500ms, a total of 4 times of environmental information is obtained, and the shielding is judged based on the 4 times of obtained environmental information.

[0129] Through the above steps 1001 to 1002, when the shielding ratio is greater than or equal to the preset ratio, since the shielding is easy to identify, the detection time is set to the shorter first time; and when the shielding ratio is less than the preset ratio, since the shielding is not easy to identify, the detection time is set to the longer second time. The shielding detection time of the first visual sensor is dynamically adjusted based on the shielding ratio, which solves the problems of low efficiency when the shielding is easy to identify and insufficient accuracy when the shielding is not easy to identify caused by setting a fixed detection time.

[0130] When the shielding ratio is greater than or equal to the preset ratio, by reducing the shielding detection time, the shielding can be quickly identified, and the detection efficiency is improved. When the shielding ratio is less than the preset ratio, by increasing the shielding detection time, more environmental information can be accumulated to detect the shielding, and the accuracy of the shielding detection is improved.

[0131] Referring to FIG. 12, in some embodiments, after determining that the target shielding state of the first visual sensor is that there is shielding, the method can further include the following steps 1101 to 1102.

[0132] Step 1101, obtain the motion state of the robot.

[0133] At step 1102, the occlusion detection time of the first vision sensor is determined according to the motion state.

[0134] The steps 1101-1102 are described in detail as follows.

[0135] It can be understood that the system can obtain the current motion state of the robot in real time, such as moving, stopping, straight line motion, or curve motion, etc. According to the real-time motion state of the robot, the system can dynamically adjust the occlusion detection time of the first vision sensor. For example, when the robot is in a moving state, the occlusion situation can be detected more frequently, and at this time, the occlusion detection time of the first vision sensor needs to be extended to ensure navigation safety. When the robot is in a stationary state, the detection frequency can be reduced, and at this time, the occlusion detection time of the first vision sensor needs to be shortened to reduce unnecessary system overhead.

[0136] Through the above steps 1101-1102, the occlusion detection strategy is optimized according to the actual running motion state of the robot, the flexibility and efficiency of the system are improved, unnecessary system overhead is reduced as much as possible under the premise of ensuring the safety of the robot, the overall performance is improved, and adaptive adjustment of the occlusion detection time in a complex environment is realized.

[0137] Referring to FIG. 13, in some embodiments, determining the occlusion detection time of the first vision sensor according to the motion state can include the following steps 1201-1202.

[0138] At step 1201, if the motion state is straight line motion, the occlusion detection time of the first vision sensor is determined as a third time.

[0139] At step 1202, if the motion state is curve motion, the occlusion detection time of the first vision sensor is determined as a fourth time.

[0140] The third time is greater than the fourth time.

[0141] The steps 1201-1202 are described in detail as follows.

[0142] Exemplarily, the motion state of the robot is divided into straight line motion and curve motion. When the motion state of the robot is straight line motion, the occlusion detection time (i.e. the third time) is extended to detect the occlusion because the robot obtains relatively single environmental information. When the motion state of the robot is curve motion, the occlusion detection time (i.e. the fourth time) is shortened to detect the occlusion because the robot can obtain more directional environmental information.

[0143] Based on the above steps 1201 to 1202, by dynamically adjusting the occlusion detection time according to the motion state, the detection accuracy can be improved during straight-line motion, ensuring navigation safety. During curved motion, by shortening the occlusion detection time, not only the detection efficiency is improved, but also unnecessary system overhead is reduced, and the overall performance is improved.

[0144] Referring to FIG. 14, in some embodiments, after determining that the target occlusion state of the first visual sensor is occlusion, the method can further include the following steps 1301 to 1302.

[0145] Step 1301, obtaining the working mode of the robot.

[0146] Step 1302, determining the occlusion detection time of the first visual sensor according to the working mode.

[0147] The following describes steps 1301 to 1302 in detail.

[0148] It can be understood that the robot can obtain the current working mode of the robot in real time. When the robot is a mower robot, the working mode can include edge-following mode, mowing mode, transition mode, escape mode, and return-to-charge mode. The edge-following mode refers to the robot navigating and moving along a specific edge or boundary when performing certain tasks. The mowing mode refers to the working mode adopted by the robot when performing lawn mowing and other mowing operations. The transition mode refers to the robot moving from one mowing area to another mowing area. The escape mode is the escape process of the robot in a trapped situation. The return-to-charge mode is the process of the robot returning to the charging station from the current position.

[0149] Through the above steps 1301 to 1302, the working mode of the robot is obtained, and the occlusion detection time of the first visual sensor is determined according to the working mode. Different working modes set different occlusion detection times, which can discover potential safety hazards earlier and take corresponding measures for processing, thereby enhancing the safety of the robot.

[0150] Improve the detection efficiency of robot occlusion detection under different working modes.

[0151] Referring to FIG. 15, in some embodiments, determining the occlusion detection time of the first visual sensor according to the motion state can include the following steps 1401 to 1402.

[0152] Step 1401, if the working mode is the edge-following mode, determining the occlusion detection time of the first visual sensor as the fifth time.

[0153] Step 1402, if the working mode is other working mode, determining the occlusion detection time of the first visual sensor as the sixth time.

[0154] wherein the fifth time is greater than the sixth time.

[0155] The steps 1401-1402 are described in detail as follows.

[0156] It can be understood that if the working mode is the edge-following mode, the occlusion detection time of the first vision sensor is determined as the fifth time, and if the working mode is other working modes such as the mowing mode, the occlusion detection time of the first vision sensor is determined as the sixth time. In the edge-following mode, the robot is required to accurately track and navigate, and the dependence on the vision sensor is greater. The system sets the occlusion detection time of the first vision sensor as the fifth time, that is, increases the detection time, ensures that any possible occlusion problem is discovered and processed in time, and guarantees the accuracy and safety of the edge-following operation.

[0157] For example, for other working modes except the edge-following mode, the environment is relatively stable, and the dependence on the vision navigation is lower. Therefore, the system sets the occlusion detection time of the first vision sensor as the sixth time, that is, a relatively low detection frequency, which can reduce unnecessary system overhead while guaranteeing the basic safety, and improve the overall efficiency.

[0158] Through the above steps 1401-1402, the occlusion detection time in the edge-following mode is set as the fifth time, and the fifth time is greater than the sixth time in other working modes, which ensures the accuracy and efficiency of the occlusion detection in the edge-following mode.

[0159] In some embodiments, referring to FIG. 16, which is a flowchart of a method for determining an occlusion state according to an embodiment of the present application, first, it is determined whether the first vision sensor is occluded based on the environment information detected by the first vision sensor in S1. If the first vision sensor is not occluded, it is determined that the first vision sensor is not occluded in S6. If the first vision sensor is determined to be occluded, it is determined whether the first vision sensor is in a state of being illuminated by the light in S2. If the first vision sensor is in the state of being illuminated by the light, it is determined that the first vision sensor is occluded in S12. If the first vision sensor is determined not to be in the state of being illuminated by the light, it is determined whether the second vision sensor is in the state of being illuminated by the light in S3. If the second vision sensor is not in the state of being illuminated by the light, it is determined whether there is a dark environment based on the environment information detected by the second vision sensor in S5. If there is a dark environment, the robot is controlled to rotate in S10, and the environment information detected by the second vision sensor is obtained in S11. It is determined whether there is a dark environment based on the environment information detected by the second vision sensor in S11. If there is a dark environment, it is determined that the first vision sensor is not occluded, but there is a dark environment in S9. If there is no dark environment, it is determined that the first vision sensor is occluded in S12. If it is determined that there is no dark environment in S5, the robot is controlled to rotate in S7, and the environment information detected by the first vision sensor is obtained in S8. It is determined whether there is a bright environment based on the environment information detected by the first vision sensor in S8. If there is no bright environment, it is determined that the first vision sensor is not occluded in S9. If there is a bright environment, it is determined that the first vision sensor is occluded in S12.

[0160] It can be understood that after it is determined that the first vision sensor is occluded, the robot is controlled to stop moving in S13, the cleaning mechanism is controlled to clean the optical window of the first vision sensor, and then the robot is controlled to move in S14. It is determined whether the occlusion is cleaned away. If the occlusion is cleaned away, the method returns to the beginning. If the occlusion is not cleaned away, the cleaning mechanism is controlled to clean the optical window again for a preset number of times in S15, and then the robot is controlled to move in S16. It is determined whether the occlusion is cleaned away. If the occlusion is cleaned away, the method returns to the beginning. If the occlusion is not cleaned away, the robot is controlled to move to the charging station in S17, and the robot is stopped and an error is reported.

[0161] Referring to FIG. 17, the application further provides a shielding state determination device, which can implement the shielding state determination method. The shielding state determination device comprises: an initial shielding state module, configured to acquire first environment information detected by a first vision sensor of a robot during movement of the robot, and determine an initial shielding state of the first vision sensor according to the first environment information; a second environment information module, configured to acquire second environment information detected by a second vision sensor when the initial shielding state of the first vision sensor is existence of shielding; an environment brightness determination module, configured to determine whether the robot is in a bright environment according to the second environment information; and a shielding determination module, configured to determine that a target shielding state of the first vision sensor is existence of shielding if the robot is in the bright environment.

[0162] The specific implementation of the shielding state determination device is basically the same as the specific embodiments of the shielding state determination method, and will not be repeated here. The shielding state determination device can further comprise other functional modules to implement the shielding state determination method in the above embodiments, as long as the requirements of the embodiments of the application are met.

[0163] The application further provides an electronic device, which comprises a memory and a processor. The memory stores a computer program, and the processor implements the shielding state determination method when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0164] Referring to FIG. 18, FIG. 18 shows the hardware structure of the electronic device according to another embodiment. The electronic device comprises:

[0165] The processor 1801 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the application.

[0166] The memory 1802 can be implemented by a ROM (Read-Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 1802 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the application are implemented by software or firmware, the related program codes are stored in the memory 1802 and are called and executed by the processor 1801 to implement the shielding state determination method of the embodiments of the application.

[0167] The input / output interface 1803 is configured to realize information input and output.

[0168] The communication interface 1804 is configured to realize communication interaction between the device and other devices, and the communication can be realized through a wired manner (for example, a USB, a network cable, and the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, and the like).

[0169] The bus 1805 is configured to transmit information between various components (for example, the processor 1801, the memory 1802, the input / output interface 1803, and the communication interface 1804) of the device.

[0170] The processor 1801, the memory 1802, the input / output interface 1803, and the communication interface 1804 are connected to each other through the bus 1805 to realize communication connection between devices.

[0171] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the above-mentioned occlusion state determination method.

[0172] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0173] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0174] Those skilled in the art can understand that the technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figure, or combine certain steps, or different steps.

[0175] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0176] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware or appropriate combination thereof.

[0177] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover a changeable order, arrangement, grouping, numbering, and / or sequence. Also, the term "comprises" and variations thereof, such as "comprising" and "including", are intended to cover a non-exclusive inclusion, such that any process, method, article, or apparatus that includes a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, the term "coupled" and variations thereof, are intended to cover a direct or indirect coupling or connection.

[0178] It should be understood that, in the present application, "at least one" and "several" refer to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0179] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described system embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0181] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0182] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0183] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method of determining a state of occlusion, characterized by, The method comprises the following steps: In the process of controlling the movement of the robot, first environment information detected by a first vision sensor of the robot is acquired, and an initial shielding state of the first vision sensor is determined according to the first environment information; When the initial shielding state of the first vision sensor is shielding, second environment information detected by a second vision sensor is acquired; According to the second environment information, it is judged whether the robot is in a bright light environment; If the robot is in a bright light environment, it is determined that the target shielding state of the first vision sensor is shielding.

2. The occlusion state determination method according to claim 1, characterized by, The first vision sensor is configured with a first light supplement lamp, and the method further comprises: If the robot is in a dark light environment, it is judged whether the first light supplement lamp is in a closed state; If the first light supplement lamp is in a closed state, the first light supplement lamp is controlled to be opened, and third environment information detected by the first vision sensor is acquired; According to the third environment information, the target shielding state of the first vision sensor is determined.

3. The occlusion state determination method according to claim 1, characterized by, The second vision sensor is configured with a second light supplement lamp, and before the second environment information detected by the second vision sensor is acquired, the method further comprises: It is judged whether the second light supplement lamp is in an open state; If the second light supplement lamp is in an open state, the second light supplement lamp is controlled to be closed.

4. The occlusion state determination method according to claim 2, characterized by, According to the second environment information, it is judged whether the robot is in a bright light environment, which comprises: According to the second environment information, the ambient light brightness is acquired; If the ambient light brightness is greater than or equal to a preset brightness value, it is determined that the robot is in a bright light environment; If the ambient light brightness is less than the preset brightness value, it is determined that the robot is in a dark light environment.

5. The occlusion state determination method according to claim 1, characterized by, The method further comprises: If the robot is in a dark light environment, the robot is controlled to rotate, and fourth environment information detected by the second vision sensor is acquired; According to the fourth environment information, it is judged whether the robot is in a bright light environment; If the robot is in a bright light environment, it is determined that the target shielding state of the first vision sensor is shielding.

6. The occlusion state determination method according to claim 5, characterized by, The control of the robot rotation comprises: The robot is controlled to rotate so that the second vision sensor rotates towards the direction where the first vision sensor is located.

7. The occlusion state determination method according to claim 1, characterized by, The method further comprises: If the robot is in a dark light environment, the robot is controlled to rotate, and fifth environment information detected by the first vision sensor is acquired; According to the fifth environment information, it is judged whether the robot is in a bright light environment; If the robot is in a bright light environment, it is determined that the target shielding state of the first vision sensor is shielding.

8. The occlusion state determination method according to claim 7, characterized by, The method for controlling the robot rotation comprises: The robot is controlled to rotate so that the first vision sensor rotates away from the second vision sensor.

9. The occlusion state determination method according to claim 1, characterized by, After the target shielding state of the first vision sensor is determined to be shielding, the method further comprises: It is judged whether the first vision sensor is located at a first position of the robot; If the first vision sensor is located at the first position of the robot, the robot is controlled to continue moving.

10. The occlusion state determination method according to claim 9, wherein The method further comprises: If the first visual sensor is not located at the first position of the robot and the first visual sensor is located at the second position of the robot, it is determined whether the robot can receive a satellite positioning signal; If the satellite positioning signal can be received, the robot is controlled to move to a charging station; If the satellite positioning signal cannot be received, the robot is controlled to stop and report an error.

11. The occlusion state determination method according to claim 1, characterized by, After determining that the target blocking state of the first visual sensor is blocked, the method further comprises: controlling the robot to continue moving and simultaneously controlling a cleaning assembly to clean the optical window of the first visual sensor during movement.

12. The occlusion state determination method according to claim 1, characterized by, After determining that the target blocking state of the first visual sensor is blocked, the method further comprises: controlling the robot to stop moving and controlling a cleaning assembly to clean the optical window of the first visual sensor.

13. The occlusion state determination method according to claim 1, characterized by, After determining that the target blocking state of the first visual sensor is blocked, the method further comprises: obtaining a blocking ratio of the first visual sensor; determining a blocking detection time of the first visual sensor according to the blocking ratio.

14. The occlusion state determination method according to claim 13, wherein The determination of the blocking detection time of the first visual sensor according to the blocking ratio comprises: if the blocking ratio is greater than or equal to a preset ratio, the blocking detection time of the first visual sensor is determined as a first time; if the blocking ratio is less than the preset ratio, the blocking detection time of the first visual sensor is determined as a second time; wherein the first time is less than the second time.

15. The occlusion state determination method according to claim 1, wherein After determining that the target blocking state of the first visual sensor is blocked, the method further comprises: obtaining a motion state of the robot; determining a blocking detection time of the first visual sensor according to the motion state.

16. The occlusion state determination method according to claim 15, wherein The determination of the blocking detection time of the first visual sensor according to the motion state comprises: if the motion state is linear motion, the blocking detection time of the first visual sensor is determined as a third time; if the motion state is curved motion, the blocking detection time of the first visual sensor is determined as a fourth time; wherein the third time is greater than the fourth time.

17. The occlusion state determination method according to claim 1, wherein After determining that the target blocking state of the first visual sensor is blocked, the method further comprises: obtaining a working mode of the robot; determining a blocking detection time of the first visual sensor according to the working mode.

18. The occlusion state determination method according to claim 17, wherein The determination of the blocking detection time of the first visual sensor according to the working mode comprises: if the working mode is an edge-following mode, the blocking detection time of the first visual sensor is determined as a fifth time; if the working mode is another working mode, the blocking detection time of the first visual sensor is determined as a sixth time; wherein the fifth time is greater than the sixth time.

19. A shield state determining apparatus characterized by comprising: comprises: an initial blocking state module configured to, during movement of the robot, obtain first environment information detected by a first visual sensor of the robot, and determine an initial blocking state of the first visual sensor according to the first environment information; a second environment information module, configured to acquire second environment information detected by a second visual sensor when the initial shielding state of the first visual sensor is that shielding exists; an environment brightness determination module, configured to determine whether the robot is in a bright light environment according to the second environment information; a shielding determination module, configured to determine that the target shielding state of the first visual sensor is that shielding exists if the robot is in the bright light environment.

20. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the shielding state determination method according to any one of claims 1 to 18 when executing the computer program.

Images