A sweeping machine and control method thereof

By integrating binocular vision module and sensor module on the sweeper, combining brightness compensation, texture compensation and deep learning technologies, the sweeper's low recognition rate of solid color obstacles is solved, and more accurate obstacle recognition and cleaning paths are achieved. Planning enhances the navigation capabilities of the sweeper.

CN114903374BActive Publication Date: 2024-11-05BEIJING INDEMIND TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110178121.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-09
Publication Date
2024-11-05
Estimated Expiration
2041-02-09

AI Technical Summary

Technical Problem

The existing sweeper has a low recognition rate of solid-color obstacles and cannot completely replace traditional lidar, resulting in frequent push-pull or stuck in complex environments.

Method used

The binocular vision module is combined with the sensor module, and through brightness compensation, texture compensation and deep learning technology, the surrounding environment images are acquired and the cleaning paths are generated to achieve accurate identification and avoidance of obstacles.

Benefits of technology

It improves the recognition rate of non-transparent target objects, enhances the navigation capability of the sweeper, reduces the missed targets, and achieves more accurate cleaning path planning and obstacle avoidance capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114903374B_ABST
    Figure CN114903374B_ABST
Patent Text Reader

Abstract

The present invention discloses a sweeping machine and a control method thereof. The sweeping machine comprises: a sweeping machine body, a sensor module, a control module, a motion cleaning body, and a binocular vision module. The sensor module and the binocular vision module are placed at the edge of the sweeping machine body. When acquiring an image, the binocular vision module performs brightness compensation and texture compensation on the surrounding environment, and processes the environmental image to obtain target object information. The sensor module acquires distance information between the sweeping machine body and surrounding objects, and motion information of the sweeping machine body. The control module calculates the position information of the sweeping machine body and establishes an environmental map based on the information sent by the sensor module, and generates a cleaning path in combination with the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and controls the motion cleaning body to move and clean along the cleaning path, thereby realizing accurate recognition and timely cleaning of the target object.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of robots, and in particular to a sweeping machine and a control method thereof. Background Art

[0002] Most of the obstacle detection of sweeping machines on the market uses ultrasonic, infrared, mechanical collision switch (industry terminology refers to front gear or front collision) and laser radar, or a combination of the above methods; among them, mechanical collision switch can only detect the presence of obstacles, while the essence of ultrasonic, infrared, and laser radar (TOF ranging) is to calculate the time difference between the active transmission signal and the received echo signal to achieve distance measurement. The obstacle detection achieved by this method has a limited detection range, incomplete target obstacle information, and weak system scalability. Specifically, it can only measure the distance from the two-dimensional working plane to the sweeping machine itself, and cannot measure the three-dimensional size of the obstacle. It is easy to push and pull obstacles, climb obstacles, or get stuck under obstacles. With the development of computer hardware level and the advancement of image processing technology, it has become possible to use binocular cameras on sweeping machines to identify obstacles in the working area, but the recognition rate of pure color obstacles by binocular vision modules is low, and the lack of cooperation with other sensors makes it impossible for binocular modules to completely replace lasers as the navigation and positioning module of sweeping machines, and also hinders the practical application of intelligent recognition technologies such as deep learning. Summary of the invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defect of low recognition rate of pure color obstacles of binocular sweeping robots in the prior art, thereby providing a sweeping robot and a control method thereof.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides a sweeping machine, comprising: a sweeping machine body, a sensor module, a control module, a motion cleaning body, and a binocular vision module, wherein the sensor module and the binocular vision module are placed at the edge of the sweeping machine body, the control module is placed inside the sweeping machine body, the motion cleaning body is installed at the bottom of the sweeping machine body, and the motion cleaning body comprises a motion body and a cleaning body, wherein the binocular vision module has a built-in binocular camera for acquiring an image of the surrounding environment of the sweeping machine body and motion information of the sweeping machine body, and when acquiring the image, brightness compensation and texture compensation are performed on the surrounding environment, and the environmental image is processed to obtain target object information and send it to the control module; the sensor module is used to obtain distance information between the sweeping machine body and surrounding objects, motion information of the sweeping machine body, and send it to the control module; the control module is connected to the motion cleaning body, and is used to calculate the posture information of the sweeping machine body and establish an environmental map according to the information sent by the sensor module, and generate a cleaning path in combination with the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and control the motion cleaning body to move and clean according to the cleaning path.

[0006] In one embodiment, the binocular vision module also includes: an IMU inertial sensor, which is connected to the control module and is used to obtain the acceleration and angular velocity of the sweeping machine body; a brightness compensation sensor, which is used to obtain the current ambient brightness, and based on the current ambient brightness, determine that the sweeping machine body is in a dark environment and perform brightness compensation; a texture compensation sensor, which is connected to the control module and is used to disperse the emitted infrared point light source into light spots or stripes, and produce a texture pattern on the surface of the object; an image processing module, which is connected to the control module and is used to process the surrounding environment image to obtain target object information, and the target object information includes obstacle information and information on the object to be cleaned.

[0007] In one embodiment, the sensor module includes: an optical flow sensor installed at the bottom of the sweeping machine body, used to take two pictures of the surrounding environment at preset time intervals, and calculate the movement speed of the sweeping machine body based on the two pictures of the surrounding environment; an edge TOF sensor installed at the edge of the sweeping machine body, used to obtain the distance between the sweeping machine body and the surrounding obstacles; an ultrasonic sensor installed at the front of the sweeping machine body, used to obtain the distance between the sweeping machine body and the obstacles in front; an odometer installed on the driving motor of the moving body, used to obtain the displacement information of the sweeping machine body; an anti-fall TOF sensor installed at the bottom of the sweeping machine body, used to calculate the distance from the sweeping machine body to the ground; a collision sensor installed at the edge of the sweeping machine body, used to send a collision signal to the control module when the sweeping machine body collides with an obstacle.

[0008] In one embodiment, the sweeping machine also includes a power module, which includes: a power supply unit and a power management unit, wherein the power management unit is placed in the sweeping machine body and is used to obtain the power information of the power supply unit in real time. The control module determines whether the power supply unit is in low power according to the power information. When it is in low power, the moving body is controlled to find a charging pile.

[0009] In one embodiment, the binocular camera is installed with a preset upward tilt degree.

[0010] In a second aspect, an embodiment of the present invention provides a control method for a sweeping machine. Based on the sweeping machine of Example 1, the control method includes: acquiring an image of the surrounding environment of the sweeping machine body and motion information of the sweeping machine body, and when acquiring the image, performing brightness compensation and texture compensation on the surrounding environment, and processing the environmental image to obtain target object information; acquiring distance information between the sweeping machine body and surrounding objects, and motion information of the sweeping machine body; calculating the posture information of the sweeping machine body and establishing an environmental map based on the information sent by the sensor module, and generating a cleaning path by combining the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and controlling the moving cleaning body to move and clean according to the cleaning path.

[0011] In one embodiment, the distance information between the sweeping machine body and the surrounding objects includes: the distance between the sweeping machine body and the surrounding obstacles, the distance between the sweeping machine body and the obstacles in front, and the distance from the sweeping machine body to the ground; the motion information of the sweeping machine body includes: the acceleration and angular velocity of the sweeping machine body, the motion speed of the sweeping machine body, and the displacement information of the sweeping machine body.

[0012] In one embodiment, an image of the surrounding environment of a sweeping machine body and movement information of the sweeping machine body are obtained, and when obtaining the image, brightness compensation and texture compensation are performed on the surrounding environment, and the environmental image is processed to obtain target object information, including: obtaining the current environmental brightness, the acceleration and angular velocity of the binocular camera, and performing brightness compensation when it is determined that the sweeping machine body is in a dark environment based on the current environmental brightness; after a texture pattern is printed on the surface of an object in the surrounding environment, an image of the surrounding environment of the sweeping machine body is obtained; and the environmental image is processed based on a deep learning method to obtain target object information, wherein the target object information includes obstacle information and information on objects to be cleaned.

[0013] In one embodiment, the process of obtaining the distance information between the sweeping machine body and the surrounding objects and the movement information of the sweeping machine body includes: obtaining the distance between the sweeping machine body and the surrounding obstacles, the distance from the sweeping machine to the ground, the distance between the sweeping machine body and the obstacles in front, and the rotation speed of the driving motor of the moving body, and calculating the movement displacement of the sweeping machine body according to the rotation speed of the driving motor and the diameter of the moving body; taking two pictures of the surrounding environment at preset time intervals, and calculating the movement speed of the sweeping machine body based on the two pictures of the surrounding environment.

[0014] In one embodiment, the posture information of the sweeping machine body is calculated based on the information sent by the sensor module and an environmental map is established. In combination with the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, a cleaning path is generated, and the moving cleaning body is controlled to move and clean along the cleaning path, including: obtaining the depth information of the surrounding environment image based on the learning algorithm to obtain the target object information; combining the acceleration and angular velocity of the binocular camera, using a preset mapping method to establish an environmental map, and calculating the posture information of the sweeping machine body and the three-dimensional coordinate information of the target object; intelligently planning the walking cleaning path according to the posture information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and the surrounding objects, and controlling the motion state of the moving cleaning body.

[0015] In one embodiment, the walking cleaning path is intelligently planned based on the position information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and surrounding objects, and the process of controlling the motion state of the moving cleaning body also includes: selecting the optimal path to the target object based on the position information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance between the sweeping machine body and surrounding obstacles; during the walking cleaning process, maintaining or changing the optimal path to the target object in real time based on the distance between the sweeping machine body and surrounding obstacles, the position information of the sweeping machine body, and the movement displacement of the sweeping machine body until the target object is cleaned.

[0016] In one embodiment, the process of intelligently planning a walking and cleaning path and controlling the motion state of the moving cleaning body based on the posture information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and surrounding objects also includes: determining whether the distance from the sweeping machine body to the ground is greater than a preset distance threshold, and when it is greater than the preset distance threshold, controlling the sweeping machine body to stop or retreat.

[0017] The technical solution of the present invention has the following advantages:

[0018] 1. In the sweeping machine provided by the present invention, the binocular vision module performs brightness compensation and texture compensation on the surrounding environment when acquiring images, and processes the environmental image to obtain target object information. The sensor module acquires the distance information between the sweeping machine body and the surrounding objects, and the movement information of the sweeping machine body. The control module calculates the posture information of the sweeping machine body and establishes an environmental map based on the information sent by the sensor module, and generates a cleaning path by combining the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and controls the moving cleaning body to move and clean along the cleaning path, thereby realizing accurate recognition and cleaning of the target object.

[0019] 2. The sweeper provided by the present invention has a binocular vision module integrated with an IMU inertial sensor, which can provide the robot's posture changes at all times; the ultrasonic sensor can identify transparent obstacles, reducing the situation of missing target objects; the texture compensation sensor adds texture to the target object, so that the binocular camera can increase the recognition rate of various non-transparent targets; the brightness compensation sensor allows the target to be better exposed to the binocular camera's field of view; the odometer and optical flow sensor can calculate the speed and displacement of the sweeper body to assist in visual positioning. The binocular camera is tilted up 15° relative to the horizontal plane, reducing the binocular field of view from hitting the ground and increasing the recognition range of the surrounding environment; the collision sensor can achieve timely obstacle avoidance; the anti-fall TOF sensor identifies ground pits or cliffs to prevent falling.

[0020] 3. The control method of the sweeping machine provided by the present invention calculates the posture information of the sweeping machine body and establishes an environmental map based on the environmental image, the acceleration and angular velocity of the binocular camera, and comprehensively considers various factors affecting the cleaning trajectory of the sweeping machine body, thereby intelligently planning the cleaning path, which can achieve accurate cleaning of target objects, timely avoid obstacles and prevent falling.

[0021] 4. The control method of the sweeping machine provided by the present invention increases texture compensation when acquiring images, thereby increasing the recognition rate of various non-transparent targets, and increases brightness compensation to better expose the target in the field of view of the binocular camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] FIG1( a ) is a top view of a sweeping machine provided by an embodiment of the present invention;

[0024] FIG1( b ) is a bottom view of the sweeping machine provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of another specific example of a sweeping machine provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a specific example of a binocular vision module provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of another specific example of a sweeping machine provided by an embodiment of the present invention;

[0028] Figure 5 A flowchart of a specific example of a method for controlling a sweeping machine provided by an embodiment of the present invention;

[0029] Figure 6 A flowchart of another specific example of the control method of the sweeping machine provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0032] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0033] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0034] Example 1

[0035] An embodiment of the present invention provides a sweeping machine, which is used in situations where the target recognition rate of the sweeping machine needs to be improved, as shown in Figure 1(a) and Figure 1(b), including: a sweeping machine body 1, a sensor module 2, a control module 3, a moving cleaning body composed of a moving body 41 and a cleaning body 42, and a binocular vision module 5. The sensor module and the binocular vision module are placed at the edge of the sweeping machine body, the control module is placed inside the sweeping machine body, and the moving cleaning body is installed at the bottom of the sweeping machine body.

[0036] The binocular vision module of the embodiment of the present invention has a built-in binocular camera 51 for acquiring images of the surrounding environment of the sweeping machine body and movement information of the sweeping machine body. When acquiring images, brightness compensation and texture compensation are performed on the surrounding environment, and the environmental image is processed to obtain target object information and send it to the control module. The binocular camera is installed with a preset upward tilt degree (for example, 15°) to reduce the situation where the binocular field of view hits the ground and increase the recognition range of the surrounding environment.

[0037] The binocular vision module of the embodiment of the present invention has a built-in binocular camera, wherein the binocular camera can be any one of a ToF-based depth camera, an RGB binocular depth camera, a structured light depth camera and a binocular structured light depth camera, which is not limited here; in addition, when acquiring images and the sweeper is placed in a dark place, an infrared light source is used to compensate for brightness, and when acquiring images of transparent objects, texture compensation is applied to the surface of the object, thereby making the image clearer and improving the accuracy of the control module in identifying the target object.

[0038] The sensor module of the embodiment of the present invention is used to obtain the distance information between the sweeping machine body and the surrounding objects, and the motion information of the sweeping machine body, and send them to the control module; specifically, the distance information between the sweeping machine body and the surrounding objects includes: the distance between the sweeping machine body and the surrounding obstacles, the distance between the sweeping machine body and the obstacles in front, and the distance from the sweeping machine body to the ground; the motion information of the sweeping machine body includes: the movement speed of the sweeping machine body, and the displacement information of the sweeping machine body.

[0039] The control module of the embodiment of the present invention is connected to the motion cleaning body, and is used to calculate the posture information of the sweeping machine body and establish an environmental map based on the information sent by the sensor module, and combine the information sent by the binocular vision module, the information sent by the sensor module and the environmental map to generate a cleaning path, and control the motion cleaning body to move and clean along the cleaning path.

[0040] The control module of the embodiment of the present invention calculates the posture information of the sweeping robot body and establishes a generated environment map, which is a mature solution method in the prior art. In practical applications, the existing algorithm can be integrated into the control module, and no specific limitation is made here.

[0041] In a specific embodiment, if Figure 2 As shown, the binocular vision module also includes: a brightness compensation sensor 52 and a texture compensation sensor 53.

[0042] The brightness compensation sensor 52 of the embodiment of the present invention is used to obtain the current ambient brightness, and based on the current ambient brightness, determine that the sweeping machine body is in a dark environment and perform brightness compensation; specifically, when the sweeping machine is under a sofa or at night and other times when the light is dark, the brightness compensation sensor works and can provide an 850nm infrared light source. This is only an example and is not intended to be limiting.

[0043] The texture compensation sensor 53 of the embodiment of the present invention is connected to the control module, and is used to disperse the emitted infrared point light source into light spots or stripes, and produce a texture pattern on the surface of the object; specifically, the texture compensation sensor keeps working during use, emitting infrared light, and the texture compensation sensor disperses the infrared point light source into light spots or stripes through a grating, etc., and produces a dot or stripe pattern on the surface of the target object, thereby increasing the recognition rate of the binocular vision module for the target object.

[0044] In a specific embodiment, the binocular vision module further includes: an IMU inertial sensor and an image processing module, wherein the IMU inertial sensor and the image processing module are placed in the binocular vision module.

[0045] The IMU inertial sensor of the embodiment of the present invention is connected to the control module and is used to obtain the acceleration and angular velocity of the sweeping machine body. The control module can obtain the position and posture of the sweeping machine body based on the acceleration and angular velocity, as well as the environmental image.

[0046] The image processing module of the embodiment of the present invention is connected to the control module, and is used to process the surrounding environment image to obtain target object information, where the target object information includes obstacle information and information of the object to be cleaned. The image processing method involved in the image processing module can be an algorithm based on deep learning.

[0047] In a specific embodiment, if Figure 2 As shown, the sensor module 2 includes: an optical flow sensor 21 , an edge TOF sensor 22 , an ultrasonic sensor 23 , an odometer 24 , an anti-fall TOF sensor 25 , and a collision sensor 26 .

[0048] like Figure 2As shown, the optical flow sensor 21 of the embodiment of the present invention is installed at the bottom of the sweeping machine body, and is used to take two surrounding environment pictures at preset time intervals, and calculate the movement speed of the sweeping machine body based on the two surrounding environment pictures.

[0049] like Figure 2 As shown, the edge TOF sensor 22 of the embodiment of the present invention is installed on the edge of the sweeping machine body, and is used to obtain the distance between the sweeping machine body and the surrounding obstacles; specifically, the edge TOF sensor calculates the distance from the sweeping machine to the obstacle by using a laser ranging method, and transmits the distance information to the control module in real time. It is mostly used for cleaning along walls, maintaining a certain distance from the wall, usually 5-20mm.

[0050] like Figure 2 As shown, the ultrasonic sensor 23 of the embodiment of the present invention is installed at the front of the sweeping machine body to obtain the distance between the sweeping machine body and the obstacle in front; the specific structure of the binocular vision module of the embodiment of the present invention is as shown Figure 3 As shown, the edge of the ultrasonic sensor is sealed with a rubber sleeve 56, thereby ensuring the accuracy of the ultrasonic sensor's distance measurement. Figure 3 In the embodiment of the present invention, the binocular vision module further includes a mechanical interface 54 and an electrical interface USB 55, wherein the mechanical interface 54 is used to connect to the sweeping machine body, and the electrical interface USB 55 is used to electrically connect to the sweeping machine body.

[0051] like Figure 2 As shown, the odometer 24 of the embodiment of the present invention is installed on the driving motor of the moving body, and is used to obtain the displacement information of the sweeping machine body; specifically, the odometer is an odometer in the prior art, which will not be described in detail here.

[0052] like Figure 2 As shown, the anti-fall TOF sensor 25 of the embodiment of the present invention is installed at the bottom of the sweeping machine body, and is used to calculate the distance from the sweeping machine body to the ground; specifically, the anti-fall TOF sensor radiates and receives infrared signals downward, calculates the distance from the sweeping machine body to the target on the ground, and feeds back to the control module. When the control module detects that the distance is greater than a certain limit value, the control module controls the walking body to stop or reverse.

[0053] like Figure 2 As shown, the collision sensor 26 of the embodiment of the present invention is installed on the edge of the sweeping machine body, and is used to send a collision signal to the control module when the sweeping machine body collides with an obstacle; specifically, a buffer is installed on the outside of the collision sensor, and when the buffer collides with the obstacle, the buffer is squeezed and deformed, so that the obstacle squeezes the collision sensor, and the collision sensor sends a collision signal. After receiving the collision signal, the control module controls the walking body to slow down or stop.

[0054] In a specific embodiment, if Figure 4As shown, the sweeping machine also includes a power module 6, which includes: a power supply unit and a power management unit, wherein the power management unit is placed in the sweeping machine body and is used to obtain the power information of the power supply unit in real time. The control module determines whether the power supply unit is in low power according to the power information. When it is in low power, the moving body is controlled to find a charging pile.

[0055] Specifically, solving the position and posture information of the sweeping machine body may include: acquiring an environmental image, which is an RGB image in the XZ-axis plane that is at an angle of 30-90 degrees to the X-axis, where X and Z are coordinate axes of three-dimensional Cartesian coordinates, the Z-axis represents the vertical upward coordinate, and the X-axis represents the direction in which the sweeping machine body moves; extracting and tracking the visual local feature information of each frame of the image; collecting the information obtained by the IMU inertial sensor and the information obtained by the odometer between the two frames of images, and using the information obtained by the IMU inertial sensor and the pre-integration method to calculate the relative position and relative posture of the sweeping machine body between the two frames of images corresponding to the inertial navigation unit information, and obtaining the pre-integration result; calculating the current three-dimensional position and posture of the sweeping machine body based on a nonlinear optimization method, as well as the three-dimensional position of the map point corresponding to the tracked visual local feature information, and screening the key frame images, and fusing all the visual local information through the nonlinear least squares method. The local feature information, pre-integration results and odometer information are used to obtain the current three-dimensional position and posture of the sweeping robot body in the current local map, as well as the optimal solution of the three-dimensional position of the map point corresponding to the visual local feature information on tracking, so as to obtain the current positioning information of the sweeping robot body and the sweeping robot body positioning information corresponding to the key frame image; a pose graph of the key frame is constructed according to the sweeping robot body positioning information corresponding to the key frame image, the visual closed loop in the key frame image set is detected by the visual feature bag method, and the position and posture of the sweeping robot body corresponding to the key frame image is globally optimized to eliminate the accumulated error in the positioning of the sweeping robot body during long-distance operation, so as to obtain the pose of the sweeping robot body, and a three-dimensional submap of the environmental space is constructed based on the determined pose information of the sweeping robot body at the current position and the acquired depth information of the sweeping robot at the current position, and the above method is repeated to obtain the final environmental map.

[0056] The embodiment of the present invention comprehensively analyzes the distance between the sweeper body and the surrounding objects, and obtains the movement speed and movement displacement of the sweeper body in real time, so that the control module can integrate various influencing factors of the environment in which the sweeper is located, intelligently plan the cleaning path, and quickly avoid obstacles and prevent falling. The sweeper provided by the embodiment of the present invention has a binocular vision module integrated with an IMU inertial sensor, which can provide the robot's posture changes at all times; the ultrasonic sensor can identify transparent obstacles, reducing the situation of missing target objects; the texture compensation sensor adds texture to the target object, so that the binocular camera increases the recognition rate of various non-transparent targets; the brightness compensation sensor makes the target better exposed in the binocular camera field of view; the odometer and optical flow sensor can calculate the speed and displacement of the sweeper body to assist visual positioning. The binocular camera is tilted up 15° relative to the horizontal plane, reducing the binocular field of view from hitting the ground and increasing the recognition range of the surrounding environment; the collision sensor can achieve timely obstacle avoidance; the anti-fall TOF sensor identifies ground pits or cliffs to prevent falling.

[0057] Example 2

[0058] The embodiment of the present invention provides a control method for a sweeping machine, based on the sweeping machine of embodiment 1, such as Figure 5 As shown, the control method includes:

[0059] Step S11: Obtain the image of the surrounding environment of the sweeping machine body and the movement information of the sweeping machine body. When acquiring the image, perform brightness compensation and texture compensation on the surrounding environment, and process the environmental image to obtain target object information; obtain the distance information between the sweeping machine body and the surrounding objects, and the movement information of the sweeping machine body.

[0060] Step S12: Calculate the posture information of the sweeping machine body and establish an environmental map based on the information sent by the sensor module, and generate a cleaning path by combining the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and control the moving cleaning body to move and clean along the cleaning path.

[0061] The control module of the embodiment of the present invention calculates the posture information of the sweeping machine body and establishes an environmental map based on the environmental image obtained by the binocular vision module, the acceleration and angular velocity of the sweeping machine body, and comprehensively considers various factors that affect the cleaning trajectory of the sweeping machine body, such as: light, transparent objects, the distance between the sweeping machine body and surrounding obstacles, the distance between the sweeping machine body and obstacles in front, the distance from the sweeping machine body to the ground, the movement speed of the sweeping machine body, the displacement information of the sweeping machine body, etc., so as to intelligently plan the cleaning path, accurately clean the target object, avoid obstacles in time and prevent falling.

[0062] It should be noted that the solution algorithm involved in the control module and the image processing method involved in the binocular vision module are mature algorithms in the prior art, and specific examples have been described in Example 1 and will not be repeated here.

[0063] In a specific embodiment, if Figure 6 As shown, the process of obtaining the image of the surrounding environment of the sweeping machine body and the movement information of the sweeping machine body, performing brightness compensation and texture compensation on the surrounding environment when obtaining the image, and processing the environment image to obtain the target object information includes:

[0064] Step S21: Obtain the current ambient brightness, the acceleration and angular velocity of the binocular camera, and perform brightness compensation when it is determined based on the current ambient brightness that the sweeping robot body is in a dark environment.

[0065] Step S22: After the texture pattern is printed on the surface of the object in the surrounding environment, an image of the surrounding environment of the sweeping robot body is obtained.

[0066] Step S23: Process the environment image based on a deep learning method to obtain target object information, where the target object information includes obstacle information and information about objects to be cleaned.

[0067] The texture compensation sensor of the embodiment of the present invention is in a state of being constantly working. When the binocular camera acquires an environmental image, the texture compensation sensor emits infrared light. The texture compensation sensor disperses the infrared point light source into light spots or stripes through a grating, etc., and produces a dot or stripe pattern on the surface of the target object, thereby increasing the recognition rate of the binocular vision module for the target object.

[0068] In a specific embodiment, the process of obtaining the distance information between the sweeping machine body and surrounding objects and the movement information of the sweeping machine body includes:

[0069] Step S31: Obtain the distance between the sweeper body and surrounding obstacles, the distance between the sweeper and the ground, the distance between the sweeper body and obstacles in front, and the rotation speed of the driving motor of the moving body, and calculate the moving displacement of the sweeper body based on the rotation speed of the driving motor and the diameter of the moving body.

[0070] Step S32: taking two pictures of the surrounding environment at a preset time interval, and calculating the movement speed of the sweeping robot body based on the two pictures of the surrounding environment.

[0071] The embodiment of the present invention adopts an optical flow sensor to take two pictures of the surrounding environment at preset time intervals, and calculates the movement speed of the sweeping machine body based on the two pictures of the surrounding environment; the edge TOF sensor calculates the distance from the sweeping machine to the obstacle by using a laser ranging method, and transmits the distance information to the control module in real time; the ultrasonic sensor obtains the distance between the sweeping machine body and the obstacle in front; the odometer obtains the displacement information of the sweeping machine body; the anti-fall TOF sensor 25 calculates the distance from the sweeping machine body to the ground.

[0072] In a specific embodiment, the posture information of the sweeping machine body is calculated according to the information sent by the sensor module and the environment map is established, and the cleaning path is generated by combining the information sent by the binocular vision module, the information sent by the sensor module and the environment map, and the moving cleaning body is controlled to move and clean according to the cleaning path, including:

[0073] Step S41: Obtaining depth information of the surrounding environment image based on the deep learning algorithm to obtain target object information.

[0074] Step S42: combining the acceleration and angular velocity of the sweeping robot body, using a preset mapping method, to establish a generated environment map, and to calculate the position information of the sweeping robot body and the three-dimensional coordinate information of the target object.

[0075] Step S43: According to the position information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and the surrounding objects, the walking and cleaning path is intelligently planned to control the motion state of the moving cleaning body.

[0076] The image processing module of the embodiment of the present invention uses a deep learning algorithm to perform deep learning on the acquired surrounding environment image and identify the target object information. The image recognition method of the image processing module is not limited to the deep learning method, but can also include other recognition methods, such as: image comparison method, etc.

[0077] In a specific embodiment, the process of intelligently planning a walking cleaning path and controlling the motion state of the moving cleaning body according to the posture information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and the surrounding objects also includes:

[0078] Step S51: Selecting the optimal path to the target object according to the position information of the sweeping robot body, the three-dimensional coordinate information of the target object, and the distance between the sweeping robot body and surrounding obstacles.

[0079] Step S52: During walking cleaning, the optimal path toward the target object is maintained or changed in real time according to the distance between the sweeping machine body and surrounding obstacles, the position information of the sweeping machine body, and the movement displacement of the sweeping machine body until the target object is cleaned.

[0080] The control module of the embodiment of the present invention selects the optimal path according to the posture information of the sweeping machine body, the posture information of the sweeping machine body, the three-dimensional coordinate information of the target object, the distance between the sweeping machine body and the surrounding obstacles, and the environmental map, until the target object is cleaned. Specifically, when the distance between the sweeping machine body and the surrounding obstacles is less than a certain threshold, the obstacles are avoided in time.

[0081] In a specific embodiment, the control module also determines whether the distance from the sweeping machine body to the ground is greater than a preset distance threshold. When it is greater than the preset distance threshold, the sweeping machine body is controlled to stop or retreat to prevent falling.

[0082] The control method of the sweeping machine provided by the embodiment of the present invention calculates the posture information of the sweeping machine body and establishes an environmental map based on the environmental image, the acceleration and angular velocity of the binocular camera, and comprehensively considers various factors affecting the cleaning trajectory of the sweeping machine body, thereby intelligently planning the cleaning path, and can achieve accurate cleaning of target objects, timely avoid obstacles and prevent falls; when acquiring images, texture compensation is added to increase the recognition rate of various non-transparent targets, and brightness compensation is added to better expose the target in the field of view of the binocular camera.

[0083] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the scope of protection of the invention.

Claims

1. A sweeping machine, characterized in that: include: The sweeping machine body, sensor module, control module, motion cleaning body, binocular vision module, the sensor module and binocular vision module are placed at the edge of the sweeping machine body, the control module is placed inside the sweeping machine body, the motion cleaning body is installed at the bottom of the sweeping machine body, the motion cleaning body includes a motion body and a cleaning body, wherein, The binocular vision module has a built-in binocular camera, which is used to obtain the image of the surrounding environment of the sweeper and the movement information of the sweeper. When acquiring the image, it performs brightness compensation and texture compensation on the surrounding environment, processes the environmental image, obtains the target object information and sends it to the control module; The sensor module is used to obtain the distance information between the sweeper body and the surrounding objects, the movement information of the sweeper body, and send it to the control module; The control module is connected to the motion cleaning body, and is used to calculate the posture information of the sweeping machine body and establish an environmental map according to the information sent by the sensor module, and to generate a cleaning path by combining the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and to control the motion cleaning body to move and clean according to the cleaning path; The binocular vision module also includes: An IMU inertial sensor, which is connected to the control module and is used to obtain the acceleration and angular velocity of the sweeper body; A brightness compensation sensor is used to obtain the current ambient brightness and perform brightness compensation when it is determined that the sweeping machine body is in a dark environment based on the current ambient brightness; A texture compensation sensor connected to the control module is used to disperse the emitted infrared point light source into light spots or stripes to produce a texture pattern on the surface of the object; An image processing module, connected to the control module, for processing the surrounding environment image to obtain target object information, wherein the target object information includes obstacle information and information of an object to be cleaned; The sensor module comprises: An optical flow sensor is installed at the bottom of the sweeper body, and is used to take two pictures of the surrounding environment at a preset time interval, and calculate the movement speed of the sweeper body based on the two pictures of the surrounding environment; The edge TOF sensor is installed on the edge of the sweeper body to obtain the distance between the sweeper body and surrounding obstacles; Ultrasonic sensor, installed at the front of the sweeper body, used to obtain the distance between the sweeper body and the obstacle in front; An odometer, mounted on the driving motor of the moving body, for obtaining displacement information of the sweeping machine body; An anti-fall TOF sensor is installed at the bottom of the sweeper body and is used to calculate the distance from the sweeper body to the ground; The collision sensor is installed on the edge of the sweeping machine body and is used to send a collision signal to the control module when the sweeping machine body collides with an obstacle.

2. The sweeping machine according to claim 1, characterized in that: Also includes: The power module includes: a power supply unit and a power management unit, wherein the power management unit is placed in the body of the sweeper and is used to obtain the power information of the power supply unit in real time. The control module determines whether the power supply unit is in low power according to the power information. When it is in low power, the control module controls the moving body to find a charging pile.

3. The sweeping machine according to claim 1, characterized in that: The binocular camera is installed with a preset upward tilt degree.

4. A method for controlling a sweeping machine, characterized in that: Based on the sweeping machine according to any one of claims 1 to 3, the control method comprises: Acquire the surrounding environment image of the sweeping machine body and the movement information of the sweeping machine body, perform brightness compensation and texture compensation on the surrounding environment when acquiring the image, and process the environment image to obtain the target object information; obtain the distance information between the sweeping machine body and the surrounding objects, and the movement information of the sweeping machine body; The position information of the sweeper body is calculated based on the information sent by the sensor module and the environmental map is established. The cleaning path is generated by combining the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and the moving cleaning body is controlled to move and clean according to the cleaning path.

5. The control method of the sweeping machine according to claim 4, characterized in that: The distance information between the sweeping machine body and surrounding objects includes: the distance between the sweeping machine body and surrounding obstacles, the distance between the sweeping machine body and obstacles in front, and the distance between the sweeping machine body and the ground; The motion information of the sweeping machine body includes: the acceleration and angular velocity of the sweeping machine body, the motion speed of the sweeping machine body, and the displacement information of the sweeping machine body.

6. The control method of the sweeping machine according to claim 5, characterized in that: The process of acquiring the image of the surrounding environment of the sweeping machine body and the movement information of the sweeping machine body, performing brightness compensation and texture compensation on the surrounding environment when acquiring the image, and processing the environment image to obtain the target object information includes: Acquire the current ambient brightness, the acceleration and angular velocity of the binocular camera, and perform brightness compensation when it is determined that the sweeping robot body is in a dark environment based on the current ambient brightness; After the texture pattern is printed on the surface of the object in the surrounding environment, an image of the surrounding environment of the sweeper body is obtained; The environment image is processed based on a deep learning method to obtain target object information, where the target object information includes obstacle information and information about objects to be cleaned.

7. The control method of the sweeping machine according to claim 5, characterized in that: The process of obtaining the distance information between the sweeping machine body and surrounding objects and the movement information of the sweeping machine body includes: Obtain the distance between the sweeping machine body and surrounding obstacles, the distance between the sweeping machine and the ground, the distance between the sweeping machine body and obstacles in front, and the rotation speed of the driving motor of the moving body, and calculate the movement displacement of the sweeping machine body according to the rotation speed of the driving motor and the diameter of the moving body; Two surrounding environment pictures are taken at preset time intervals, and the movement speed of the sweeping machine body is calculated based on the two surrounding environment pictures.

8. The control method of the sweeping machine according to claim 5, characterized in that: The process of calculating the posture information of the sweeping machine body according to the information sent by the sensor module and establishing an environmental map, and generating a cleaning path by combining the information sent by the binocular vision module, the information sent by the sensor module and the environmental map, and controlling the moving cleaning body to move and clean along the cleaning path includes: Based on the deep learning algorithm, the depth information of the surrounding environment image is obtained from the surrounding environment image, and the target object information is obtained; in combination with the acceleration and angular velocity of the sweeping robot body, a preset mapping method is used to establish a generated environment map, and the posture information of the sweeping robot body and the three-dimensional coordinate information of the target object are solved; According to the posture information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and the surrounding objects, the walking and cleaning path is intelligently planned to control the motion state of the moving cleaning body.

9. The control method of the sweeping machine according to claim 8, characterized in that: According to the position information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and the surrounding objects, the process of intelligently planning the walking and cleaning path and controlling the motion state of the moving cleaning body also includes: According to the position information of the sweeper body, the three-dimensional coordinate information of the target object, and the distance between the sweeper body and surrounding obstacles, the optimal path to the target object is selected; During the walking cleaning process, the optimal path toward the target object is maintained or changed in real time according to the distance between the sweeper body and the surrounding obstacles, the position information of the sweeper body, and the movement displacement of the sweeper body until the target object is cleaned.

10. The control method of the sweeping machine according to claim 8, characterized in that: According to the position information of the sweeping machine body, the three-dimensional coordinate information of the target object, and the distance information between the sweeping machine and the surrounding objects, the process of intelligently planning the walking and cleaning path and controlling the motion state of the moving cleaning body also includes: Determine whether the distance from the sweeping machine body to the ground is greater than a preset distance threshold, and when it is greater than the preset distance threshold, control the sweeping machine body to stop or retreat.