TOF camera and sweeping robot

Abstract

A kind of TOF camera, characterized in that, including flood projector, first TOF receiver and processor;The flood projector is used to project floodlight to target area;The first TOF receiver is used to receive reflected signal, and generate first infrared map;The processor is used to obtain first depth according to the phase difference of the floodlight and the reflected signal, obtain the first area of depth mutation value in first range, and utilize first model to identify the first area on the first infrared map, judge whether it is small object.The present application simultaneously obtains first depth value and first infrared map using first TOF receiver, and identifies first area in first depth using mutation value, and then identifies first area on first infrared map, to judge whether there is small object, without increasing additional structure.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of depth camera, in particular, to a TOF camera and a sweeping robot. BACKGROUND

[0002] With the development of smart home, sweeping robots have been more and more applied to family life. At present, the cameras applied to sweeping robots include TOF cameras, structured light cameras, and RGB cameras. Various cameras are suitable for different scenes. Among them, the TOF camera is mainly an iTOF camera, which can measure a large area in front of the sweeping robot.

[0003] Due to the characteristics of TOF technology, the iTOF camera is easily affected by multipath effect, which can easily cause distortion at corners and the like. For a larger area of distortion, it can be easily judged, but for a small object, it is difficult to judge whether it is a small object or the surface characteristics of the object. Some small objects may be needles, nails and other solid objects, which can easily damage the sweeping robot. Therefore, some robots need to detour or use corresponding processing mode for processing. However, there is no effective solution for small objects in the prior art.

[0004] The disclosure of the above background art content is only used to assist in understanding the inventive concept and technical solutions of the present application, and it does not necessarily belong to the prior art of the present patent application. In the absence of explicit evidence that the above content has been disclosed on the filing date of the present patent application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. SUMMARY

[0005] Therefore, the present application uses a first TOF receiver to obtain a first depth value and a first infrared image at the same time, identifies a first region in the first depth by using a mutation value, and identifies the first region on the first infrared image to determine whether there is a small object without increasing additional structures.

[0006] In a first aspect, the present application provides a TOF camera, characterized in that it comprises a floodlight projector, a first TOF receiver and a processor.

[0007] The floodlight projector is used to project a floodlight to a target area.

[0008] The first TOF receiver is used to receive a reflected signal and generate a first infrared image.

[0009] The processor is used to obtain a first depth according to the phase difference between the floodlight and the reflected signal, obtain a first region in a first range of depth mutation value, and identify the first region on the first infrared image by using a first model to determine whether it is a small object.

[0010] Optionally, the TOF camera further comprises a second TOF receiver for receiving the reflected signal and generating a second infrared image; and the floodlight projector is located at a midpoint between the second TOF receiver and the first TOF receiver.

[0011] Optionally, the TOF camera has the same parameters for the first TOF receiver and the second TOF receiver.

[0012] Optionally, the processor identifies the first region on the second infrared image using a first model and obtains a depth value of the first region by calculating a parallax of the first region on the first infrared image and the second infrared image, as a final depth value of the first region.

[0013] Optionally, the first model is obtained by machine learning training.

[0014] In a second aspect, the present application provides a TOF camera, comprising a floodlight projector, a first TOF receiver and a processor.

[0015] The floodlight projector is configured to project a floodlight to a target region.

[0016] The first TOF receiver is configured to receive a reflected signal and generate a first infrared image.

[0017] The processor is configured to obtain a first depth value according to a phase difference between the floodlight and the reflected signal, identify the first infrared image using a first model, obtain a feature of a small object, and take a corresponding first depth value as a depth value of the small object.

[0018] Optionally, the small object refers to an object with an area between 10 pixels and 300 pixels on the infrared image.

[0019] Optionally, when a small object is identified on the first infrared image, the power of the floodlight projector is increased.

[0020] In a third aspect, the present application provides a sweeping robot, comprising the TOF camera according to any one of the above aspects.

[0021] Optionally, the TOF camera has a viewing angle including a region 0.1 cm away from the front of the sweeping robot.

[0022] Compared with the prior art, the present application has the following beneficial effects:

[0023] The present application generates an infrared image and a first depth simultaneously by using a TOF receiver, has natural consistency, and does not need to be aligned, so that conversion between the infrared image and the first depth can be directly performed.

[0024] The present application uses the multipath effect of a TOF, filters a depth mutation value, obtains a region that may be a small object, and then performs accurate identification by using a first infrared image, so that the amount of data to be processed by the first infrared image is reduced, the amount of calculation is reduced, and the efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings. Other features, objects and advantages of the present application will become more apparent through reading the following detailed description of the non-limiting embodiments with reference to the following drawings:

[0026] Figure 1 FIG. 1 is a structural schematic diagram of a TOF camera according to an embodiment of the present application;

[0027] Figure 2 FIG. 2 is a TOF signal schematic diagram according to an embodiment of the present application;

[0028] Figure 3 FIG. 3 is another structural schematic diagram of a TOF camera according to an embodiment of the present application;

[0029] Figure 4 FIG. 4 is a structural schematic diagram of another TOF camera according to an embodiment of the present application;

[0030] Figure 5 FIG. 5 is a structural schematic diagram of a sweeping robot according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The present application will be described in detail below in combination with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0032] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-mentioned drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so construed can be interchanged, under appropriate circumstances, to describe the embodiments of the present application described herein, for example, can be practiced in other than the illustrated order if it is assumed from any embodiment that any intervening steps are inherently relatively insignificant whether or not same are shown in the drawings or recited in the description. Moreover, the terms "comprise" and "have", and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that comprises a list of steps or units is not necessarily limited to those steps or units which are clearly recited, but can include other steps or units that are not expressly listed or inherent to such process, method, product or apparatus.

[0033] The TOF camera and system provided by the embodiments of the present application are aimed at solving the problems in the prior art.

[0034] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0035] The TOF camera provided by the embodiments of the present application obtains the first depth value and the first infrared image simultaneously by using the first TOF receiver, identifies the first region in the first depth by using the abrupt value, and then identifies the first region on the first infrared image to determine whether there is a small object, without the need to increase additional structures.

[0036] Figure 1 The structure diagram of a TOF camera in the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the TOF camera in the embodiments of the present application comprises: Figure 1

[0037] The floodlight projector 1 is used to project a floodlight to a target region.

[0038] Specifically, the floodlight projector 1 can uniformly project a floodlight to the target region. The floodlight projected by the floodlight projector 1 can be modulated into a sine wave signal or a pulse signal.

[0039] The first TOF receiver 2 is used to receive a reflected signal and generate a first infrared image.

[0040] ​Specifically, the first TOF receiver 2 operates synchronously with the floodlight projector 1 to receive the reflected signal of the floodlight projected by the floodlight projector 1. After receiving the reflected signal, the first TOF receiver 2 can directly generate a first infrared image based on the reflected signal. The first infrared image includes all targets within the field of view. The first TOF receiver 2 performs exposures at fixed time intervals. The signal strength received by the first TOF receiver 2 determines the intensity of the first infrared image. When the power of the floodlight projector 1 is high, the light signal received by the first TOF receiver 2 is strong. When the power of the floodlight projector 1 is low, the signal received by the first TOF receiver 2 is weak.

[0041] Processor 3 is configured to obtain a first depth based on the phase difference between the floodlight and the reflected signal, acquire a first region with a depth abrupt change value in a first range, and use a first model to identify the first region on the first infrared image to determine whether it is a small object.

[0042] Specifically, such as Figure 2 As shown, the closer the area is to the first TOF receiver 2, the stronger the received reflected signal. This is mainly because the floodlight projector 1 is relatively close to the first TOF receiver 2, and the illumination range of the emitted floodlight is small, resulting in a larger illumination intensity per unit area, and therefore a stronger signal. Figure 2 The data shown represents point cloud data for a flat surface, not for an uneven surface. Because the data is not a complete plane, it is usually necessary to fit a plane to determine its location. For this reason, small objects are often treated as errors during plane fitting and thus cannot be identified.

[0043] Processor 3 does not need to fit the plane; it only needs to calculate the difference in depth data at adjacent locations, i.e., the depth abrupt change value, to make a judgment. When the depth abrupt change value at a certain location is within the first range, the area consisting of that location and several surrounding locations is marked as the first region. The first region is a region shared by the first infrared image and the depth image. If only one location in several adjacent regions has a depth abrupt change value within the first range, that point is determined to be an error point and is not marked. The first range is determined by the size of the small object to be identified in this embodiment. When the height of the small object to be identified in this embodiment is between 0.1cm and 2cm, the first range is 0.15cm to 3cm, i.e., the first range is 1.5n, where n is the size of the target object to be identified. Setting the first range to 1.5 times the size of the target object to be identified can effectively identify the target object while filtering out interference from other signals to the greatest extent, ensuring accuracy.

[0044] The first model is obtained by machine learning. The training sample uses a large number of infrared images containing small objects as a training set, wherein at least part of the infrared images are taken at the same shooting angle as the TOF camera. The processor 3 only uses the first model to identify the first region, thereby identifying the small object with the smallest amount of data. The small object refers to an object with an area of between 10 pixels and 300 pixels on the infrared image.

[0045] Figure 3 The structure of another TOF camera in an embodiment of the present application is shown in the figure. As shown in the figure, unlike the previous embodiment, the TOF camera in the embodiment of the present application further comprises: Figure 3

[0046] The second TOF receiver 4 is used to receive the reflected signal and generate a second infrared image; the floodlight projector is located at the midpoint of the first TOF receiver and the second TOF receiver.

[0047] Specifically, the second TOF receiver 4 works synchronously with the first TOF receiver 2 to obtain the same reflected signal of the floodlight. The processor 3 controls the operation of the floodlight projector 1, the first TOF receiver 2 and the second TOF receiver 4. The second infrared image is generated at the same time as the first infrared image. Since the floodlight projector 1 is located at the midpoint of the first TOF receiver 2 and the second TOF receiver 4, the first TOF receiver 2 and the second TOF receiver 4 are symmetrical relative to the floodlight projector 1, and the signals received by the first TOF receiver 2 and the second TOF receiver 4 are the same. The parameters of the first TOF receiver 2 and the second TOF receiver 4 are the same to ensure that the obtained signals are not affected by the equipment. The first TOF receiver 2 and the second TOF receiver 4 form a binocular system to provide additional depth data.

[0048] In some embodiments, the processor 3 uses the first model to identify the first region on the second infrared image, and obtains the depth value of the first region by calculating the parallax of the first region on the first infrared image and the second infrared image, as the final depth value of the first region.

[0049] Specifically, this embodiment not only identifies the target object, but also recalculates the depth value of the target object. Since the first infrared image and the second infrared image do not have data distortion caused by multipath effect, the depth value obtained by parallax calculation. Since the processor 3 has obtained the first region, it only needs to calculate the first region, thereby reducing the amount of calculation and reducing the time required for calculation, improving the efficiency. The first TOF receiver 2 and the second TOF receiver 4 form a binocular system, which can obtain accurate depth values of various angles of fold, effectively supplementing the TOF depth data.

[0050] ​The embodiment sets the first TOF receiver and the second TOF receiver, can form a binocular system by using the two receivers while obtaining the TOF depth data, and can obtain the accurate depth value of the first area by using the parallax principle to calculate the first area which may have deviation due to the multipath effect, replace the first area depth value in the TOF depth data, and obtain the accurate depth value of all areas, so that the accurate depth value of all areas is obtained with the least calculation amount and high efficiency.

[0051] Figure 4 Fig. 2 is a structural schematic diagram of another TOF camera in the embodiment of the present application. Figure 4 As shown in the figure, the another TOF camera in the embodiment of the present application comprises:

[0052] The flood projector 1 is used for projecting the flood light to the target area.

[0053] Specifically, the power of the flood projector 1 is adjustable and has at least two powers. The power of the flood projector 1 is adjusted by the current size. When the current is large, the power of the flood projector 1 is also large, and the flood light intensity of the projected flood light is large. When the current is small, the power of the flood projector 1 is also small, and the flood light intensity of the projected flood light is small.

[0054] The first TOF receiver 2 is used for receiving the reflected signal and generating the first infrared image.

[0055] The processor 3 is used for obtaining the first depth according to the phase difference between the flood light and the reflected signal, identifying the first infrared image by using the first model, obtaining the feature of the small object, and taking the corresponding first depth value as the depth value of the small object.

[0056] Specifically, unlike the identification of the small object by using the mutation value of the first depth in the foregoing embodiment, the processor 3 in the embodiment directly identifies the first infrared image by using the first model to obtain the feature of the small object.

[0057] After the characteristics of the fine object are obtained, the boundary of the fine object is identified, and the depth value of the boundary is taken as a constraint to perform linear regression on the depth value beyond the boundary around the boundary. An xy coordinate system of the image is established, x is the horizontal coordinate, and y is the vertical coordinate. Take a boundary as an example for description. The boundary is y = 3x + 2, and 2 ≤ x ≤ 3. The depth value at the point (2.4, 9.2) is 15 cm, and the value perpendicular to the boundary direction is taken as a constraint to perform linear regression. The function perpendicular to the boundary at the point (2.4, 9.2) is y = -3x + 16.4. The depth values of the adjacent points (2.1, 10.1), (2.2, 9.8), (2.3, 9.5), (2.5, 8.9), (2.6, 8.6), and (2.7, 8.3) of the point (2.4, 9.2) are 14.9 cm, 15.1 cm, 15.2 cm, 15.3 cm, 15.1 cm, and 14.8 cm, respectively. Due to the existence of multipath effect, the depth values of the four middle points are obviously deviated. The depth values of (2.1, 10.1) and (2.7, 8.3) are relatively accurate. Therefore, the depth value linear relationship of (2.1, 10.1), (2.7, 8.3), and (2.4, 9.2) is established, and the depth values of all points are fitted. For example, the linear relationship of (2.1, 10.1, 14.9) and (2.4, 9.2, 15) in the x-axis direction is established, and z represents the depth value. The parameters a and b in z = ax + b need to be solved, and the calculation result is a = 1 / 3 and b = 14.2, that is, z = x / 3 + 14.2, and the depth values of (2.2, 9.8) and (2.3, 9.5) are 14.93 cm and 14.97 cm, respectively.

[0058] In some embodiments, a regulator 5 is further included for controlling the power of the flood projector. The regulator 5 can control the power of the flood projector 1 by adjusting the current of the flood projector 1. The regulator 5 is connected in series with the flood projector 1 in the same circuit. The resistance of the regulator 5 is adjustable. When the resistance of the regulator 5 becomes larger, the total resistance in the circuit becomes larger, the current becomes smaller, and the current I passing through the flood projector 1 becomes smaller. According to the relationship between power and current P = IR, it can be known that the power of the flood projector 1 becomes smaller. 2 When the resistance of the regulator 5 becomes smaller, the total resistance in the circuit becomes smaller, the current becomes larger, the current I passing through the flood projector 1 becomes larger, and the power of the flood projector 1 becomes larger.

[0059] The regulator 5 can also control the emission energy of the flood projector 1 by controlling the projection time length of the flood projector 1. The regulator 5 controls the continuous projection time length of the flood projector 1 each time, and since the total energy Q = Pt, it can be known that the emission energy and the time length are in a positive correlation. When the projection time t is longer, the total energy of the projection is more. When the projection time t is shorter, the total energy of the projection is less.

[0060] In some embodiments, when a small object is identified on the first infrared image, the power of the flood projector is increased. In this embodiment, the TOF camera has two modes. In mode one, the power of the flood projector 1 is small, and clear and accurate identification of large objects can be achieved. When a small object is identified in mode one, mode two is switched. In mode two, the power of the flood projector 1 is large, and more information about small objects can be obtained, so that more data and detail information can be obtained. In mode two, the power of the flood projector 1 is large, and more detail information can be obtained, and the detail identification ability is stronger. At the same time, more information about small objects can be obtained, and the identification effect of the gathered small objects is very good. After running in mode two for a period of time, when no new small object is found, the TOF camera switches back to mode one. By alternating the two modes, the power of the TOF camera is low, and sufficient detail information can be obtained, which is beneficial to energy saving.

[0061] In this embodiment, the first model is used for identification on the first infrared image, and the characteristics of small objects can be stably obtained. Then the corresponding first depth value is used as the depth value of the small object. When the small object is identified, the depth value of the small object is obtained, and the positioning of the small object can be quickly realized, so that the obstacle avoidance function can be realized.

[0062] Figure 5 A structure diagram of a sweeping robot in an embodiment of the present application is shown in FIG. 1. As shown in the figure, the sweeping robot in the embodiment of the present application comprises a body 6 and the TOF camera 9 described in the foregoing embodiment mounted on the body 6. Figure 5

[0063] The body 6 is mounted with a moving component to realize autonomous movement of the sweeping robot. The body 6 is in the shape of a cylinder to realize more flexible turning and movement posture adjustment. The TOF camera is mounted on the front end of the body 6 to obtain information about the forward direction. The body 6 is provided with a storage box 7 for storing garbage. The body 6 is provided with a roller 8 for sweeping garbage into the storage box 7, and the position of the roller 8 is adjustable. When the TOF camera identifies a small object, the position of the roller 8 is adjusted downward after the small object enters the sweepable area, so as to increase the sweeping intensity and sweep the small object into the storage box 7. The amplitude of the downward adjustment of the position of the roller 8 is set according to the size of the roller 8 and the height of the roller 8 from the ground. The downward adjustment of the roller 8 increases the contact surface of the roller 8 with the ground and the friction, so that the small object can be wrapped and rolled into the storage box 7.

[0064] ​In some embodiments, at least two TOF cameras are provided to cover different angles of view in the vertical plane. The angle of view of at least one TOF camera includes a region 0.1 cm in front of the robot. A first TOF camera is arranged horizontally with a FOV of 100 degrees, which can be used for obstacle avoidance. A second TOF camera is arranged downwardly with a FOV of 80 degrees, which can be used for object recognition and cleaning.

[0065] The embodiment adjusts the roller according to the identified small object to increase the friction and clean the small object into the storage box, effectively cleans the small object, solves the poor cleaning effect of the prior art on the small object, and reduces the maintenance cost of the robot.

[0066] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0067] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the specific embodiments described above, and various modifications or changes can be made by those skilled in the art within the scope of the claims, which do not affect the essential content of the present application.

Claims

1. A TOF camera applied to a sweeping robot, characterized in that, The TOF camera comprises a floodlight projector, a first TOF receiver and a processor. The floodlight projector is configured to project a floodlight to a target area. The first TOF receiver is configured to receive a reflected signal and generate a first infrared image. The processor is configured to obtain a first depth according to a phase difference between the floodlight and the reflected signal, acquire a first region in which a depth mutation value is within a first range, and identify the first region on the first infrared image by using a first model to determine whether the first region is a small object.

2. The TOF camera according to claim 1, characterized in that The TOF camera further comprises a second TOF receiver configured to receive a reflected signal and generate a second infrared image; and the floodlight projector is located at a midpoint between the second TOF receiver and the first TOF receiver.

3. A TOF camera according to claim 2, characterized in that, The first TOF receiver and the second TOF receiver have the same parameters.

4. The TOF camera according to claim 2, characterized in that The processor identifies the first region on the second infrared image by using the first model, and obtains a depth value of the first region by calculating a parallax of the first region on the first infrared image and the second infrared image, as a final depth value of the first region.

5. The TOF camera according to claim 1, characterized in that, The first model is obtained by machine learning training.

6. A robot vacuum cleaner characterised in that, The TOF camera comprises the TOF camera according to any one of claims 1-5.

7. The robot vacuum cleaner of claim 6, wherein, An angle of view of the TOF camera comprises a region 0.1 cm away from a front of the robot.

Images