A depth camera module with dirt detection function and robot

Abstract

A kind of depth camera module with dirt detection function, characterized in that, including laser emitter, for emitting laser;Receiver, for receiving the reflected signal of the laser;Controller, for controlling the laser emitter respectively with first power and second power emit laser, the receiver respectively receives first signal and second signal, generates first depth map and second depth map, the first signal is subtracted from the second signal, obtains third depth map, according to the depth on the first depth map, the second depth map and the third depth map judge whether there is dirt on lens;Wherein, the first power is greater than the second power.This application does not need additional device, can carry out dirt detection at any time, and does not affect data acquisition, with very high efficiency and convenience.

Description

Technical Field

[0001] The present invention relates to the technical field of depth cameras, and in particular to a depth camera module and a robot with a dirt detection function. Background Art

[0002] Depth cameras are the eyes of many AI devices, such as robots, and are crucial for the efficient operation of various smart devices. Due to their unique characteristics, depth cameras are often damaged by environmental factors, such as water stains, dust, and scratches. When the depth camera lens is located on the outermost surface, it is susceptible to damage. When a depth camera has a protective lens on the outside, the protective lens is also susceptible to damage. Damage to both the depth camera lens and the protective lens will affect the data captured by the depth camera, making it crucial to promptly identify such contamination.

[0003] In the prior art, additional detection devices are often used to detect dirt on the depth camera lens or the protective lens.

[0004] For example, an invention discloses a dirt monitoring system for a transmitting module, a depth camera, an intelligent terminal, a dirt detection method and a computer-readable storage medium. The dirt monitoring system includes a light source, an optical element, a detection element and a processor. The light source is used to emit a light signal. The optical element is located on the projection path of the light source. The detection element generates a photocurrent after receiving the light signal reflected by the optical element. The processor is used to calculate the reflectivity of the optical element to the light signal based on the photocurrent. When the reflectivity of the optical element is greater than a first preset threshold, it is determined that the optical element is dirty.

[0005] However, these technologies are complex to operate and require additional equipment, making them difficult to be widely adopted in practical applications.

[0006] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of this application. Summary of the Invention

[0007] To this end, the present invention utilizes the characteristics of the depth camera itself, emits two lasers of different powers, obtains depth maps at different powers, calculates a third depth map, and then uses the depth on the third depth map to determine whether there is dirt on the lens. The present invention does not require additional equipment, can perform dirt detection at any time, and does not affect data acquisition, with very high efficiency and convenience.

[0008] In a first aspect, the present application provides a depth camera module with dirt detection function, characterized in that it comprises a laser emitter, a receiver and a controller.

[0009] The laser emitter is configured to emit laser.

[0010] The receiver is configured to receive the reflected signal of the laser.

[0011] The controller is configured to control the laser emitter to emit laser at a first power and a second power respectively, control the receiver to receive a first signal and a second signal respectively, generate a first depth map and a second depth map, subtract the second signal from the first signal to obtain a third depth map, and determine whether there is dirt on the lens according to the depths on the first depth map, the second depth map and the third depth map; wherein the first power is greater than the second power.

[0012] Optionally, the depth camera module with dirt detection function is characterized in that the controller adjusts the power by controlling the current size of the laser emitter.

[0013] Optionally, the depth camera module with dirt detection function is characterized in that the laser emitter emits laser alternately at the first power and the second power.

[0014] Optionally, the depth camera module with dirt detection function is characterized in that a first region N1 with a depth value less than N and a signal average intensity M1 are obtained on the first depth image, a second region N2 with a depth value less than N and a signal average intensity M2 are obtained on the second depth image, and a third region N3 corresponding to the first region N1 and a signal average intensity M3 are obtained on the third depth image; if M3 is greater than or equal to M2, it is determined that there is dirt; wherein N is a preset depth value, and the first power is twice the second power.

[0015] Optionally, the depth camera module with dirt detection function is characterized in that the error value n of the depth camera module with dirt detection function on the lens and N are positively correlated.

[0016] Optionally, the depth camera module with dirt detection function is characterized in that N=3n.

[0017] In a second aspect, the present application provides a depth camera module with dirt detection function, characterized in that it comprises a laser emitter, a receiver and a controller.

[0018] The laser emitter is configured to emit laser.

[0019] The receiver is used to receive the reflected signal of the laser;

[0020] The controller is used to control the laser emitter to emit laser at a first power F1 and a second power F2 respectively, and the receiver receives the first signal and the second signal respectively, generates a first depth map and a second depth map, and determines whether there is dirt on the lens based on the ratio of the depth value of the first depth map to the depth value of the second depth map.

[0021] Optionally, the depth camera module with a dirt detection function is characterized in that a first area N1 with a depth value less than N and a signal average strength M1 are obtained on the first depth image, and a second area N2 with a depth value less than N and a signal average strength M2 are obtained on the second depth image. If It is determined that there is dirt on the lens; where σ is the attenuation coefficient related to the lens.

[0022] In a third aspect, the present invention provides a robot, characterized in that it comprises a depth camera module with a dirt detection function as described in any one of the above items.

[0023] Optionally, the robot is characterized in that it further includes a distance sensor; the distance sensor is arranged adjacent to the depth camera module with a dirt detection function in the same direction, and when the distance sensor detects a close object, the depth camera module with a dirt detection function no longer determines that there is dirt.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention utilizes the signal obtained by the depth camera module itself for detection to obtain dirt data information, does not require additional devices or equipment for detection, and is very convenient. At the same time, it can be performed synchronously with normal data collection to improve efficiency.

[0026] The present invention transmits transmission signals of different powers to obtain reflection signals at different powers, and then calculates and determines whether there is dirt based on the reflection signals at different powers, thereby realizing real-time detection in various application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without inventive work. Other features, purposes and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0028] Figure 1 Schematic diagram of the structure of a depth camera module with a dirt detection function according to an embodiment of the present invention;

[0029] Figure 2 Schematic diagram of the structure of a robot in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several variations and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0031] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatus.

[0032] The embodiments of the present invention provide a depth camera module and system with a dirt detection function, aiming to solve the problems existing in the prior art.

[0033] The following describes in detail the technical solutions of the present invention and how the technical solutions of this application solve the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0034] The present invention utilizes the characteristics of the depth camera itself, emits two lasers of different powers, obtains depth maps at different powers, calculates a third depth map, and then uses the depth in the third depth map to determine whether there is dirt on the lens. The present invention does not require additional equipment, can perform dirt detection at any time, and does not affect data acquisition, with very high efficiency and convenience.

[0035] Figure 1FIG is a structural diagram of a depth camera module with a dirt detection function according to an embodiment of the present invention. Figure 1 As shown, a depth camera module with a dirt detection function in an embodiment of the present invention includes a laser transmitter 1, a receiver 2 and a controller 3.

[0036] The laser transmitter 1 is used for transmitting laser light.

[0037] Specifically, the laser emitter 1 can be various types of laser emitters, such as a structured light projector, a flood light projector, a switchable projector or a radar projector. The structured light projector is used to emit a laser beam with a structured light image. The flood light projector is used to project uniformly illuminated flood light. The switchable projector is used to project structured light or flood light with a certain distribution. The radar transmitter is used to emit a high-power laser signal to measure long-distance signals. The effective measurement distance of the radar transmitter is much greater than the effective measurement distance of the structured light projector, the flood light projector or the switchable projector. For different laser emitters, if there is dirt on the lens, the impact on the reflected signal is different, which will be explained in detail in the subsequent parts of this specification.

[0038] The power of the laser emitter 1 is adjustable. This refers to the power of the laser emitted by the laser emitter 1 being adjustable, rather than just the power of the laser emitter 1 itself being adjustable.

[0039] In some embodiments, the power of laser emitter 1 is adjusted by adjusting the current flowing through laser emitter 1. When the voltage is constant, power is proportional to current, so current adjustment can be used to adjust power. Connecting a resistor in parallel with the light source in laser emitter 1 allows current adjustment while maintaining the voltage constant, thereby adjusting the power of laser emitter 1. The more resistors connected in parallel, the more power can be adjusted. In this embodiment, only one resistor can be connected in parallel to achieve two levels of power adjustment for laser emitter 1.

[0040] In some embodiments, the power of laser emitter 1 is adjusted by adjusting the density of the laser light emitted by laser emitter 1. A filtering device is provided at the output end of laser emitter 1 to adjust the density of the emitted laser light. The filtering device can regulate the intensity of all emitted laser light or a portion of the laser light. The filtering device can be a polarizer, a semi-transparent membrane, or other device to achieve overall regulation of all emitted laser light. Alternatively, it can be a lens with a light-through hole, a display screen with a variable pattern, or other devices to adjust the amount of light transmitted.

[0041] The receiver 2 is used to receive the reflected signal of the laser.

[0042] Specifically, the receiver 2 receives signals in cooperation with the laser emitter 1. The signals received by the receiver 2 include reflection signals of the target area. If there is dirt on the lens, the receiver 2 can also receive reflection signals of the dirt on the lens. However, because the lens is too close to the receiver 2, the depth data obtained by the receiver 2 has too much error, and it is difficult to directly determine whether there is dirt on the lens through the depth data.

[0043] The controller 3 controls the laser emitter to emit laser beams at a first power and a second power respectively, controls the receiver to receive a first signal and a second signal respectively, and generates a first depth map and a second depth map. The controller 3 subtracts the second signal from the first signal to obtain a third depth map, and determines whether there is dirt on the lens according to the depths on the first depth map, the second depth map, and the third depth map. The first power is greater than the second power.

[0044] Specifically, the controller 3 controls the operation of the laser emitter 1 and the receiver 2, and can adjust the power of the laser emitter 1. When the laser emitter 1 emits laser beams at a first power, the receiver 2 receives a first signal; when the laser emitter 1 emits laser beams at a second power, the receiver 2 receives a second signal. Because the first power is greater than the second power, the intensity of the first signal is greater than the intensity of the second signal. Because the third depth map is generated by subtracting the second signal from the first signal, there is a linear relationship between the data on the first depth map, the second depth map, and the third depth map. However, because the distance between the dirt on the lens and the depth camera module is too close, the error is too large, so the dirt area can be identified according to the abnormal data on the first depth map, the second depth map, and the third depth map, i.e., the area that is not linearly positively correlated with the intensity of the emission signal of the laser emitter 1.

[0045] In some embodiments, the laser emitter 1 emits laser beams at the first power and the second power alternately. The method by which the controller 3 controls the laser emitter 1 to adjust the power can be any of the feasible schemes. The ratio of the first power to the second power can be any ratio, such as 1:1, 1:2, 1:3, 2:1, 3:1, etc. When the ratio of the first power to the second power is 1:1, the images obtained by the adjacent first power and second power are used to determine whether there is dirt on the lens. When the ratio of the first power to the second power is not 1:1, taking 2:1 as an example, each laser beam emitted at the second power is surrounded by laser beams emitted at the first power, and then each laser beam emitted at the second power is calculated with the laser beams emitted at the first power before and after it to obtain the dirt condition.

[0046] In some embodiments, a first area N1 with a depth value less than N and an average signal intensity M1 are obtained on the first depth image; a second area N2 with a depth value less than N and an average signal intensity M2 are obtained on the second depth image; and a third area N3 corresponding to the first area N1 and an average signal intensity M3 are obtained on the third depth image. If M3 ≥ M2, contamination is determined. N is a preset depth value, and the first power is twice the second power. This embodiment provides a computationally simple contamination detection method that can quickly and effectively detect contamination. N is set based on the type of depth camera module. For example, when laser emitter 1 is a radar emitter, the value of N is greater than when it is a structured light projector, a flood projector, or a switchable projector. N is positively correlated with the lens error n of a depth camera module with contamination detection. To effectively identify contamination while preventing objects from being misidentified as contaminated during normal measurement, 5n>N>2n. Testing with different types of depth cameras has shown that N=3n is suitable for most depth camera modules.

[0047] Different from the controller function in the previous embodiments, in some embodiments, the controller controls the laser emitter to emit laser light at a first power F1 and a second power F2, respectively. The receiver receives the first and second signals, generates a first depth map and a second depth map, and determines whether the lens is contaminated based on the ratio of the depth values ​​in the first depth map to the depth values ​​in the second depth map. Compared to the previous embodiment, this embodiment does not require calculating the third depth map using the first and second signals. Instead, the determination is made directly based on the ratio of the first and second signals, thus simplifying the process and improving efficiency. Because the time interval between the acquisition of the first and second signals is very short, typically less than 0.1 seconds, the change in the target area of ​​the depth camera module is very small. Within the effective measurement range, the ratio of the depth values ​​in the first depth map to the second depth map is equal to F1 / F2. However, on the lens, the depth values ​​in the first depth map and the second depth map are not equal to F1 / F2. Therefore, this characteristic can be used to identify contaminated areas.

[0048] In some embodiments, a first area N1 with a depth value less than N and a signal average intensity M1 are obtained on the first depth image, and a second area N2 with a depth value less than N and a signal average intensity M2 are obtained on the second depth image. If If the result is true, it is determined that there is dirt on the lens; wherein, σ is an attenuation coefficient related to the lens. In this embodiment, the depth value is used to screen the regions on the first depth image and the second depth image, and the first region N1 and the second region N2 are obtained. The value of N is the same as that in the foregoing embodiments. The first region N1 and the second region N2 can be continuous regions or discontinuous regions. The area of the first region N1 and the second region N2 can be large or small. When the dirt is a particle, the area of the first region N1 and the second region N2 is the smallest. σ is not only related to the parameters of the lens itself, but also related to the positions of the lens from the laser emitter 1 and the receiver 2. This embodiment is especially suitable for the scenario where the laser emitter is a radar emitter.

[0049] Figure 2 A structural schematic diagram of a robot in an embodiment of the present application. Figure 2 The robot shown in FIG. 6 includes a depth camera 601, a robot body 602, and a display screen 603. The depth camera 601 is the depth camera module with the dirt detection function described in any of the foregoing embodiments. The depth camera 601 can obtain the scene in front of the robot, so as to obtain three-dimensional information. Generally, the depth camera 601 also includes an RGB camera, which can obtain an RGB image, so as to generate an RGBD image in combination with the depth image. The robot body 602 can select different functions according to different types of robots. For example, a food delivery robot can have a tray, a support, a driving wheel, etc.; a welcome robot can have a driving wheel, a human-shaped body, etc. The display screen 603 is used for displaying information and interacting with the user. The display screen 603 can be one-way or two-way. Due to the use of the high-integration automatic switching depth camera, the robot can accommodate more sensors in the same size, so as to have better environmental perception capability. Of course, the robot can also reduce the size, so as to adapt to more scenarios with strict size requirements. The automatic switching depth camera can also set different trigger distances by monitoring the range, so as to realize the function of automatically starting the automatic switching depth camera at a short distance and closing the automatic switching depth camera at a long distance, thereby saving energy consumption.

[0050] In some embodiments, a distance sensor 604 is further included; the distance sensor is arranged adjacent to the depth camera module with dirt detection function in the same direction, and when the distance sensor detects a close object, the depth camera module with dirt detection function no longer determines that it is dirty. The distance sensor 604 is used to detect close-range target objects, which can prevent the depth camera module with dirt detection function from judging close-range target objects as dirty, and can further improve the recognition accuracy of the depth camera module with dirt detection function. Only one distance sensor 604 needs to be set, and it must be set adjacent to the depth camera module with dirt detection function to improve stability and reliability. The distance between the distance sensor 604 and the edge of the depth camera module with dirt detection function does not exceed 1 cm.

[0051] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be apparent to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0052] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A depth camera module with a dirt detection function, characterized in that: Includes laser transmitter, receiver and controller; The laser emitter is used to emit laser; The receiver is used to receive the reflected signal of the laser; The controller is configured to control the laser transmitter to emit laser light at a first power and a second power, respectively; the receiver receives a first signal and a second signal, respectively, generates a first depth map and a second depth map, subtracts the second signal from the first signal to obtain a third depth map, and determines whether the lens is dirty based on the depths in the first depth map, the second depth map, and the third depth map; wherein the first power is greater than the second power; A first area N1 with a depth value less than N and an average signal intensity M1 are obtained on the first depth image, a second area N2 with a depth value less than N and an average signal intensity M2 are obtained on the second depth image, and a third area N3 corresponding to the first area N1 and an average signal intensity M3 are obtained on the third depth image. If M3 ≥ M2, it is determined that there is dirt; wherein N is a preset depth value and the first power is twice the second power.

2. The depth camera module with dirt detection function according to claim 1, characterized in that: The controller adjusts the power by controlling the current of the laser emitter.

3. The depth camera module with dirt detection function according to claim 1, characterized in that: The laser transmitter emits alternately at the first power and the second power.

4. The depth camera module with dirt detection function according to claim 1, characterized in that: The N is positively correlated with the error value n of the lens of the depth camera module with dirt detection function.

5. The depth camera module with dirt detection function according to claim 4, characterized in that: N=3n.

6. A depth camera module with a dirt detection function, characterized in that: Includes laser transmitter, receiver and controller; The laser emitter is used to emit laser; The receiver is used to receive the reflected signal of the laser; The controller is configured to control the laser emitter to emit laser light at a first power F1 and a second power F2, respectively; the receiver receives the first signal and the second signal, respectively, generates a first depth map and a second depth map, and determines whether the lens is dirty based on a ratio of a depth value in the first depth map to a depth value in the second depth map; A first area N1 with a depth value less than N and a signal average intensity M1 are obtained on the first depth image, and a second area N2 with a depth value less than N and a signal average intensity M2 are obtained on the second depth image. If , it is determined that there is dirt on the lens; among them, is the attenuation coefficient related to the lens.

7. A robot, characterized in that: A depth camera module with a dirt detection function comprising any one of claims 1-6.

8. A robot according to claim 7, characterized in that: It also includes a distance sensor; the distance sensor is arranged adjacent to the depth camera module with a dirt detection function in the same direction, and when the distance sensor detects a close object, the depth camera module with a dirt detection function no longer determines whether there is dirt.

Images