Dirt detection method, dirt detection system, and active camera
By setting a reflective occlusion structure in an active camera to generate amplitude images and determine the intensity of invalid field-of-view areas, the problem of requiring additional equipment or large computational load for dirt detection in existing technologies is solved, and real-time and accurate dirt detection and level determination are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG SUNNY INTELLIGENT OPTICAL TECH CO LTD
- Filing Date
- 2024-12-03
- Publication Date
- 2026-06-05
AI Technical Summary
Existing active camera-based dirt detection solutions require additional lighting or involve large computational loads, making it impossible to achieve real-time and efficient dirt detection, thus affecting depth detection accuracy.
By setting a reflective blocking structure outside the effective field of view of the receiver of the active camera, part of the light emitted by the projector is projected onto the reflective blocking structure and received by the receiver, an amplitude image is generated, and the intensity of the invalid field of view area is judged to determine the dirt.
It enables simple and rapid dirt detection without the need for additional supplementary lighting, improving the real-time performance and accuracy of detection, and distinguishing dirt levels for timely treatment.
Smart Images
Figure CN122160494A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of camera technology, and in particular to a dirt detection method, a dirt detection system, and an active camera. Background Technology
[0002] Active light depth cameras, such as TOF (Time of Flight) cameras or structured light cameras, are widely used and popularized due to their high depth detection accuracy. Taking a TOF camera as an example, a TOF camera first emits continuous light pulses onto the object being measured, then receives the light pulses reflected back from the object, and then calculates the distance between the object and the camera by detecting the time of flight of the light pulses.
[0003] However, when the emitter of an active camera is contaminated with dirt such as dust, fingerprints, water droplets, or smudges, the light that was originally concentrated within the field of view will be refracted and / or reflected due to the dirt, thus severely affecting the quality of the imaged point cloud. Therefore, timely detection of dirt on the camera emitter and taking effective measures to resolve it when it is present are crucial for obtaining accurate depth image information.
[0004] Currently, one existing dirt detection solution involves setting two sub-structures in the light-emitting direction of the camera's emitting end. Although it is possible to detect whether there is dirt on the transparent cover surface of the emitting end by judging whether the average gray value of the block corresponding to the two sub-structures changes in a specific way, the normal measurement of the active camera will be affected when this solution is applied, and real-time detection will not be possible.
[0005] In addition, another existing dirt detection scheme is to first set up supplementary lights around the protective cover to illuminate the protective cover, and then turn off the projector during calibration to obtain an image of the protective cover in a clean state. Although comparing the image obtained by the active camera when the projector is off during actual application with the image during calibration can determine whether there is dirt, this scheme has a relatively large computational load and requires additional supplementary lights, which may cause waterproof and dustproof problems. Summary of the Invention
[0006] One advantage of this application is that it provides a dirt detection method, a dirt detection system, and an active camera, which can achieve both depth detection and dirt detection without the need for additional lighting in practical applications, making it easy to realize real-time dirt detection.
[0007] Another advantage of this application is that it provides a dirt detection method, a dirt detection system, and an active camera. In one embodiment of this application, the dirt detection method can determine the presence and level of dirt based on the intensity of the invalid field of view in the amplitude diagram. The calculation is simple and requires little computation.
[0008] Another advantage of this application is that it provides a dirt detection method, a dirt detection system, and an active camera. In one embodiment of this application, the dirt detection method can further determine whether there is an influence of ambient light, so as to improve the accuracy of dirt detection.
[0009] Another advantage of this application is that it provides a dirt detection method, a dirt detection system, and an active camera, wherein complex structures and designs are not required to achieve the above objectives. Therefore, this application successfully and effectively provides a solution that not only offers a simple dirt detection method, dirt detection system, and active camera, but also increases the practicality and reliability of the dirt detection method, dirt detection system, and active camera.
[0010] To achieve at least one of the above advantages or other benefits and objectives of this application, this application provides a method for detecting dirt, comprising the steps of:
[0011] Acquire images captured by an active camera, wherein the active camera includes a projector, a receiver arranged corresponding to the projector, and a reflective shielding structure disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, and causes light rays projected by the projector outside the emission field of view to be partially projected onto the reflective shielding structure so that they are received by the receiver after reflection.
[0012] The image acquired via the receiver when the projector is turned on is processed to obtain an amplitude image, wherein the amplitude image includes an effective field of view region corresponding to the effective field of view range of the receiver and an ineffective field of view region located outside the effective field of view region; and
[0013] Determine the intensity of the invalid field of view region on the amplitude image to determine if there is dirt.
[0014] According to one embodiment of this application, the step of determining the intensity of the invalid field of view region on the amplitude image to determine whether there is dirt includes the following steps:
[0015] Calculate the intensity of the invalid field of view region on the amplitude image to obtain the amplitude value intensity; and
[0016] The amplitude value is compared with a preset intensity threshold to determine that the active camera is dirty when the amplitude value is greater than the preset intensity threshold.
[0017] According to one embodiment of this application, the step of determining the intensity of the invalid field of view region on the amplitude image to determine whether there is dirt further includes the following steps:
[0018] In response to the active camera being dirty, the amplitude value is compared sequentially with multiple level thresholds; and
[0019] Based on the comparison results of the level thresholds, the level of dirtiness of the active camera is determined.
[0020] According to one embodiment of this application, the dirt detection method further includes the step of determining the presence of dirt before the step of determining the intensity of the invalid field of view region on the amplitude image:
[0021] The image acquired via the receiver when the projector is off is processed to obtain the average ambient light intensity; and
[0022] Determine whether the average ambient light intensity of the ambient frame is greater than or equal to a preset ambient threshold: if yes, it is determined that the active camera is affected by ambient light, so the subsequent dirt judgment is skipped; if no, it is determined that the active camera is not affected by ambient light, so the subsequent dirt judgment is performed.
[0023] According to another aspect of this application, this application further provides a dirt detection system, including components communicatively connected to each other:
[0024] An image acquisition module is used to acquire images captured by an active camera, wherein the active camera includes a projector, a receiver arranged corresponding to the projector, and a reflective shielding structure disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, and causes light rays projected by the projector outside the emission field of view to be partially projected onto the reflective shielding structure so that they are received by the receiver after reflection.
[0025] An image processing module is used to process an image acquired via the receiver when the projector is turned on to obtain an amplitude image, wherein the amplitude image includes an effective field of view region corresponding to the effective field of view range of the receiver and an ineffective field of view region located outside the effective field of view region; and
[0026] The dirt detection module is used to determine the intensity of the invalid field of view region on the amplitude image in order to determine whether dirt exists.
[0027] According to one embodiment of this application, the dirt determination module includes an intensity calculation module and an intensity comparison module that are communicatively connected to each other; the intensity calculation module is used to calculate the intensity of the invalid field of view region on the amplitude image to obtain the amplitude value intensity; the intensity comparison module is used to compare the amplitude value intensity with a preset intensity threshold, so as to determine that the active camera is in a dirty state when the amplitude value intensity is greater than the preset intensity threshold.
[0028] According to one embodiment of this application, the dirt determination module further includes a level comparison module and a level determination module that are communicatively connected to each other; the level comparison module is used to compare the amplitude value intensity with the magnitude of a plurality of level thresholds in response to the active camera being in a dirty state; the level determination module is used to determine the dirt level of the active camera based on the level threshold comparison results.
[0029] According to one embodiment of this application, the dirt detection system further includes an environment judgment module communicatively connected to the image processing module and the dirt judgment module; the image processing module is further configured to process the image acquired by the receiver when the projector is turned off to obtain the average ambient frame light intensity; the environment judgment module is configured to determine whether the average ambient frame light intensity is greater than or equal to a preset environmental threshold: if so, it is determined that the active camera is affected by ambient light, so as to skip the subsequent dirt judgment; if not, it is determined that the active camera is not affected by ambient light, so as to perform the subsequent dirt judgment.
[0030] According to another aspect of this application, this application further provides an active camera, comprising:
[0031] The camera body includes a projector, a receiver arranged corresponding to the projector, and a light-transmitting cover plate disposed within the emission field of view of the projector and providing an emission end surface;
[0032] A reflective shielding structure, disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, is used to partially project light rays projected by the projector outside the emission field of view onto the reflective shielding structure so that they are received by the receiver after reflection; and
[0033] The aforementioned dirt detection system can be communicatively connected to the receiver for detecting dirt on the surface of the projector's transmitting end.
[0034] According to one embodiment of this application, the reflective shielding structure is part of the camera housing; wherein the reflective shielding structure is located on the side of the receiver away from the projector, and the reflective shielding structure protrudes forward from the light-transmitting cover.
[0035] According to one embodiment of this application, the reflective blocking structure is located within the invalid field of view of the receiver, so that the image information of the reflective blocking structure is located within the invalid field of view region of the amplitude image. Attached Figure Description
[0036] Figure 1 This is a schematic flowchart of a dirt detection method according to an embodiment of this application;
[0037] Figure 2A A schematic diagram of the dirt detection principle in the dirt detection method according to the above embodiments of this application is shown.
[0038] Figure 2B A schematic diagram of the structure of the amplitude image in the dirt detection method according to the above embodiments of this application is shown;
[0039] Figure 3 A schematic flowchart of the dirt determination step in the dirt detection method according to the above embodiments of this application is shown.
[0040] Figure 4A An example of an amplitude image from an active camera in a dirty state is shown;
[0041] Figure 4B An example of an amplitude image from an active camera in a clean state is shown;
[0042] Figure 5 This is a schematic flowchart of a dirt detection method according to another embodiment of this application;
[0043] Figure 6 This is a block diagram of a dirt detection system according to an embodiment of this application;
[0044] Figure 7 This is an application example of an active camera in a clean state according to an embodiment of this application;
[0045] Figure 8 This is an application example of an active camera in a dirty state according to the above embodiments of this application.
[0046] Explanation of key component symbols:
[0047] 1. Active camera; 10. Dirt detection system; 11. Image acquisition module; 12. Image processing module; 13. Dirt judgment module; 131. Intensity calculation module; 132. Intensity comparison module; 133. Grade comparison module; 134. Grade determination module; 14. Environmental judgment module; 20. Camera body; 21. Projector; 22. Receiver; 221. Image sensor; 222. Optical lens; 23. Light-transmitting cover plate; 30. Reflective blocking structure; 100. Amplitude image; 101. Effective field of view; 102. Ineffective field of view.
[0048] The above description of the main component symbols, together with the accompanying drawings and specific embodiments, provides a further detailed explanation of this application. Detailed Implementation
[0049] The following description is intended to disclose this application and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of this application defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of this application.
[0050] Those skilled in the art should understand that, in the disclosure of this application, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limitations on this application.
[0051] In this application, the term "a" in the claims and specification should be understood as "one or more," meaning that in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple. Unless explicitly indicated in the disclosure of this application that the number of the element is only one, the term "a" should not be construed as unique or singular, nor should it be construed as a limitation on the quantity.
[0052] In the description of this application, it should be understood that terms such as "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this application, it should be noted that, unless otherwise expressly specified and limited, terms such as "connected" or "linked" should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral connection; it can refer to a mechanical connection or an electrical connection; it can refer to a direct connection or an indirect connection through a medium. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0053] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0054] Considering that dirt on the transmitter surface of an active camera can cause refraction and / or reflection of light concentrated within the field of view, resulting in more light illuminating the camera's surface, not only exacerbates stray light but also increases the amplitude of the image captured by the active camera in the invalid field of view area. Therefore, this application creatively proposes a dirt detection method, a dirt detection system, and an active camera that, in practical applications, can simultaneously perform depth detection and dirt detection without the need for additional supplementary lighting, facilitating real-time dirt detection.
[0055] Specifically, refer to the accompanying drawings in the specification of this application. Figure 1 According to one embodiment of this application, a method for detecting dirt is provided, which may include the steps of:
[0056] S100: Acquire an image captured by an active camera, wherein the active camera includes a projector, a receiver arranged corresponding to the projector, and a reflective shielding structure disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, and partially project light rays projected by the projector outside the emission field of view onto the reflective shielding structure so that they are received by the receiver after reflection.
[0057] S200: The image acquired by the receiver when the projector is turned on is processed to obtain an amplitude image, wherein the amplitude image includes an effective field of view area corresponding to the effective field of view range of the receiver and an ineffective field of view area located outside the effective field of view area; and
[0058] S300: Determine the intensity of the invalid field of view region on the amplitude image to determine whether there is dirt.
[0059] It is worth noting that when there is dirt on the front surface of the projector of the active camera, the light projected by the projector within the emission field of view will be refracted and / or reflected at the dirt, causing more light to be reflected onto the reflective shielding structure and then received by the receiver to be displayed on the amplitude image, thereby increasing the intensity of the invalid field of view area on the amplitude image. Therefore, the dirt detection method of this application only needs to calculate the intensity of the invalid field of view area on the amplitude image to determine whether dirt exists, without having to calculate the intensity of the valid field of view area on the amplitude image. This simplifies the calculation of dirt detection and reduces the computational load, which is beneficial to improving the real-time performance of dirt detection.
[0060] Preferably, in step S100 of this application: the reflective blocking structure is located within the invalid field of view of the receiver, so that the image information of the reflective blocking structure is located within the invalid field of view of the amplitude image.
[0061] Thus, as Figure 2A As shown, since the reflective blocking structure 30 is located outside the effective field of view (i.e., the range corresponding to the effective receiving field of view angle β) of the receiver 22 and within the ineffective field of view (i.e., the range corresponding to the ineffective receiving field of view angle γ-β), although the light projected by the projector 21 outside the emission field of view angle α will be partially projected onto the reflective blocking structure 30 and received by the receiver 22 after reflection, the image of the reflective blocking structure 30 will be located within the ineffective field of view region of the amplitude image (i.e., the region corresponding to the ineffective field of view range) for dirt detection, but this will not affect the depth detection of the active camera. It is understood that the effective receiving field of view angle β mentioned in this application corresponds to the emission field of view angle α of the projector 21, and the two are usually equal.
[0062] In other words, when the front surface of the projector 21 of the active camera is dirty, the light projected by the projector 21 within the emission field of view (e.g., Figure 2AThe red light shown will be refracted and / or reflected at the dirt, causing more light to shine on the reflective shielding structure 30, be reflected, and then be received by the receiver 22 to be mainly presented in the invalid field of view area of the amplitude image, so that the intensity of the invalid field of view area of the amplitude image is significantly increased; that is, the effective field of view area of the amplitude image (i.e., the area corresponding to the effective field of view range) mainly records depth information for use in depth detection, and the invalid field of view area of the amplitude image mainly records dirt information for use in dirt detection, so that depth detection and dirt detection can be performed simultaneously without the addition of additional supplementary lighting, which facilitates real-time dirt detection.
[0063] Furthermore, limited by the rectangular photosensitive chip in the receiver, the amplitude image 100 is typically implemented as follows: Figure 2B The rectangular image shown, wherein the effective field of view 101 on the amplitude image 100 is implemented as follows Figure 2B The rounded rectangular region located at the center is shown, and the invalid field of view 102 on the amplitude image 100 is implemented as follows: Figure 2B The four corner regions shown are located outside the rounded rectangle region. Since the area of the effective field of view 101 on the amplitude image 100 is much larger than the area of the ineffective field of view 102, and the dirt detection method of this application only needs to calculate the intensity of the ineffective field of view 102 on the amplitude image 100 to determine whether dirt exists, without having to calculate the intensity of the effective field of view 101 on the amplitude image 100, the dirt detection method of this application is simple to calculate and has a small computational load, which is beneficial to improving the real-time performance of dirt detection.
[0064] For example, such as Figure 3 As shown, step S300 of the dirt detection method of this application may include the following steps:
[0065] S310: Calculate the intensity of the invalid field of view region on the amplitude image to obtain the amplitude value intensity; and
[0066] S320: Compare the amplitude value intensity with a preset intensity threshold to determine that the active camera is dirty when the amplitude value intensity is greater than the preset intensity threshold.
[0067] It is worth noting that the preset intensity threshold T mentioned in this application can be, but is not limited to, pre-set through experience or experimentation: when the amplitude intensity of the invalid field of view is greater than the preset intensity threshold T, it indicates that the light projected by the projector within the emission field of view is refracted and / or reflected at the dirt, causing more light to illuminate the reflective shielding structure, thus increasing the amplitude intensity of the invalid field of view to exceed the preset intensity threshold T. In other words, the surface of the active camera's emitting end is dirty, making the active camera dirty, thus obtaining the desired effect. Figure 4A The amplitude image shown; when the amplitude value intensity of the invalid field of view is less than or equal to the preset intensity threshold T, it indicates that the surface of the active camera's transmitter is clean and no more light will illuminate the reflective shielding structure, meaning the active camera is in a clean state, thus obtaining the image shown. Figure 4B The amplitude image shown.
[0068] Furthermore, when the surface of the active camera's transmitter is dirty, the dirt detection method of this application can further determine the dirt level based on the amplitude value intensity, so as to adopt different treatment schemes for different dirt levels. For example, when the dirt level is low, it indicates that there is less dirt on the surface of the active camera's transmitter, which has little impact on depth detection, and no cleaning measures can be taken temporarily until the depth detection is completed; while when the dirt level is high, it indicates that there is a lot of dirt on the surface of the active camera's transmitter, which has seriously affected depth detection, and cleaning measures must be taken immediately to avoid errors in the depth detection results, which would affect subsequent mapping or navigation applications.
[0069] For example, such as Figure 3 As shown, step S300 of the dirt detection method of this application may further include the following steps:
[0070] S330: In response to the active camera being dirty, sequentially compare the amplitude value intensity with multiple level thresholds; and
[0071] S340: Based on the comparison results of the level threshold, determine the level of dirtiness of the active camera.
[0072] It is worth noting that the dirt detection method of this application can be configured with different numbers of dirt levels according to the actual application scenario, such as four levels based on the strength of the dirt's impact. Specifically, step S300 of the dirt detection method of this application may further include the following steps:
[0073] In response to the active camera being dirty, the amplitude value intensity is compared with the first level threshold, the second level threshold, and the third level threshold in sequence;
[0074] In response to the amplitude value being less than or equal to the first level threshold, the active camera is determined to be in the first level of dirtiness.
[0075] In response to the amplitude value being greater than the first level threshold and less than or equal to the second level threshold, the active camera is determined to be in the second level of dirtiness.
[0076] In response to the amplitude value being greater than the second-level threshold and less than or equal to the third-level threshold, the active camera is determined to be in a third-level dirtiness condition; and
[0077] In response to the amplitude value being greater than the third level threshold, the active camera is determined to be in the fourth level of dirtiness.
[0078] Thus, when the active camera's dirt level is at the first level, it indicates that although the surface of the active camera's transmitter is dirty, its impact on the accuracy of depth detection is minimal, and dirt removal is not required. When the active camera's dirt level is at the second level, it indicates that although the surface of the active camera's transmitter is dirty, its impact on the accuracy of depth detection is small, and dirt removal can be temporarily postponed until the current depth detection is completed. When the active camera's dirt level is at the third level, it indicates that the surface of the active camera's transmitter has a small area of dirt, which has a significant impact on the accuracy of depth detection. Simple dirt removal is required before continuing depth detection, followed by deep cleaning of the transmitter surface after depth detection. When the active camera's dirt level is at the fourth level, it indicates that the surface of the active camera's transmitter has a large area of dirt, which has seriously affected the accuracy of depth detection. Immediate deep cleaning of the transmitter surface is required to ensure normal depth detection. It is understood that the dirt levels mentioned in this application can be set as needed, and the response strategies for different dirt levels can also be set according to actual needs, which will not be elaborated upon in this application.
[0079] It is worth noting that since the active camera is affected by strong ambient light during depth detection, the dirt detection method of this application is also affected by strong ambient light. To avoid significant deviations in dirt detection results due to strong ambient light, another embodiment of this application provides a dirt detection method. Compared to the above embodiments according to this application, as... Figure 5 As shown, another embodiment of the dirt detection method of this application differs in that: before step S300, it may further include the following step:
[0080] S400: Processes the image acquired via the receiver when the projector is off to obtain the average ambient frame light intensity; and
[0081] S500: Determine whether the average light intensity of the ambient frame is greater than or equal to a preset ambient threshold: if yes, determine that the active camera is affected by ambient light and skip the subsequent dirt judgment; if no, determine that the active camera is not affected by ambient light and proceed with the subsequent dirt judgment.
[0082] For example, such as Figure 5 As shown, steps S400 and S500 can be set between steps S100 and S200 to determine that the active camera is not affected by ambient light before processing the image captured by the receiver when the projector is turned on, thereby performing a dirt judgment and helping to avoid invalid calculations.
[0083] It is understood that the image captured by the receiver of the active camera when the projector is turned on is defined as the target frame image for depth detection; the image captured by the receiver of the active camera when the projector is turned off is defined as the environment frame image. In step S400 of the dirt detection method of this application, the data of the environment frame image can be extracted for detection to calculate the overall light intensity average of the environment frame, which is used to determine whether there is an influence of ambient light. In addition, the active camera being affected by ambient light mentioned in this application means that the active camera is greatly affected by ambient light, which will seriously interfere with the subsequent dirt judgment, and the current dirt detection needs to be terminated so that the dirt detection can be re-performed after adjusting the orientation of the active camera; the active camera not being affected by ambient light mentioned in this application means that the active camera is less affected by ambient light, which will not interfere with the subsequent dirt judgment, and the subsequent dirt judgment can be directly performed to complete the current dirt detection.
[0084] It is worth mentioning that, according to another aspect of this application, such as Figure 6 As shown, this application further provides a dirt detection system 10, which may include an image acquisition module 11, an image processing module 12, and a dirt determination module 13 that are communicatively connected to each other. The image acquisition module 11 is used to acquire images captured by an active camera, wherein the active camera includes a projector, a receiver arranged corresponding to the projector, and a reflective shielding structure disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, and partially projects light rays projected by the projector outside the emission field of view to the reflective shielding structure so that they are received by the receiver after reflection. The image processing module 12 is used to process the image captured by the receiver when the projector is turned on to obtain an amplitude image, wherein the amplitude image includes an effective field of view area corresponding to the effective field of view of the receiver and an ineffective field of view area located outside the effective field of view area. The dirt determination module 13 is used to determine the intensity of the ineffective field of view area on the amplitude image to determine whether dirt exists.
[0085] It is worth noting that, in one example of this application, such as Figure 6 As shown, the dirt determination module 13 may include an intensity calculation module 131 and an intensity comparison module 132 that are communicatively connected to each other; the intensity calculation module 131 is used to calculate the intensity of the invalid field of view region on the amplitude image to obtain the amplitude value intensity; the intensity comparison module 132 is used to compare the amplitude value intensity with a preset intensity threshold to determine that the active camera is in a dirty state when the amplitude value intensity is greater than the preset intensity threshold.
[0086] Furthermore, in one example of this application, such as Figure 6 As shown, the dirt judgment module 13 may further include a level comparison module 133 and a level determination module 134 that are communicatively connected to each other. The level comparison module 133 is used to compare the amplitude value intensity with the magnitude of a plurality of level thresholds in response to the active camera being in a dirty state. The level determination module 134 is used to determine the dirt level of the active camera based on the level threshold comparison results.
[0087] Furthermore, in the above embodiments of this application, such as Figure 6 As shown, the dirt detection system 10 may further include an environment judgment module 14 communicatively connected to the image processing module 12 and the dirt judgment module 13. The image processing module 12 is further used to process the image acquired by the receiver when the projector is turned off to obtain the average ambient frame light intensity. The environment judgment module 14 is used to determine whether the average ambient frame light intensity is greater than or equal to a preset environmental threshold. If so, it is determined that the active camera is affected by ambient light, so as to skip the subsequent dirt judgment. If not, it is determined that the active camera is not affected by ambient light, so as to perform the subsequent dirt judgment.
[0088] It is worth mentioning that, according to another aspect of this application, such as Figure 7 and Figure 8 As shown, this application further provides an active camera 1, which may include a camera body 20, a reflective shielding structure 30, and the aforementioned dirt detection system 10. The camera body 20 includes a projector 21, a receiver 22 corresponding to the projector 21, and a light-transmitting cover 23 disposed within the emission field of view of the projector 21 and providing an emission end surface. The reflective shielding structure 30 is disposed outside the effective field of view of the receiver 22 and outside the emission field of view of the projector 21, for partially projecting light projected by the projector 21 outside the emission field of view onto the reflective shielding structure 30 so that it is received by the receiver 22 after reflection. The dirt detection system 10 is communicatively connected to the receiver 22 of the camera body 20 for detecting dirt on the emission end surface of the projector 21.
[0089] It is understood that the light-transmitting cover 23 mentioned in this application can be implemented as a protective glass that only covers the projector 21, or it can be implemented as a protective glass that covers both the projector 21 and the receiver 22. This application will not elaborate further on this. In addition, the receiver 22 may include an image sensor 221 and an optical lens 222 located in the light-sensing path of the image sensor 221, so that light first passes through the optical lens 222 and is then received by the image sensor 221 to form an image.
[0090] Optionally, the reflective blocking structure 30 is located within the invalid field of view of the receiver 22, so that the image information of the reflective blocking structure 30 is located within the invalid field of view of the amplitude image.
[0091] It is worth noting that, such as Figure 7 and Figure 8 As shown, the reflective shielding structure 30 mentioned in this application can be, but is not limited to, implemented as part of the camera housing, that is, the reflective shielding structure 30 can be a structural component of the active camera 1 itself. For example, when the active camera 1 is used in scenarios such as robots, sweepers, or lawnmowers, due to the overall appearance design and installation requirements, the camera body 20 of the active camera 1 is usually recessed and installed inside the camera housing. Therefore, the reflective shielding structure 30 can be implemented as a structural component with a high reflectivity surface treatment on the camera housing, such as a white shielding structure, making it a stray light reflection part for dirt detection; that is, the part of the camera housing with a high reflectivity surface treatment located close to the front of the camera body 20 can serve as the reflective shielding structure 30, which usually avoids the emission field of view and the receiving field of view, and is outside the conventional field of view.
[0092] Optionally, such as Figure 7 and Figure 8 As shown, the reflective shielding structure 30 can be located on the side of the receiver 22 away from the projector 21, and the reflective shielding structure 30 protrudes forward from the light-transmitting cover plate 23 to ensure that the reflective shielding structure 30 is outside the effective field of view of the receiver 22 and within the ineffective field of view of the receiver 22.
[0093] It is worth noting that in other examples of this application, the reflective shielding structure 30 may not be a structural component of the active camera 1 itself, but may be a highly reflective shielding structure additionally added to the camera housing, which will not be elaborated further in this application.
[0094] Furthermore, the camera body 20 mentioned in this application can be implemented as a vision module such as a TOF module or a structured light module to project invisible light to obtain image data; it can also be implemented as a vision module such as an infrared camera module to receive invisible light to obtain image data; in other words, the active camera 1 of this application can be implemented as a vision camera with the aforementioned dirt detection system 10 in order to perform dirt detection.
[0095] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0096] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this invention, and these all fall within the protection scope of this application.
Claims
1. A method for detecting dirt and grime, characterized in that, include: Acquire images captured by an active camera, wherein the active camera includes a projector, a receiver arranged corresponding to the projector, and a reflective shielding structure disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, and causes light rays projected by the projector outside the emission field of view to be partially projected onto the reflective shielding structure so that they are received by the receiver after reflection. The image acquired by the receiver when the projector is turned on is processed to obtain an amplitude image, wherein the amplitude image includes an effective field of view area corresponding to the effective field of view range of the receiver and an ineffective field of view area located outside the effective field of view area; as well as Determine the intensity of the invalid field of view region on the amplitude image to determine if there is dirt.
2. The method for detecting dirt and grime according to claim 1, characterized in that, The step of determining the intensity of the invalid field of view region on the amplitude image to determine whether there is dirt includes the following steps: Calculate the intensity of the invalid field of view region on the amplitude image to obtain the amplitude value intensity; and The amplitude value is compared with a preset intensity threshold to determine that the active camera is dirty when the amplitude value is greater than the preset intensity threshold.
3. The method for detecting dirt and grime according to claim 2, characterized in that, The step of determining the intensity of the invalid field of view region on the amplitude image to determine whether there is dirt further includes the following steps: In response to the active camera being dirty, the amplitude value intensity is compared with the magnitude of multiple level thresholds in turn; and Based on the comparison results of the level thresholds, the level of dirtiness of the active camera is determined.
4. The method for detecting dirt according to any one of claims 1 to 3, characterized in that, Before the step of determining the presence of contamination by measuring the intensity of the invalid field of view region on the amplitude image, the method further includes the following step: The image acquired by the receiver when the projector is off is processed to obtain the average ambient light intensity. and Determine whether the average ambient light intensity of the ambient frame is greater than or equal to a preset ambient threshold: if yes, it is determined that the active camera is affected by ambient light, so the subsequent dirt judgment is skipped; if no, it is determined that the active camera is not affected by ambient light, so the subsequent dirt judgment is performed.
5. A dirt detection system, characterized in that, Including those that are communicatively connected: An image acquisition module is used to acquire images captured by an active camera, wherein the active camera includes a projector, a receiver arranged corresponding to the projector, and a reflective shielding structure disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, and causes light rays projected by the projector outside the emission field of view to be partially projected onto the reflective shielding structure so that they are received by the receiver after reflection. An image processing module is used to process an image acquired by the receiver when the projector is turned on to obtain an amplitude image, wherein the amplitude image includes an effective field of view area corresponding to the effective field of view range of the receiver and an invalid field of view area located outside the effective field of view area; as well as The dirt detection module is used to determine the intensity of the invalid field of view region on the amplitude image in order to determine whether dirt exists.
6. The dirt detection system according to claim 5, characterized in that, The dirt detection module includes an intensity calculation module and an intensity comparison module that are communicatively connected to each other; the intensity calculation module is used to calculate the intensity of the invalid field of view region on the amplitude image to obtain the amplitude value intensity; the intensity comparison module is used to compare the amplitude value intensity with a preset intensity threshold, so as to determine that the active camera is in a dirty state when the amplitude value intensity is greater than the preset intensity threshold.
7. The dirt detection system according to claim 6, characterized in that, The dirt detection module further includes a level comparison module and a level determination module that are communicatively connected to each other; the level comparison module is used to compare the amplitude value intensity with the magnitude of multiple level thresholds in turn when the active camera is in a dirty state; the level determination module is used to determine the dirt level of the active camera based on the level threshold comparison results.
8. The dirt detection system according to any one of claims 5 to 7, characterized in that, It also includes an environment judgment module communicatively connected to the image processing module and the dirt judgment module; the image processing module is further used to process the image acquired by the receiver when the projector is off to obtain the average ambient frame light intensity; the environment judgment module is used to determine whether the average ambient frame light intensity is greater than or equal to a preset environmental threshold: if so, it is determined that the active camera is affected by ambient light, so as to skip the subsequent dirt judgment; if not, it is determined that the active camera is not affected by ambient light, so as to perform the subsequent dirt judgment.
9. An active camera, characterized in that, include: The camera body includes a projector, a receiver arranged corresponding to the projector, and a light-transmitting cover plate disposed within the emission field of view of the projector and providing an emission end surface; A reflective shielding structure is disposed outside the effective field of view of the receiver and outside the emission field of view of the projector, for causing light rays projected by the projector outside the emission field of view to be partially projected onto the reflective shielding structure so that they are received by the receiver after reflection; as well as The dirt detection system as described in any one of claims 5 to 8 is communicatively connected to the receiver for detecting dirt on the emitting surface of the projector.
10. The active camera according to claim 9, characterized in that, The reflective shielding structure is part of the camera housing; wherein the reflective shielding structure is located on the side of the receiver away from the projector, and the reflective shielding structure protrudes forward from the light-transmitting cover.
11. The active camera according to claim 9, characterized in that, The reflective blocking structure is located within the invalid field of view of the receiver, so that the image information of the reflective blocking structure is located within the invalid field of view region of the amplitude image.