Detection apparatus, sensing module and method

Abstract

A detection apparatus, a sensing module, and a method, which can be applied to scenarios such as perimeter monitoring and smart highways to improve the accuracy of detection and achieve all-day and all-weather imaging and detection. On the basis of a 3D point cloud and a 2D color image of a detection object, or the 3D point cloud, an 2D grayscale image and the 2D color image of the detection object, the detection apparatus can generate an alarm.

Description

A detection device, sensing module and method

[0001] This application claims priority to Chinese Patent Application No. 202411628934.X, filed on November 13, 2024, entitled "A Detection Device, Sensing Module and Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of intelligent security, and more specifically, to a detection device, sensing module, and method. Background Technology

[0003] In scenarios such as perimeter monitoring and smart highways, when facing the challenges of complex weather conditions such as rain, fog, and no light, the detection of small targets such as animals and scattered objects at long distances often suffers from missed detections and low accuracy when using common cameras or solutions that combine cameras and radar. Therefore, how to achieve high accuracy and low false negative rate detection around the clock and in all weather conditions is a problem that needs to be solved. Summary of the Invention

[0004] This application provides a detection device, a sensing module, and a method. The detection device, sensing module, and method provided in this application can be applied to scenarios such as perimeter monitoring and smart highways, improving detection accuracy and enabling all-weather, 24 / 7 imaging and detection.

[0005] In a first aspect, this application provides a detection device. The detection device includes: a sensing module, an imaging module, and a processing module, wherein the sensing module is used to generate a three-dimensional 3D point cloud of the target to be detected and send the 3D point cloud to the processing module; the imaging module is used to generate a 2D color image of the target to be detected and send the 2D color image to the processing module; and the processing module is used to generate an alarm based on the 2D color image and the 3D point cloud.

[0006] It should be noted that 3D point clouds contain the three-dimensional geometric shape and spatial position information of objects, which helps to more accurately locate and identify small targets. That is, by outputting 3D point clouds, this application enables the detection device provided to support small target detection. Furthermore, the detection device provided by this application, combined with the rich color information provided by 2D color images, makes the contrast between the target and the background more obvious, thereby improving the detectability and recognition rate of the target, and further enhancing the accuracy of detection.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the sensing module includes a light source, a beam splitter, a scanning unit, a reflection unit, a spatial light modulator, and a detector, wherein the light source is used to emit a first light beam toward the beam splitter; the beam splitter is used to transmit the first light beam to the scanning unit and reflect a fourth light beam from the scanning unit to the detector; the scanning unit is used to reflect the first light beam toward the spatial light modulator and reflect the fourth light beam from the spatial light modulator to the beam splitter; the spatial light modulator is used to generate a second light beam based on random speckle coding of the first light beam and emit the second light beam, and to generate the fourth light beam based on random speckle decoding of a third light beam reflected from the detected target and emit the fourth light beam toward the scanning unit; the detector is used to receive the fourth light beam from the beam splitter and generate the 3D point cloud of the detected target based on the fourth light beam, or generate the 3D point cloud and the 2D grayscale image based on the fourth light beam.

[0008] In this application, the spatial light modulator utilizes random speckle coding to generate random speckle patterns. These patterns modulate light through a scattering medium, thereby achieving full or partial sampling of object information. Therefore, the detection device provided in this application can effectively capture object information even in complex environments such as rain and fog. Simultaneously, random speckle decoding uses algorithms to handle the relationship between the speckle pattern and reflected light intensity information, eliminating the need for complex hardware. This simplifies the design of the detection device and further improves image quality. Furthermore, the spatial light modulator can dynamically generate and adjust the speckle pattern, enabling the detection device provided in this application to quickly adapt to environmental changes under rain and fog conditions. It not only supports all-weather, rain- and fog-penetrating imaging for video evidence collection but also maintains high-resolution imaging.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the sensing module further includes a reflection unit located in the optical path between the scanning unit and the spatial light modulator. The reflection unit is used to reflect the first light beam from the scanning unit to the spatial light modulator and to reflect the fourth light beam from the spatial light modulator to the scanning unit.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the perception module is further configured to generate a 2D grayscale image of the detected target and send the 2D grayscale image to the processing module; the processing module is further configured to generate the alarm based on the 2D grayscale image, the 3D point cloud and the 2D color image.

[0011] Using 2D grayscale images for target detection allows the detection device to maintain good detection performance even in environments with large changes in lighting.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, when the processing module is further configured to generate the alarm based on the 2D grayscale image, the 3D point cloud, and the 2D color image, the processing module is specifically configured to generate a pseudo-color image based on the 2D grayscale image, and generate the alarm based on the pseudo-color image, the 3D point cloud, and the 2D color image.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the detector is also used to generate a 2D grayscale image of the detected target based on the fourth beam.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the light source is a laser light source.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the scanning unit is a microelectromechanical system (MEMS) micromirror array.

[0016] MEMS micromirror arrays enable the light beam emitted by the sensing module to perform fast and more precise scanning and sensing.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the imaging module is a visible light imaging module.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the sensing module, the imaging module, and the processing module are integrated into a single module.

[0019] By integrating the sensing module, imaging module, and processing module into one module, the detection device provided in this application can be made smaller in size, easier to install, and parameter settings and coordinate matching can be completed before installation.

[0020] Secondly, this application provides a perception module for generating a three-dimensional point cloud of a detection target, wherein the 3D point cloud is used to generate an alarm.

[0021] In conjunction with the second aspect, in some implementations of the second aspect, the sensing module includes a light source, a beam splitter, a scanning unit, a spatial light modulator, and a detector. The light source is used to emit a first light beam toward the beam splitter; the beam splitter is used to transmit the first light beam to the scanning unit and reflect a fourth light beam from the scanning unit to the detector; the scanning unit is used to reflect the first light beam toward the spatial light modulator and reflect the fourth light beam from the spatial light modulator back to the beam splitter; the spatial light modulator is used to generate a second light beam based on random speckle coding of the first light beam and emit the second light beam, and to generate the fourth light beam based on random speckle decoding of a third light beam reflected from the detected target and emit the fourth light beam toward the scanning unit; the detector is used to receive the fourth light beam from the beam splitter and generate the 3D point cloud of the detected target based on the fourth light beam.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, the sensing module further includes a reflection unit located in the optical path between the scanning unit and the spatial light modulator. The reflection unit is used to reflect the first light beam from the scanning unit to the spatial light modulator and to reflect the fourth light beam from the spatial light modulator to the scanning unit.

[0023] In conjunction with the second aspect, in some implementations of the second aspect, the perception module is further configured to generate a 2D grayscale image of the detected target, and the 2D grayscale image and the 3D point cloud are used to generate the alarm.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, when the 2D grayscale image and the 3D point cloud are used to generate the alarm, the 2D grayscale image is used to generate a pseudo-color image, and the pseudo-color image and the 3D point cloud are used to generate the alarm.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the detector is also used to generate a 2D grayscale image of the detected target based on the fourth beam.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the scanning unit is a microelectromechanical system (MEMS) micromirror array.

[0027] In conjunction with the second aspect, in some implementations of the second aspect, the light source is a laser light source.

[0028] Thirdly, this application provides a detection method. The method includes: generating a three-dimensional 3D point cloud of the target to be detected; generating a 2D color image of the target to be detected; and generating an alarm based on the 2D color image and the 3D point cloud.

[0029] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: generating a 2D grayscale image of the detected target; wherein, generating an alarm based on the 2D color image and the 3D point cloud includes: generating the alarm based on the 2D grayscale image, the 3D point cloud, and the 2D color image.

[0030] In conjunction with the third aspect, in some implementations of the second aspect, the method further includes: generating a pseudo-color image based on the 2D grayscale image; wherein, generating the detection report based on the 2D grayscale image, the 3D point cloud, and the 2D color image includes: generating the alarm based on the pseudo-color image, the 3D point cloud, and the 2D color image.

[0031] The beneficial effects of the second and third aspects mentioned above can be found in the description of the beneficial effects in the first aspect, and will not be repeated here. Attached Figure Description

[0032] Figure 1 is a schematic diagram of the structure of a detection device 100 provided in an embodiment of this application.

[0033] Figure 2 is a schematic diagram of the structure of a sensing module 110 provided in an embodiment of this application.

[0034] Figure 3 is a schematic diagram of the structure of an imaging module 120 provided in an embodiment of this application.

[0035] Figure 4 is a schematic diagram of the structure of a processing module 130 provided in an embodiment of this application.

[0036] Figure 5 is a schematic flowchart of a detection method 500 provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0038] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. In the textual descriptions or drawings of the embodiments of this application shown below, the terms "first," "second," etc., and various numerical designations are merely for descriptive convenience and are not necessarily used to describe a specific order or sequence, nor are they intended to limit the scope of the embodiments of this application. For example, distinguishing different light beams, etc.

[0039] References to "some embodiments" and the like in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in some embodiments," "in other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiments, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically emphasized.

[0040] The terms “comprising” and “having” and any variations thereof used in the embodiments of this application shown below are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0041] In the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Embodiments or designs described as "exemplary" or "for example" should not be construed as being more preferred or advantageous than other embodiments or designs. The use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0042] The detection device and method provided in this application can be applied to intelligent transportation systems, such as highway event detection and violation detection, and also to perimeter intrusion prevention scenarios, such as construction site security protection, energy facility security protection, and airport security protection. Perimeter intrusion prevention can be understood as monitoring and protecting the boundaries of a specific area to prevent unauthorized personnel or objects from entering. This type of protection is typically used in sensitive areas such as airports, industrial parks, and warehouses to ensure security and prevent potential threats such as theft and vandalism. The solution provided in this application can meet the all-weather, all-time imaging and small target detection requirements of highway event detection or perimeter intrusion prevention in complex scenarios such as nighttime darkness, heavy rain, and dense fog.

[0043] Figure 1 is a schematic diagram of a detection device 100 provided in an embodiment of this application. As shown in Figure 1, the detection device 100 includes a sensing module 110, an imaging module 120, and a processing module 130. The sensing module 110 generates a three-dimensional (3D) point cloud of the target and sends the 3D point cloud to the processing module 130. The imaging module 120 generates a 2D color image of the target and sends the 2D color image to the processing module 130. The processing module 130 generates an alarm based on the 2D color image and the 3D point cloud.

[0044] In this application, an alarm can be understood as an event alarm, such as the presence of spilled material or an intruding small animal at a certain location on a highway, or the presence of an intruding small animal or a stranger at a certain location in a security protection scenario. For example, an alarm may include the following: 1. Alarm time: Recording the time the intrusion occurred. 2. Alarm location: Indicating which specific area's perimeter was violated. 3. Alarm type: Specifying the type of intrusion, such as whether it is a human intrusion or a vehicle intrusion. In some embodiments, the perception module 110 may only generate a 3D point cloud of the detected target, for example, during the day, in good weather, or in a relatively simple detection environment where the background objects are relatively fixed. In this case, the processing module 130 generates an alarm based on the 3D point cloud of the detected target and the 2D color image of the detected target from the imaging module 120. It should be noted that this application does not limit the method by which the perception module 110 generates the 3D point cloud of the detected target; for example, it may be through laser scanning or through a time-of-flight (TOF) camera. Understandably, the structure of the perception module 110 may differ depending on the method used to generate 3D point clouds.

[0045] To enable the detection device provided in this application to support all-weather, rain- and fog-penetrating imaging, in some embodiments, the sensing module 110 can also simultaneously generate a 3D point cloud and a 2D grayscale image of the target, for example, in scenarios with low visibility at night or in severe weather. In this case, the sensing module 110 can generate a 2D grayscale image in low-light or even no-light environments, and simultaneously send the 3D point cloud and the 2D grayscale image of the target to the processing module. The processing module 130 then generates an alarm based on the 3D point cloud, the 2D grayscale image, and the 2D color image of the target.

[0046] It should be noted that a grayscale image can be understood as each pixel having only one sampled color, i.e., a grayscale value. There are typically 256 grayscale values, ranging from 0 (black) to 255 (white). This representation of grayscale values ​​allows grayscale images to more accurately express the brightness information in an image without requiring color information. Therefore, 2D grayscale images have good adaptability and robustness under different lighting conditions. In other words, even in environments with significant lighting variations, grayscale images can still maintain good target detection performance. Therefore, in this application, generating 2D grayscale images in low-light environments through the perception module 110 can enrich the application scenarios of the detection device provided in this application, enabling detection under different lighting intensities. Furthermore, it should be noted that after the 2D grayscale image generated by the perception module 110 is input to the processing module 130, the processing module 130 processes the grayscale image. Since the grayscale image only contains brightness information, not color information, the processing process of the processing module 130 can be simplified.

[0047] It should be noted that in the scheme of this application, 3D point clouds, 2D grayscale images, 2D color images, and pseudo-color images (hereinafter referred to as pseudo-color images) can all be understood as information or data. For example, a 3D point cloud can be understood as 3D point cloud information or 3D point cloud data, which is a collection of points in three-dimensional space, usually described by three-dimensional coordinates (X, Y, Z), and may contain additional attribute information, such as color or reflection intensity. 2D grayscale images, 2D color images, and pseudo-color images can be understood as corresponding 2D grayscale image data (or 2D grayscale image information), 2D color image data (or 2D grayscale image information), and pseudo-color image data (or 2D grayscale image information). For example, the processing module 130 generates an alarm based on 3D point clouds, 2D grayscale images, and 2D color images, which can be understood as the processing module 130 generating an alarm based on 3D point cloud information, 2D grayscale image information, and 2D color image information.

[0048] Next, the sensing module 110, imaging module 120 and processing module 130 included in the detection device 100 will be described in detail.

[0049] Figure 2 is a schematic diagram of the structure of two sensing modules 110 provided in the embodiments of this application. As shown in Figure 2(a), the sensing module 110 includes a light source 210, a beam splitter 220, a scanning unit 230, a spatial light modulator 250, and a detector 260. Specifically, when the sensing module 110 detects a target, the light source 210 emits a first beam to the beam splitter 220. The beam splitter 220 transmits the first beam to the scanning unit 230 and reflects a fourth beam from the scanning unit 230 to the detector 260. The scanning unit 230 reflects the first beam to the spatial light modulator 250 and reflects the fourth beam from the spatial light modulator 250 to the beam splitter 220. The spatial light modulator 250 generates a second beam based on random speckle coding of the first beam and emits the second beam, and generates a fourth beam based on random speckle decoding of the third beam reflected from the target and emits the fourth beam to the scanning unit 230. The detector 260 is used to receive the fourth beam from the beam splitter 220 and generate a 3D point cloud of the target based on the fourth beam, or generate and output a 3D point cloud and a 2D grayscale image of the target based on the fourth beam.

[0050] Optionally, the light source 210 can be a laser light source or a light-emitting diode (LED) light source. In some embodiments, the laser light source can be an edge-emitting laser (EEL), a vertical cavity surface-emitting laser (VCSEL), a solid-state laser, or a fiber laser, etc. It can be a monochromatic laser, such as, but not limited to, a near-infrared laser of 905nm-1550nm; or it can be a white light laser, such as a laser that excites white light through a phosphor wheel using a blue laser. In other embodiments, the LED light source can be a white LED, an infrared LED, etc. It is understood that by using a laser light source or an LED light source, the lifespan of the sensing module 110 provided in this application can be extended, avoiding frequent replacement and adjustment of the light source.

[0051] Beam splitter 220 enables the transmission of the first beam and the reflection of the fourth beam. In some embodiments, to increase the transmittance of the first beam, the surface of beam splitter 220 used for transmitting the first beam, such as surface 1 in FIG2, may be provided with an antireflection coating (or an anti-reflection coating). In other embodiments, to increase the reflectance of the fourth beam, the surface of beam splitter 220 used for transmitting the fourth beam, such as surface 2 in FIG2, may be provided with an anti-reflection coating.

[0052] The scanning unit 230 can be a fixed deflection component or a dynamic deflection component. Fixed deflection components include, but are not limited to, mirrors, prisms, etc. Dynamic deflection components include, but are not limited to, micro-electro-mechanical systems (MEMS), motorized mirrors (capable of two-dimensional angle deflection), motorized prisms (capable of two-dimensional angle deflection), and fast-reflecting mirrors (using piezoelectric ceramics or voice coil motors to rapidly deflect or shift the beam).

[0053] In this application, a spatial light modulator 250 is used to perform random speckle encoding on a first beam and random speckle decoding on a third beam. Specifically, the process of the spatial light modulator 250 performing random speckle encoding on the first beam to generate a second beam can be as follows: First, when the first beam enters the spatial light modulator 250, the light waves at different points interfere with each other, forming a random intensity distribution pattern, i.e., a speckle pattern. Subsequently, the spatial light modulator 250 modulates the first beam using the generated speckle pattern, for example, by applying phase and / or amplitude modulation to generate the second beam. The process of the spatial light modulator 250 performing random speckle decoding on the third beam to generate a fourth beam can be understood as follows: the spatial light modulator 250 generates a wavefront with an opposite phase to the speckle pattern, thereby correcting and / or shaping the third beam. At the same time, in order to reduce errors and improve the accuracy of decoding, the third beam is also denoised to filter out the target wavelength (e.g., 905nm laser), thereby generating the fourth beam.

[0054] Optionally, detector 260 may be a silicon photomultiplier (SiPM) detector or an avalanche photodiode (APD), etc., and this application does not limit its application. It is understood that when detector 260 generates a 3D point cloud of the target based on the fourth beam, or generates a 3D point cloud and a 2D grayscale image of the target based on the fourth beam, the output of the 3D point cloud and the 2D grayscale image of the target is completed by a signal processing device (or signal processing module) connected to detector 260. In other words, the generation of the 3D point cloud and the 2D grayscale image of the target by detector 260 requires the use of corresponding processing circuits and algorithms, such as the ToF algorithm, and this application does not limit its application.

[0055] It should be noted that Figure 2(a) above is only an exemplary illustration of the sensing module 110 shown in Figure 1. That is to say, the sensing module 110 in the detection device 100 is not limited to the structure shown in Figure 2(a). For example, for Figure 2(a), one or more reflection units can be provided between the scanning unit 230 and the spatial light modulator 250. When a reflection unit is provided between the scanning unit 230 and the spatial light modulator 250, as shown in Figure 2(b), the detection device 100 also includes a reflection unit 240. The reflection unit 240 is used to reflect the first beam from the scanning unit 230 to the spatial light modulator 250, and to reflect the fourth beam from the spatial light modulator 250 to the scanning unit 230. It can be understood that when the detection device 100 also includes a reflection unit 240, the beam splitter 220 and the reflection unit 240 allow the second beam emitted from the sensing module 110 and the incident fourth beam to share part of the optical path. The advantage of doing so is that the spatial layout of the sensing module 110 can be more reasonable and the volume smaller. It is also understood that in some other embodiments, the second beam emitted from the sensing module 110 and the incident fourth beam may be designed to no longer share a portion of the optical path, for example, by adding more transmission and reflection elements to the sensing module 110.

[0056] Figure 3 is a schematic diagram of an imaging module 120 provided in an embodiment of this application. As shown in Figure 3, the imaging module 120 includes an imaging lens 310 and an image sensor 320. The image sensor 320 is arranged on the imaging focal plane of the imaging lens 310. The imaging lens 310 is used to receive a fifth light beam from the target being detected and to generate a 2D color image of the target on the image sensor 320 based on the fifth light beam. The image sensor 320 is used to output the 2D color image of the target being detected.

[0057] In this application, the specific structure of the imaging lens 310 is not limited. For example, depending on the scenario in which the detection device is used, a fixed-focus lens, a zoom lens, a wide-angle lens, or a telephoto lens can be used.

[0058] Optionally, the image sensor 320 is a complementary metal-oxide-semiconductor (CMOS) image sensor, or a charge-coupled device (CCD) image sensor. It should be noted that the imaging module 120 generates a 2D color image using light reflected from the detected target; therefore, in this application, the image sensor 320 is a visible light image sensor.

[0059] Figure 4 is a schematic diagram of the structure of a processing module 130 provided in an embodiment of this application. As shown in Figure 4, the processing module 130 includes a pixel calibration and registration module 410, a grayscale image data and color image data acquisition module 420, and a report generation module 440. The pixel calibration and registration module 410 is used to match the physical size of pixels with the coordinates of the acquired 2D grayscale image data (from the perception module 110) and 2D color image data (from the imaging module 120), that is, to align the 2D grayscale image and 2D color image to the same coordinate system for subsequent image analysis and processing. The grayscale image data and color image data acquisition module 420 is used to receive 2D grayscale image data from the perception module 110 and 2D color image data from the imaging module 120. The report generation module 440 is used to generate alarms based on the calibrated 2D grayscale image data, the calibrated 2D color image data, and the 3D point cloud data, or, to generate alarms based on the calibrated 2D color image data and the 3D point cloud data.

[0060] It should be noted that this application does not limit the algorithm used in the pixel calibration and registration module 410. For example, it can be a feature-based registration algorithm, an image grayscale-based registration algorithm, a deep learning-based registration algorithm, etc.

[0061] In some embodiments, the pixel calibration and registration module 410 can be a pre-calibration module, meaning that the pixel calibration and registration module 410 has already completed pixel calibration and registration before leaving the factory. The pre-calibrated pixel calibration and registration module 410 is suitable for scenarios where the relative positions of the sensing module 110, imaging module 120, and processing module 130 are fixed. In this case, the detection device 100 composed of the sensing module 110, imaging module 120, and processing module 130 can be in the form of an integrated unit; for example, the sensing module 110, imaging module 120, and processing module 130 can be integrated into the same module. In other embodiments, the pixel calibration and registration module 410 can be a dynamic calibration module, meaning that the pixel calibration and registration module 410 completes pixel calibration and registration based on the positions of the sensing module 110, imaging module 120, and processing module 130 after installation. At this time, the detection device 100 composed of the sensing module 110, the imaging module 120 and the processing module 130 can be a split device. In this case, the sensing module 110, the imaging module 120 and the processing module 130 can be three independent modules, or the sensing module 110 and the imaging module 120 can be integrated into the same module. This application does not limit this.

[0062] To significantly enhance the visual effect of the image, making different gray levels correspond to different colors, thereby making it easier to identify and distinguish different regions and targets in the image, the processing module 130 optionally includes a pseudo-color image generation module 430. The pseudo-color image generation module 430 is used to convert the 2D grayscale image into a pseudo-color image. In this case, the report generation module 440 can generate an alarm based on the pseudo-color image data, the calibrated 2D color image data, and the 3D point cloud data. It should be noted that this application does not limit the algorithm used by the pseudo-color image generation module 430; for example, it can be a spatial domain method, a frequency domain method, a machine learning method, etc.

[0063] Furthermore, to improve detection accuracy, in some embodiments, the pseudo-color image generation module 430 is also used to output 2D grayscale image data and pseudo-color image data to the cloud computing platform. This allows the cloud computing platform to train a more accurate model based on the 2D grayscale image data and pseudo-color image data, thereby enabling the pseudo-color image generation module 430 to output a highly accurate color image based on the trained model. It is understood that when the pseudo-color image generation module 430 is also used to output 2D grayscale image data and pseudo-color image data to the cloud computing platform, the detection device 100 provided in this application also interacts with the cloud computing platform. It should be noted that this application does not limit the connection method between the detection device 100 and the cloud computing platform; it can be connected via a wired link or a wireless link, and can be a mechanical connection or an electrical connection. Furthermore, it can be a direct connection or an indirect connection through other network devices, controllers, etc. For example, when the detection device 100 is also electrically connected to the cloud computing platform, the processing module 130 sends a first electrical signal to the cloud computing platform, which carries 2D grayscale image data and pseudo-color image data. Meanwhile, the processing module 130 receives a second electrical signal from the cloud computing platform, which carries the training model.

[0064] It is understood that Figure 4 above is merely an exemplary illustration of the processing module 130 shown in Figure 1. That is, the processing module 130 may not be limited to the structure shown in Figure 4. For example, the processing module 130 in the detection device 100 may include only one processor, simultaneously performing pixel calibration and registration, grayscale image data and color image data acquisition, and report generation. It should also be noted that the one or more processors included in the processing module 130 are circuits with signal processing capabilities. In one implementation, the processor may be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU (which can be understood as a type of microprocessor), or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading configuration documents and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the aforementioned units. Furthermore, it can also refer to hardware circuits designed for artificial intelligence, which can be understood as a type of ASIC, such as neural network processing units (NPUs), tensor processing units (TPUs), and deep learning processing units (DPUs).

[0065] Figure 5 is a schematic flowchart of a detection method 500 provided in an embodiment of this application. Specifically, the method can be executed by the detection device 100 in Figure 1 above. The method includes the following steps.

[0066] S501 generates a 3D point cloud of the detected target.

[0067] Specifically, when the detection device 100 performs detection, the perception module 110 generates a 3D point cloud of the detection target and sends the 3D point cloud of the detection target to the processing module 130.

[0068] S502, Generate a 2D color image of the detected target.

[0069] Specifically, when the detection device 100 performs detection, the imaging module 120 generates a 2D color image of the target and sends the 2D color image of the target to the processing module 130.

[0070] S503 generates alarms based on 2D color images and 3D point clouds.

[0071] Specifically, the processing module 130 generates an alarm based on the 2D color image from the imaging module 120 and the 3D point cloud from the perception module 110.

[0072] Optionally, method 500 further includes the following S504.

[0073] S504 generates a 2D grayscale image of the detected target.

[0074] Specifically, when the detection device 100 performs detection, the perception module 110 is also used to generate a 2D grayscale image of the detected target and send the 2D grayscale image of the detected target to the processing module 130. It can be understood that when the method 500 further includes the following S504, the alarm is generated by 3D point cloud, 2D color image and 2D grayscale image.

[0075] It is understood that the specific process of the above-mentioned corresponding steps has been explained in detail in the execution of each module / unit shown in Figures 1 to 4 above, and will not be repeated here for the sake of brevity.

[0076] This application also provides a computer-readable storage medium storing computer instructions for implementing the method executed by the processing module 130.

[0077] For example, when the computer program is executed by the computer, it enables the computer to implement the method executed by the processing module 130 in the above embodiments.

[0078] Based on the above embodiments, this application also provides a computer-readable storage medium. This storage medium stores a software program, which, when read and executed by one or more processors, can implement the methods provided in the above embodiments. The computer-readable storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0079] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

[0080] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0082] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0084] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0085] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A detection device, characterized in that, include: The system comprises a sensing module, an imaging module, and a processing module, among which... The perception module is used to generate a three-dimensional 3D point cloud of the target to be detected and send the 3D point cloud to the processing module. The imaging module is used to generate a two-dimensional 2D color image of the detected target and send the 2D color image to the processing module; The processing module is used to generate an alarm based on the 2D color image and the 3D point cloud.

2. The detection device according to claim 1, characterized in that, The sensing module includes a light source, a beam splitter, a scanning unit, a spatial light modulator, and a detector, wherein... The light source is used to emit a first beam of light to the beam splitter; The beam splitter is used to transmit the first beam to the scanning unit and reflect the fourth beam from the scanning unit to the detector; The scanning unit is used to reflect the first beam reflection to the spatial light modulator and to reflect the fourth beam from the spatial light modulator to the beam splitter. The spatial light modulator is used to generate a second beam based on random speckle coding of the first beam and to emit the second beam, and to generate a fourth beam based on random speckle decoding of the third beam reflected from the detection target and to emit the fourth beam to the scanning unit. The detector is configured to receive the fourth beam from the beam splitter and generate the 3D point cloud of the detected target based on the fourth beam.

3. The detection device according to claim 2, characterized in that, The sensing module further includes a reflection unit located on the optical path between the scanning unit and the spatial light modulator. The reflection unit is used to reflect the first light beam from the scanning unit to the spatial light modulator, and to reflect the fourth light beam from the spatial light modulator to the scanning unit.

4. The detection device according to any one of claims 1 to 3, characterized in that, The perception module is also used to generate a 2D grayscale image of the detected target and send the 2D grayscale image to the processing module; The processing module is also used to generate the alarm based on the 2D grayscale image, the 3D point cloud, and the 2D color image.

5. The detection device according to claim 4, characterized in that, The processing module is also used to generate the alarm based on the 2D grayscale image, the 3D point cloud, and the 2D color image. The processing module is specifically used to generate a pseudo-color image based on the 2D grayscale image, and to generate the alarm based on the pseudo-color image, the 3D point cloud, and the 2D color image.

6. The detection device according to any one of claims 2 to 5, characterized in that, The detector is also used to generate a 2D grayscale image of the detected target based on the fourth beam.

7. The detection device according to any one of claims 2 to 6, characterized in that, The scanning unit is a microelectromechanical system (MEMS) micromirror array.

8. The detection device according to any one of claims 1 to 7, characterized in that, The imaging module is a visible light imaging module.

9. The detection device according to any one of claims 2 to 8, characterized in that, The light source is a laser light source.

10. The detection device according to any one of claims 1 to 9, characterized in that, The sensing module, the imaging module, and the processing module are integrated into one module.

11. A sensing module, characterized in that, The perception module is used to generate a 3D point cloud of the detected target, and the 3D point cloud is used to generate an alarm.

12. The sensing module according to claim 11, characterized in that, The sensing module includes a light source, a beam splitter, a scanning unit, a spatial light modulator, and a detector, wherein... The light source is used to emit a first beam of light to the beam splitter; The beam splitter is used to transmit the first beam to the scanning unit and reflect the fourth beam from the scanning unit to the detector; The scanning unit is used to reflect the first beam toward the spatial light modulator and to reflect the fourth beam from the spatial light modulator toward the beam splitter. The spatial light modulator is used to generate a second beam based on random speckle coding of the first beam and to emit the second beam, and to generate a fourth beam based on random speckle decoding of the third beam reflected from the detection target and to emit the fourth beam to the scanning unit. The detector is configured to receive the fourth beam from the beam splitter and generate the 3D point cloud of the detected target based on the fourth beam.

13. The sensing module according to claim 12, characterized in that, The sensing module further includes a reflection unit located on the optical path between the scanning unit and the spatial light modulator. The reflection unit is used to reflect the first light beam from the scanning unit to the spatial light modulator, and to reflect the fourth light beam from the spatial light modulator to the scanning unit.

14. The sensing module according to any one of claims 11 to 13, characterized in that, The perception module is also used to generate a 2D grayscale image of the detected target, and the 2D grayscale image and the 3D point cloud are used to generate the alarm.

15. The sensing module according to claim 14, characterized in that, When the 2D grayscale image and the 3D point cloud are used to generate the alarm... The 2D grayscale image is used to generate a pseudo-color image, and the pseudo-color image and the 3D point cloud are used to generate the alarm.

16. The sensing module according to any one of claims 12 to 15, characterized in that, The detector is also used to generate a 2D grayscale image of the detected target based on the fourth beam.

17. The sensing module according to any one of claims 12 to 16, characterized in that, The scanning unit is a microelectromechanical system (MEMS) micromirror array.

18. The sensing module according to any one of claims 12 to 17, characterized in that, The light source is a laser light source.

19. A detection method, characterized in that, include: Generate a 3D point cloud of the target object; Generate a two-dimensional 2D color image of the detected target; An alarm is generated based on the 2D color image and the 3D point cloud.

20. The method according to claim 19, characterized in that, The method further includes: Generate a 2D grayscale image of the detected target; The generation of alarms based on the 2D color image and the 3D point cloud includes: The alarm is generated based on the 2D grayscale image, the 3D point cloud, and the 2D color image.

21. The method according to claim 20, characterized in that, The method further includes: A pseudo-color image is generated based on the 2D grayscale image; The step of generating the alarm based on the 2D grayscale image, the 3D point cloud, and the 2D color image includes: The alarm is generated based on the pseudo-color image, the 3D point cloud, and the 2D color image.

Images