Imaging equipment device for spatial layered image processing

Abstract

The invention discloses an imaging device and system for spatial hierarchical image processing, and belongs to the technical field of image processing and computer vision, and the device comprises an imaging module which is used for capturing a scene image of a scene; the distance detection module is used for acquiring depth information of the scene and generating a depth map; and the image processing module is in communication connection with the imaging module and the distance detection module, and is used for carrying out global image processing and layered image processing on the scene image, and fusing and outputting the images subjected to global image processing and layered image processing. A distance detection and layering processing mechanism is introduced, intelligent layering processing based on the physical distance is achieved through depth information, an image processing algorithm can be dynamically applied according to the actual distance of an object, the defects of fixed area processing when a camera moves or a scene changes are effectively overcome, and the processing accuracy is improved. And the accuracy of privacy protection and the expressive force of an artistic effect are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to an imaging device for spatial layering image processing. Background Technology

[0002] Traditional imaging devices, such as cameras and surveillance cameras, typically only perform global, uniform processing on the entire image, such as white balance, noise reduction, and compression. This approach cannot distinguish the spatial relationships between different objects in an image, resulting in significant limitations when processing complex scenes with depth variations. For example, in applications requiring background occlusion to protect privacy, traditional methods require pre-setting fixed image processing areas. Once the camera moves or the viewing angle changes, the original settings become invalid and must be manually reconfigured, lacking flexibility and adaptability. Furthermore, although some existing devices possess depth sensing capabilities (such as smartphones equipped with LiDAR), their primary use is limited to 3D modeling and ranging, and they do not fully integrate depth information into the image processing workflow, failing to achieve adaptive image optimization based on physical distance. Summary of the Invention

[0003] One of the objectives of this invention is to provide an imaging device for spatial layering image processing, in order to solve the problems that traditional imaging devices cannot adaptively optimize according to scene depth, that fixed processing areas fail when the device moves, and that existing depth sensing devices fail to fully utilize depth information to achieve intelligent layering processing.

[0004] This invention is achieved through the following technical solution: an imaging device for spatial layering image processing, comprising:

[0005] Imaging module, used to capture scene images of the scene;

[0006] A distance detection module is used to acquire depth information of the scene and generate a depth map;

[0007] The image processing module is communicatively connected to the imaging module and the distance detection module. It is used to perform global image processing and layered image processing on the scene image, and to fuse and output the images after global image processing and layered image processing. The layered processing divides the scene image into multiple spatial layers based on the depth map, and applies an image processing algorithm to each spatial layer.

[0008] Furthermore, the distance detection module includes a time-of-flight (TOF) sensor, which obtains distance information of each point in the scene by measuring the time of flight of the light signal, and thus obtains the depth map.

[0009] Furthermore, the distance detection module includes a binocular stereo vision system, which calculates a disparity map between images captured by two cameras and converts the disparity map into a depth map based on the principle of triangulation.

[0010] Furthermore, the step of dividing the scene image into multiple spatial levels based on the depth map and applying an image processing algorithm to each spatial level includes:

[0011] The depth map is parsed according to the layering rules to generate mask layers that identify different distance ranges;

[0012] The scene image is combined with the mask layer, and image processing algorithms are applied to the image regions identified by each mask layer.

[0013] Furthermore, the layering rules include discrete layering or continuous layering; wherein, the discrete layering divides the image layers by a fixed distance threshold, and the continuous layering divides the image layers by a function relationship in which processing parameters change continuously with distance.

[0014] Furthermore, the image processing module is also used to perform coordinate transformation and resampling on the depth map to align the depth map with the scene image before performing layered image processing on the scene image.

[0015] Furthermore, the image processing module performs global image processing and layered image processing on the scene image in any of the following modes: layered image processing is performed first, followed by global image processing; global image processing is performed first, followed by layered image processing; or global image processing and layered image processing are performed alternately.

[0016] Furthermore, at least one of the layering rules, the order of global image processing and layered image processing, and the processing algorithms applied to each layer can be configured by the user or adaptively adjusted by the image processing module according to the scene.

[0017] Furthermore, the imaging module and the distance detection module are triggered by a hardware synchronization signal to capture the scene image and the depth information at the same time.

[0018] Furthermore, the image processing algorithm includes: blurring, mosaicking, sharpening, image enhancement, brightness adjustment, color optimization, and noise reduction.

[0019] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0020] This invention introduces a distance detection and layered processing mechanism, utilizing depth information to achieve intelligent layered processing based on physical distance. It dynamically applies image processing algorithms according to the actual distance of objects, effectively overcoming the shortcomings of fixed-area processing when the camera moves or the scene changes. This significantly improves the accuracy of privacy protection and the expressiveness of artistic effects. Furthermore, this invention provides a highly flexible and configurable architecture, allowing users to customize layering rules, processing algorithms, and execution order according to specific scenarios. This enables the device to be widely applied in various fields such as security monitoring, creative photography, and industrial vision, demonstrating promising industrialization prospects and practical value. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0022] Figure 1 A flowchart of an imaging device for spatial layering image processing provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Example 1

[0025] Traditional imaging equipment faces long-standing limitations in image processing, the most prominent being its singular processing dimension. Whether it's a home video camera, surveillance camera, or professional camera, they generally only perform uniform global optimization on the entire image, such as adjusting overall brightness, contrast, or implementing noise reduction. This approach completely ignores the spatial relationships between different objects in the image, making it impossible to differentiate between foreground subjects and background environments. Furthermore, when privacy needs to be protected by masking specific areas, traditional technology relies on pre-defined fixed image regions. Once the camera position moves or the shooting angle changes, these settings immediately become invalid, requiring manual reconfiguration. This not only increases operating costs but also limits the device's level of intelligence.

[0026] To address the aforementioned problems, this invention proposes an imaging device for spatial layering image processing, comprising the following three modules:

[0027] The imaging module, based on mature CMOS, CCD, or thermal imaging image sensor technology, captures clear, high-quality scene visual information by precisely converting incident optical or non-visible light signals into digital image data. In this invention, the imaging module's function goes beyond the simple acquisition of traditional cameras. It needs to maintain strict synchronization and cooperation with other modules in the system, and its field of view must be highly consistent with the distance detection module, thus laying the foundation for subsequent spatial alignment processing. Through precise timing control, the imaging module ensures that exposure and image reading are completed at specific points in time, thereby providing a reliable two-dimensional image source for the entire system.

[0028] The distance detection module identifies the distance relationships between objects in a scene, providing accurate depth information to traditional 2D images. This module uses a time-of-flight (TOF) sensor or a dual-camera stereo vision solution to acquire the scene's depth information.

[0029] The Time-of-Flight (TOF) sensor is implemented through an integrated sensor module, including direct time-of-flight (directly measuring the time difference between light emission and reflection) and indirect time-of-flight (calculating distance by detecting the phase difference between reflected and emitted light). This embodiment uses direct time-of-flight. It actively emits modulated infrared light pulses into the scene and accurately measures the time it takes for the light pulses to travel from emission to reflection by the object and back to the sensor. Since the speed of light is constant, the absolute distance between the object and the camera can be directly calculated by calculating the time difference, generating a dense depth point cloud. Its ranging range is typically between 0.1 meters and 10 meters, with centimeter-level accuracy. It is also less affected by ambient light, has a fast response speed, and is suitable for real-time applications.

[0030] Dual-camera stereo vision uses two cameras positioned at a horizontal distance (called the baseline, typically 2-5 cm) to simultaneously capture two grayscale images with slightly different viewpoints. Then, a stereo matching algorithm (such as SGBM) is used to find the corresponding positions of the same feature points in these two images and calculate their horizontal displacement (i.e., disparity). Finally, based on the principles of triangulation, using the known focal length and baseline length, the disparity map is converted into a depth map. The advantage of the binocular approach is its relatively low cost and good performance in mid-to-long-range ranging.

[0031] Regardless of the approach used, this module can generate a depth map that reflects the three-dimensional structure of the scene, laying the foundation for subsequent processing.

[0032] The image processing module introduces layered image processing on top of traditional global image processing (such as white balance, noise reduction, and compression). Layered image processing first utilizes a precisely aligned depth map provided by the distance detection module. Based on user-preset rules (e.g., 0-1 meters foreground, 1-5 meters midground, and beyond 5 meters background), it divides the image into different spatial layers and generates multiple binary mask layers. Each mask layer clearly defines all pixel regions within a specific distance range. These mask layers are then applied to the color image, and specific image processing algorithms are executed independently and in parallel on different layers. These algorithms can include blurring, mosaicking, sharpening, image enhancement, brightness adjustment, color optimization, and noise reduction. This processing offers high flexibility; the processing order (layered before global, global before layered, or cross-processing) and algorithm selection can be dynamically configured by the user according to the specific scenario (e.g., portrait photography, security monitoring). Finally, the processed layers are merged into a complete image. Through this series of image processing steps, the image processing module transforms depth data into personalized image effects with practical application value, solving the problem that traditional devices have a single processing dimension and cannot adapt to scene changes.

[0033] Example 2

[0034] The invention will be further described below through a specific workflow, wherein the distance detection module uses a time-of-flight (TOF) sensor.

[0035] Before commencing operation, the system needs to complete preparatory work to ensure that all modules are in a coordinated state. First, parameters are loaded. The main processor loads user-preset calibration parameters (including camera intrinsic and extrinsic parameters, lens distortion coefficients, etc.) and processing rules (such as layering rules, processing order, and processing algorithms) from memory. Then, the main processor generates a unified clock reference and frame synchronization signal. This synchronization signal is directly transmitted to the imaging and distance detection modules via hardware lines (such as GPIO), providing a precise time starting point for the parallel acquisition of the imaging and distance detection modules, ensuring global synchronization accuracy at the microsecond level.

[0036] Next, the main processor sends a hardware trigger pulse, which the sensors in the imaging module and distance detection module receive and begin working, capturing visual and spatial information of the scene simultaneously. Specifically, the CMOS / CCD image sensor in the imaging module completes exposure, converts the light signal into an electrical signal, and after analog-to-digital conversion, outputs a raw image frame. The TOF sensor in the distance detection module includes an infrared transmitter and a receiver. The transmitter synchronously emits modulated light waves, and the receiver receives the light reflected from the object. Each pixel calculates the absolute distance value of the point by calculating the time difference between the emitted and received waves, forming a low-resolution raw depth data array.

[0037] The original image and original depth data array are then processed by the image processing module, including global processing and layered processing. In this embodiment, the processing order is a cross-processing approach of global processing and layered processing: global processing first, then layered processing, and finally global processing again. First, the original depth data is filtered (e.g., median filtering) to eliminate noise and temperature and ambient light compensation is performed to obtain an optimized depth map. Simultaneously, the original image undergoes global processing (denoising, linearization, etc.) to obtain a high-quality image. Since the imaging module and distance detection module are physically separate, the data they capture differ in viewpoint. The image processing module, by calling pre-loaded calibration parameters and using coordinate transformation and resampling algorithms, maps the depth map to the coordinate system of the high-quality image, obtaining an aligned depth map with the same resolution and pixel-to-pixel correspondence as the high-quality image. At this point, any pixel in the image (e.g., a neighbor's window) can find its precise distance value (e.g., 8.5 meters) in the aligned depth map. Then, based on the aligned depth map, the following processing is performed:

[0038] First, based on the layering rules set by the user (e.g., 0-1m foreground, 1-5m for midground, >5m for background), each pixel in the depth map is classified to generate multiple binary mask layers, where each mask identifies all pixel regions within a specific distance range.

[0039] Next, the image is combined with these mask layers, and user-predefined processing algorithms are applied to each layer in parallel. For example, sharpening and brightness enhancement algorithms are applied to the foreground mask area to highlight the subject details; Gaussian blur or mosaic algorithms are applied to the background mask area to achieve background blurring or privacy masking; and the mid-ground mask area is left as is or undergoes color optimization.

[0040] The processed layers are then merged into a single image. Since the masks are precise and mutually exclusive, the merging process ensures natural boundaries.

[0041] Finally, the merged image undergoes final global processing, such as gamma correction and contrast adjustment, to optimize the overall image quality.

[0042] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An imaging device for spatial layered image processing, characterized in that, include: Imaging module, used to capture scene images of the scene; A distance detection module is used to acquire depth information of the scene and generate a depth map; The image processing module is communicatively connected to the imaging module and the distance detection module. It is used to perform global image processing and layered image processing on the scene image, and to fuse and output the images after global image processing and layered image processing. The layered processing divides the scene image into multiple spatial layers based on the depth map, and applies an image processing algorithm to each spatial layer.

2. The imaging device for spatial layering image processing according to claim 1, characterized in that, The distance detection module includes a time-of-flight (TOF) sensor, which obtains distance information of each point in the scene by measuring the time of flight of the light signal, and thus obtains the depth map.

3. The imaging device for spatial layering image processing according to claim 1, characterized in that, The distance detection module includes a binocular stereo vision system, which calculates the disparity map between images captured by two cameras and converts the disparity map into the depth map based on the principle of triangulation.

4. The imaging device for spatial layering image processing according to claim 1, characterized in that, The step of dividing the scene image into multiple spatial levels based on the depth map and applying an image processing algorithm to each spatial level includes: The depth map is parsed according to the layering rules to generate mask layers that identify different distance ranges; The scene image is combined with the mask layer, and image processing algorithms are applied to the image regions identified by each mask layer.

5. The imaging device for spatial layering image processing according to claim 4, characterized in that, The layering rules include discrete layering or continuous layering; wherein, the discrete layering divides the image layers by a fixed distance threshold, and the continuous layering divides the image layers by a function relationship in which the processing parameters change continuously with distance.

6. The imaging device for spatial layering image processing according to claim 1, characterized in that, The image processing module is further configured to perform coordinate transformation and resampling on the depth map to align the depth map with the scene image before performing layered image processing on the scene image.

7. The imaging device for spatial layering image processing according to claim 1, characterized in that, The image processing module performs global image processing and layered image processing on the scene image in any of the following modes: layered image processing is performed first, followed by global image processing; global image processing is performed first, followed by layered image processing; or global image processing and layered image processing are performed alternately.

8. An imaging device for spatial layering image processing according to claim 5 or 7, characterized in that, The hierarchical rules, the order of global image processing and hierarchical image processing, and at least one of the processing algorithms applied to each layer can be configured by the user or adaptively adjusted by the image processing module according to the scene.

9. The imaging device for spatial layering image processing according to claim 1, characterized in that, The imaging module and the distance detection module are triggered by a hardware synchronization signal to capture the scene image and the depth information at the same time.

10. An imaging device for spatial layering image processing according to claim 1, characterized in that, The image processing algorithms include: blurring, mosaicking, sharpening, image enhancement, brightness adjustment, color optimization, and noise reduction.

Images