An image processing method and apparatus based on the Internet of Things

Abstract

This invention discloses an image processing method based on the Internet of Things (IoT), specifically relating to the field of IoT image technology, including the following steps: Step S1: Visible light image data and infrared thermal imaging data are simultaneously acquired by an IoT image acquisition terminal deployed in the target area, and the current ambient light intensity parameters are obtained; Step S2: In the edge computing module of the IoT image acquisition terminal, visible light and infrared images are acquired simultaneously, and a decision on whether to perform preliminary denoising is made based on real-time illumination detection, so that the preprocessing step forms a conditional response relationship with the environmental conditions; Subsequently, the fused image is adaptively compressed and uploaded to the cloud, where the cloud model analyzes the noise characteristics and generates targeted parameters to be sent back to the terminal, which then performs final denoising accordingly; In this process, the linkage between illumination judgment and denoising action avoids invalid calculations, and multimodal fusion provides complementary information for subsequent analysis.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) imaging technology, and more specifically, to an image processing method and apparatus based on the Internet of Things. Background Technology

[0002] In low-light or complex environments, visible light imaging is susceptible to noise interference and quality degradation, and single-modal images are insufficient to meet the detail and robustness requirements of subsequent analysis. Existing technologies typically employ fixed-parameter denoising methods or rely solely on local processing, failing to dynamically adjust processing strategies based on actual imaging conditions, resulting in limited denoising effectiveness or wasted computational resources. Especially when terminal devices have limited computing power, it is difficult to complete highly complex image enhancement tasks while ensuring real-time performance. Therefore, this paper proposes an image processing method and device based on the Internet of Things (IoT) to address these issues. Summary of the Invention

[0003] To overcome the aforementioned shortcomings of existing technologies, this method simultaneously acquires visible light and infrared images and determines whether to perform preliminary denoising based on real-time illumination detection, thus establishing a conditional response relationship between the preprocessing steps and environmental conditions. Subsequently, the fused image is adaptively compressed and uploaded to the cloud, where a cloud model analyzes noise characteristics and generates targeted parameters that are then sent back to the terminal, which performs final denoising accordingly. In this process, the linkage between illumination judgment and denoising avoids unnecessary calculations, and multimodal fusion provides complementary information for subsequent analysis. Furthermore, the three-level collaborative structure of "edge preprocessing—cloud parameter generation—terminal fine processing" allows the capabilities of highly complex models to be indirectly invoked by lightweight terminals.

[0004] To achieve the above objectives, the present invention provides the following technical solution: an image processing method based on the Internet of Things, comprising the following steps:

[0005] Step S1: Simultaneously acquire visible light image data and infrared thermal imaging data through IoT image acquisition terminals deployed in the target area, and obtain the current ambient light intensity parameters;

[0006] Step S2: In the edge computing module of the IoT image acquisition terminal, based on the ambient light intensity parameter, the visible light image data is initially denoised, and the denoised visible light image is spatiotemporally aligned and fused with the infrared thermal imaging data to generate a multimodal fused image.

[0007] Step S3: In the edge computing module, perform scene complexity analysis on the multimodal fusion image, dynamically select an image compression strategy based on the analysis results, and perform adaptive compression to generate a compressed image data packet; upload the compressed image data packet to the cloud processing platform.

[0008] Step S4: On the cloud processing platform, the compressed image data packet is decoded, and a pre-trained noise classification deep learning model is used to identify the main noise types in the image and generate corresponding refined denoising parameters.

[0009] Step S5: Send the refined denoising parameters back to the corresponding IoT image acquisition terminal. In the IoT image acquisition terminal, according to the received refined denoising parameters, perform secondary refined denoising processing on the original multimodal fusion image to generate the final target image, and store or forward the target image.

[0010] In a preferred embodiment: In step S1, synchronous acquisition is achieved by triggering the exposure operations of the visible light image sensor and the infrared thermal imaging sensor with the same clock source, so that the start times of the image frames of the two are aligned; the ambient light intensity parameter is output in real time by the digital ambient light sensor integrated into the IoT image acquisition terminal, and serves as the control basis for subsequent image processing.

[0011] In a preferred embodiment: In step S2, the preliminary denoising process selects different spatial domain filtering algorithms according to the ambient light intensity parameter: when the ambient light intensity is lower than the preset light threshold, a denoising algorithm based on non-local similarity metric is used; when the ambient light intensity is not lower than the preset light threshold, a Gaussian smoothing filtering algorithm is used; the spatiotemporal alignment fusion extracts corresponding feature points from the visible light image and the infrared thermal imaging data, calculates the geometric transformation matrix, and maps the infrared thermal imaging data to the coordinate system of the visible light image to achieve pixel-level alignment.

[0012] In a preferred embodiment: In step S3, the scene complexity analysis is based on the degree of texture change and the amplitude of inter-frame motion in the multimodal fusion image; if the degree of texture change is high and the amplitude of inter-frame motion is large, it is determined to be a dynamic complex scene, and compression is performed using a video compression standard based on inter-frame prediction; if the degree of texture change is low and the amplitude of inter-frame motion is small, it is determined to be a static simple scene, and compression is performed using an image compression standard based on single-frame coding; the compressed image data packet is uploaded to the cloud processing platform through the wireless communication module.

[0013] In a preferred embodiment: In step S4, the pre-trained noise classification deep learning model is a lightweight convolutional neural network, whose input is the decoded image region and whose output is a uniquely determined noise type identifier; the noise type identifier corresponds to one of Gaussian noise, salt-and-pepper noise, Poisson noise or motion blur; the refined denoising parameters include the denoising algorithm type, filter structure size and regularization intensity parameter that uniquely correspond to the noise type identifier.

[0014] In a preferred embodiment: In step S5, the secondary refined denoising process performs the corresponding denoising operation according to the denoising algorithm type specified in the received refined denoising parameters: when the denoising algorithm type is block matching-based 3D collaborative filtering, the basic estimation and final Wiener filtering stages are performed; when the denoising algorithm type is sorting statistics-based filtering, median filtering followed by morphological opening and closing operations are performed; when the denoising algorithm type is deconvolution-based restoration, the estimated point spread function combined with Wiener filtering is used for image restoration; the final target image is written to local non-volatile memory and sent to the remote monitoring terminal via a publish / subscribe message protocol.

[0015] In a preferred embodiment: applying any one of the Internet of Things-based image processing methods of claims 1-6, comprising: the method being executed by a system including an Internet of Things image acquisition terminal and a cloud processing platform, wherein the Internet of Things image acquisition terminal integrates a visible light image sensor, an infrared thermal imaging sensor and an edge computing module, and the cloud processing platform is configured with a deep learning model for generating refined denoising parameters.

[0016] The technical effects and advantages of this invention are as follows: By simultaneously acquiring visible light and infrared images and determining whether to perform preliminary denoising based on real-time illumination detection, the preprocessing step forms a conditional response relationship with environmental conditions; subsequently, the fused image is adaptively compressed and uploaded to the cloud, where the cloud model analyzes noise characteristics and generates targeted parameters that are sent back to the terminal, which then performs final denoising accordingly; in this process, the linkage between illumination judgment and denoising action avoids invalid calculations, multimodal fusion provides complementary information for subsequent analysis, and the three-level collaborative structure of "edge preprocessing - cloud parameter generation - terminal fine processing" allows the capabilities of highly complex models to be indirectly invoked by lightweight terminals;

[0017] Therefore, without increasing the continuous high load on the terminal, the noise suppression effect and detail preservation of the system output image are improved, and the amount of data transmission is reduced due to adaptive compression, thereby enhancing the overall processing efficiency and environmental adaptability. Attached Figure Description

[0018] Figure 1 This is a flowchart of the steps of the method of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] An example implementation will now be described more fully with reference to the accompanying drawings. An image processing method based on the Internet of Things includes the following steps: Step S1: Visible light image data and infrared thermal imaging data are simultaneously acquired by an Internet of Things image acquisition terminal deployed in the target area, and the current ambient light intensity parameter is obtained.

[0021] Step S2: In the edge computing module of the IoT image acquisition terminal, based on the ambient light intensity parameter, the visible light image data is initially denoised, and the denoised visible light image is spatiotemporally aligned and fused with the infrared thermal imaging data to generate a multimodal fused image.

[0022] Step S3: In the edge computing module, perform scene complexity analysis on the multimodal fusion image, dynamically select an image compression strategy based on the analysis results, and perform adaptive compression to generate a compressed image data packet; upload the compressed image data packet to the cloud processing platform.

[0023] Step S4: On the cloud processing platform, the compressed image data packet is decoded, and a pre-trained noise classification deep learning model is used to identify the main noise types in the image and generate corresponding refined denoising parameters.

[0024] Step S5: Send the refined denoising parameters back to the corresponding IoT image acquisition terminal. In the IoT image acquisition terminal, according to the received refined denoising parameters, perform secondary refined denoising processing on the original multimodal fusion image to generate the final target image, and store or forward the target image.

[0025] In step S1, synchronous acquisition is achieved by triggering the exposure operations of the visible light image sensor and the infrared thermal imaging sensor with the same clock source, aligning the start times of their image frames; the ambient light intensity parameter is output in real time by the digital ambient light sensor integrated into the IoT image acquisition terminal, and serves as the control basis for subsequent image processing; in step S1, when the ambient light intensity is lower than a preset threshold, the edge computing module uses the Non-Local Means (NLM) algorithm to perform preliminary denoising on the visible light image;

[0026] This algorithm is based on the prior knowledge of image self-similarity and performs a weighted average by searching similar blocks across the entire image. Its estimated value... The calculation formula is as follows:

[0027]

[0028] in, For pixels The original grayscale value, For global search window, weight Determined by the following formula:

[0029]

[0030] here, and Represented by pixels , Centered Local image patch vectors This is the filtering parameter (usually set to 1.5 times the noise standard deviation). The normalization factor is used to ensure that the weight sum is 1; this method can effectively suppress Gaussian noise under low illumination; while preserving texture and edge structure.

[0031] In step S2, the preliminary denoising process selects different spatial domain filtering algorithms based on the ambient light intensity parameter: when the ambient light intensity is lower than the preset light threshold, a denoising algorithm based on nonlocal similarity metric is used; when the ambient light intensity is not lower than the preset light threshold, a Gaussian smoothing filtering algorithm is used; the spatiotemporal alignment fusion extracts corresponding feature points from the visible light image and the infrared thermal imaging data, calculates the geometric transformation matrix, and maps the infrared thermal imaging data to the coordinate system of the visible light image to achieve pixel-level alignment.

[0032] In step S2, the fusion of the visible light image and the infrared thermal imaging image adopts a weighted average strategy based on saliency weights; let... These are the pixel values ​​of a visible light image. If the pixel values ​​are from the infrared image, then the fused image... The calculation is as follows:

[0033]

[0034] Among them, weight Determined by both local contrast and thermal saliency:

[0035]

[0036] The local contrast of the visible light image (calculated using the 8-neighbor standard deviation) is given. Thermal saliency of infrared images (extracted via Gaussian pyramid residuals) This is the balance factor (default value is 1.2). To prevent small constants from being divided by zero.

[0037] In step S3, scene complexity analysis is performed based on the degree of texture change and the amplitude of inter-frame motion in the multimodal fused image. If the degree of texture change is high and the amplitude of inter-frame motion is large, it is determined to be a dynamic complex scene, and compression is performed using a video compression standard based on inter-frame prediction. If the degree of texture change is low and the amplitude of inter-frame motion is small, it is determined to be a static simple scene, and compression is performed using an image compression standard based on single-frame coding. The compressed image data packet is uploaded to the cloud processing platform through the wireless communication module.

[0038] In step S3, the system dynamically adjusts the quantization parameters of H.264 encoding according to the scene complexity;

[0039] The rate-distortion optimization model is used to select the optimal coding mode, and its cost function is:

[0040]

[0041] in, The distortion metric is SATD (sum of absolute differences after Hadamard transform). For the estimated number of coded bits For Lagrange multipliers, their relationship with QP is as follows:

[0042]

[0043] When the scene complexity is high, the system reduces the QP value (e.g., from 32 to 26) to preserve details; conversely, it increases the QP and reduces the bitrate.

[0044] In step S4, the pre-trained noise classification deep learning model is a lightweight convolutional neural network. Its input is the decoded image region, and its output is a uniquely determined noise type identifier. The noise type identifier corresponds to one of Gaussian noise, salt-and-pepper noise, Poisson noise, or motion blur. The refined denoising parameters include the denoising algorithm type, filter structure size, and regularization intensity parameter that uniquely correspond to the noise type identifier.

[0045] In step S5, the secondary refined denoising process performs the corresponding denoising operation according to the denoising algorithm type specified in the received refined denoising parameters: when the denoising algorithm type is block matching-based 3D collaborative filtering, it performs two stages: basic estimation and final Wiener filtering; when the denoising algorithm type is sorting statistics-based filtering, it performs median filtering followed by morphological opening and closing operations; when the denoising algorithm type is deconvolution-based restoration, it uses the estimated point spread function combined with Wiener filtering for image restoration; the final target image is written to local non-volatile memory and sent to the remote monitoring terminal via a publish / subscribe message protocol.

[0046] In step S5, the terminal executes the BM3D (Block-Matching and 3DFiltering) algorithm based on the denoising parameters sent from the cloud.

[0047] The algorithm consists of two stages: Stage 1 (basic estimation):

[0048] For each reference block, search for the most similar one in the image. blocks ( ), forming a 3D matrix ;

[0049] right Perform 3D cooperative transformation to obtain coefficients ;

[0050] Applying hard thresholding for noise reduction: , Specified by cloud parameters;

[0051] The inverse transform is followed by a weighted average to generate the basic estimated image. ;

[0052] Phase Two (Final Estimate):

[0053] exist Match the repeating blocks to construct a new 3D group;

[0054] Coefficient shrinkage is performed using Wiener filtering:

[0055]

[0056] in The transformation coefficients of the basic estimate, The noise standard deviation is used as the final inverse transform fusion to obtain the target image.

[0057] The method is executed by a system including an IoT image acquisition terminal and a cloud processing platform. The IoT image acquisition terminal integrates a visible light image sensor, an infrared thermal imaging sensor and an edge computing module, and the cloud processing platform is configured with a deep learning model for generating refined denoising parameters.

[0058] The terminal integrates a visible light image sensor, an infrared thermal imaging sensor, an ambient light sensor, and an edge computing module, and has local preprocessing capabilities; the cloud platform deploys deep learning models and high-performance computing resources.

[0059] The two communicate with each other via the MQTT protocol for low latency. After the terminal completes the initial denoising, fusion and adaptive compression, it packages and uploads the compressed data along with metadata (including device number, timestamp, ambient light value, infrared temperature, scene complexity score, etc.). After the cloud parses the data, it calls the noise classification model to output noise type and intensity parameters and returns the results to the terminal in the form of a structured message. The terminal then performs fine denoising based on this and finally outputs a high-quality image that can be displayed or further analyzed.

[0060] This architecture achieves "end-to-cloud collaboration": the terminal is responsible for preprocessing with high real-time requirements, reducing the load on the cloud; the cloud provides intelligent decision-making capabilities to compensate for the terminal's insufficient computing power; the entire process significantly reduces communication overhead and system response latency while ensuring processing accuracy.

[0061] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image processing method based on the Internet of Things, characterized in that, Includes the following steps: Step S1: Simultaneously acquire visible light image data and infrared thermal imaging data through IoT image acquisition terminals deployed in the target area, and obtain the current ambient light intensity parameters; Step S2: In the edge computing module of the IoT image acquisition terminal, based on the ambient light intensity parameter, the visible light image data is initially denoised, and the denoised visible light image is spatiotemporally aligned and fused with the infrared thermal imaging data to generate a multimodal fused image. Step S3: In the edge computing module, perform scene complexity analysis on the multimodal fusion image, dynamically select an image compression strategy based on the analysis results, and perform adaptive compression to generate a compressed image data packet; upload the compressed image data packet to the cloud processing platform. Step S4: On the cloud processing platform, the compressed image data packet is decoded, and a pre-trained noise classification deep learning model is used to identify the main noise types in the image and generate corresponding refined denoising parameters. Step S5: Send the refined denoising parameters back to the corresponding IoT image acquisition terminal. In the IoT image acquisition terminal, according to the received refined denoising parameters, perform secondary refined denoising processing on the original multimodal fusion image to generate the final target image, and store or forward the target image.

2. The image processing method based on the Internet of Things according to claim 1, characterized in that, In step S1, synchronous acquisition is achieved by triggering the exposure operations of the visible light image sensor and the infrared thermal imaging sensor with the same clock source, so that the start times of the image frames of the two are aligned. The ambient light intensity parameter is output in real time by the digital ambient light sensor integrated into the IoT image acquisition terminal, and serves as the control basis for subsequent image processing.

3. The image processing method based on the Internet of Things according to claim 1, characterized in that, In step S2, the preliminary denoising process selects different spatial domain filtering algorithms based on the ambient light intensity parameter: when the ambient light intensity is lower than the preset light threshold, a denoising algorithm based on non-local similarity metric is used; when the ambient light intensity is not lower than the preset light threshold, a Gaussian smoothing filtering algorithm is used; the spatiotemporal alignment fusion extracts corresponding feature points from the visible light image and the infrared thermal imaging data, calculates the geometric transformation matrix, and maps the infrared thermal imaging data to the coordinate system of the visible light image to achieve pixel-level alignment.

4. An image processing method based on the Internet of Things according to claim 1, characterized in that, In step S3, scene complexity analysis is based on the degree of texture change and the amplitude of inter-frame motion in the multimodal fused image. If the degree of texture change is high and the amplitude of inter-frame motion is large, it is determined to be a dynamic complex scene and is compressed using a video compression standard based on inter-frame prediction. If the degree of texture change is low and the amplitude of inter-frame motion is small, it is determined to be a static simple scene and is compressed using an image compression standard based on single-frame coding. The compressed image data packet is uploaded to the cloud processing platform through the wireless communication module.

5. An image processing method based on the Internet of Things according to claim 1, characterized in that, In step S4, the pre-trained noise classification deep learning model is a lightweight convolutional neural network. Its input is the decoded image region, and its output is a uniquely determined noise type identifier. The noise type identifier corresponds to one of Gaussian noise, salt-and-pepper noise, Poisson noise, or motion blur. The refined denoising parameters include the denoising algorithm type, filter structure size, and regularization intensity parameter that uniquely correspond to the noise type identifier.

6. An image processing method based on the Internet of Things according to claim 1, characterized in that, In step S5, the secondary refined denoising process performs the corresponding denoising operation according to the denoising algorithm type specified in the received refined denoising parameters: when the denoising algorithm type is block matching-based 3D collaborative filtering, it performs two stages: basic estimation and final Wiener filtering; when the denoising algorithm type is sorting statistics-based filtering, it performs median filtering followed by morphological opening and closing operations; when the denoising algorithm type is deconvolution-based restoration, it uses the estimated point spread function combined with Wiener filtering for image restoration; the final target image is written to local non-volatile memory and sent to the remote monitoring terminal via a publish / subscribe message protocol.

7. An image processing system based on the Internet of Things (IoT), employing any one of the image processing methods based on the IoT according to claims 1-6, characterized in that, include: The method is executed by a system including an IoT image acquisition terminal and a cloud processing platform. The IoT image acquisition terminal integrates a visible light image sensor, an infrared thermal imaging sensor, and an edge computing module. The cloud processing platform is configured with a deep learning model for generating refined denoising parameters.

Images