An image processing system based on the Internet of Things

Abstract

The invention discloses an image processing system based on the Internet of Things, which relates to the field of image processing technology and comprises a multispectral sensor integration module, an image feature extraction module, an edge computing module, a cloud computing guidance module, a dynamic task scheduling module and an edge-cloud feedback closed-loop module; the multispectral sensor integration module collects image data through the collaborative work of visible light and infrared light sensors; the image feature extraction module is used to extract valuable features from the image data; the edge computing module is used to realize adaptive compression; the cloud computing guidance module classifies the noise type in the image through a noise classification network; the dynamic task scheduling module comprises a high-performance mode, a basic mode and an emergency power-saving mode; the edge-cloud feedback closed-loop module regularly uploads the image quality index after noise reduction to the cloud; the invention effectively solves the problems of image noise and image quality degradation in a low-light environment while maintaining high computing efficiency and real-time performance.

Description

Technical Field

[0001] The present invention relates to the technical field related to image processing, and in particular to an image processing system based on the Internet of Things. Background Art

[0002] With the continuous development of Internet of Things (IoT) technology, more and more IoT devices (such as night surveillance cameras, environmental monitoring equipment, etc.) are widely used in various scenarios. These devices collect image and video data through sensors for further analysis and processing.

[0003] In low-light environments, the image data collected by IoT devices (such as nighttime surveillance cameras) are often severely affected by noise, such as Gaussian noise and color distortion. These noises not only affect the clarity and usability of the image, but may also cause image quality degradation, making it impossible for IoT devices to provide high-quality image data when working at night.

[0004] Traditional noise reduction algorithms are usually processed directly at the edge, but this approach can easily lead to loss of details, especially blurring of edges and textures. If processed through the cloud, although it can provide more powerful computing power, due to the existence of transmission delays, real-time optimization cannot be achieved, thus affecting the efficiency and accuracy of the system. Summary of the invention

[0005] In order to solve the defects of the prior art, the present invention provides an image processing system based on the Internet of Things.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: The present invention provides an image processing system based on the Internet of Things, comprising a multispectral sensor integration module, an image feature extraction module, an edge computing module, a cloud computing guidance module, a dynamic task scheduling module, and an edge-cloud feedback closed-loop module; The multi-spectral sensor integrated module is used to collect image data in a low-light environment through the collaborative work of a visible light sensor and an infrared light sensor; The image feature extraction module is used to extract valuable features from the image data collected by the multispectral sensor; The edge computing module is used to achieve adaptive compression and inference optimization, and optimize the utilization of computing resources by switching the model structure when the network load is high; The cloud computing guidance module classifies the noise types in the image through a noise classification network, generates optimized noise reduction kernel functions according to different types of noise, and pushes these parameters to the edge end, thereby dynamically optimizing the noise reduction process during system operation; The dynamic task scheduling module includes a high-performance mode, a basic mode and an emergency power-saving mode. The dynamic task scheduling module realizes switching between the high-performance mode, the basic mode or the emergency power-saving mode according to the ambient light intensity, the device power and the network delay factors; The edge-cloud feedback closed-loop module regularly uploads the denoised image quality indicators to the cloud, and pushes the new model to the edge through differential updates, thereby continuously optimizing the performance of the system.

[0007] As a preferred technical solution of the present invention, the image feature extraction module includes a high-frequency detail enhancement network, a semantic segmentation network and a cross-modal attention unit; The high-frequency detail enhancement network is used to enhance the thermal radiation features captured by the infrared sensor to highlight the details in a low-light environment; The semantic segmentation network is used to retain the color and texture information captured by the visible light sensor and enhance the semantic representation of the scene; The cross-modal attention unit is used to dynamically allocate weights between sensor channels and automatically adjust the fusion ratio of infrared and visible light data according to the ambient light intensity to ensure that details are not lost in low-light conditions while retaining the color and texture information of bright areas.

[0008] As a preferred technical solution of the present invention, the fusion formula of the cross-modal attention unit is: ; Among them, F fused represents the fused image features, F IR is the image feature of the infrared channel, F VIS is the image feature of the visible light channel, and α is the fusion weight, which is dynamically adjusted according to the ambient light intensity.

[0009] As a preferred technical solution of the present invention, the calculation formula of the fusion weight α in the cross-modal attention unit is: ; Among them, I avg is the average light intensity of the scene.

[0010] As a preferred technical solution of the present invention, the edge computing module reduces unnecessary calculations through a layer pruning strategy and only retains channels that are sensitive to infrared features, with a retention rate greater than 85%.

[0011] As a preferred technical solution of the present invention, the formula of the denoising kernel function is as follows: ; Among them, N filter Represented as the filter kernel function after denoising, used for noise suppression in images, Nbase It is represented by the basic denoising kernel function, which is used to deal with general noise conditions. β is represented by the dynamic adjustment weight according to the noise type and ambient light intensity. β is determined by the noise classification network and adjusted in the dynamic autoencoder using the following formula: .

[0012] As a preferred technical solution of the present invention, the noise classification network is based on the ResNet-18 architecture, the input is the frequency domain features of the low-light image, and the output is the noise type label; The denoising kernel function adjusts the size of the kernel function according to different noise types.

[0013] As a preferred technical solution of the present invention, when the light intensity is low, dual-channel full-resolution fusion is enabled, thereby entering a high-performance mode; When the battery is low or the light intensity is high, only the key areas of the infrared channel are processed to reduce the computing burden and enter the basic mode; When the network delay is too high, the local cache noise reduction parameters are used and the cloud backhaul link is closed to save bandwidth and battery consumption, thus entering the emergency power reduction mode.

[0014] As a preferred technical solution of the present invention, the edge-cloud feedback closed-loop module uploads the denoised image quality index to the cloud every 10 seconds, and triggers a cloud retraining operation when the index is lower than a set threshold.

[0015] As a preferred technical solution of the present invention, the cloud computing guidance module includes a noise classification network and a dynamic autoencoder; The noise classification network identifies the noise type in the image through a pre-trained model and generates corresponding noise reduction parameters; The dynamic autoencoder is used to generate an optimized denoising kernel function according to the noise type.

[0016] The beneficial effects of the present invention are: 1. In the present invention, the combined information of visible light and infrared light is captured by the multi-spectral sensor integration module. Combined with the adaptive compression and inference optimization strategy of the edge computing module, the image data can be quickly processed at the edge, reducing the computing burden on the cloud. In addition, the cloud computing module uses noise classification and dynamic optimization strategies to accurately classify and reduce the noise in the image, thereby further improving the image quality.

[0017] 2. In the present invention, through the continuous optimization of the edge-cloud feedback closed-loop module, the system can obtain the noise reduction effect in real time, and adjust the optimization strategy according to the image quality index, so that the system can automatically adapt to the dynamically changing environment, ensuring that high-quality image processing effects can be provided under various conditions. On the whole, the system can effectively solve the problems of image noise and image quality degradation in low-light environments, while maintaining high computing efficiency and real-time performance, and is suitable for various practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 FIG. 4 is a flow chart of the image processing system of the present invention. DETAILED DESCRIPTION

[0019] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0020] Embodiment 1: Figure 1 As shown, an image processing system based on the Internet of Things includes a multispectral sensor integration module, an image feature extraction module, an edge computing module, a cloud computing guidance module, a dynamic task scheduling module, and an edge-cloud feedback closed-loop module; The multi-spectral sensor integrated module is used to collect image data in a low-light environment through the collaborative work of a visible light sensor and an infrared light sensor; the main function of this module is to collect image data. In a low-light environment, a single visible light sensor may not be able to provide sufficient information, while an infrared sensor can supplement the deficiency of a visible light sensor. Therefore, this module cooperates with a visible light sensor (wavelength range is 400-700nm) and an infrared sensor (800-1400nm). The two sensors realize spatiotemporal consistency data collection (time error <1ms) through a synchronous control module (FPGA frequency division trigger), avoid fusion dislocation in dynamic scenes, and comprehensively utilize the advantages of both to realize efficient collection of image data in a low-light environment. In this way, the quality of images under low-light conditions can be improved and more scene information can be provided.

[0021] The image feature extraction module is used to extract valuable features from the image data collected by the multispectral sensor; the image feature extraction module is mainly used to extract valuable features from the image data collected by the multispectral sensor, and these features include but are not limited to color, texture, edge information, etc. In order to ensure that the extracted features have high availability, the module combines a high-frequency detail enhancement network, a semantic segmentation network and a cross-modal attention unit.

[0022] The edge computing module is used to achieve adaptive compression and inference optimization, and optimizes the utilization of computing resources by switching the model structure when the network load is high; the edge computing module is one of the core modules in this system, and is mainly used for real-time computing processing locally. The module dynamically adjusts the use of computing resources through adaptive compression and inference optimization, especially when the network load is high, by switching the model structure to reduce the amount of calculation. In order to improve efficiency, the edge computing module also combines a layer pruning strategy to reduce unnecessary calculations and only retains channels that are sensitive to infrared features to ensure that the retention rate is greater than 85%.

[0023] The cloud computing guidance module classifies the noise types in the image through a noise classification network, generates optimized noise reduction kernel functions according to different types of noise, and pushes these parameters to the edge, thereby dynamically optimizing the noise reduction process during system operation; the main function of the cloud computing guidance module is to generate optimized noise reduction kernel functions by classifying the image noise types, and push these parameters to the edge. This process involves a noise classification network and a dynamic autoencoder.

[0024] The dynamic task scheduling module includes a high-performance mode, a basic mode and an emergency power reduction mode. The dynamic task scheduling module realizes switching between high-performance mode, basic mode or emergency power reduction mode according to ambient light intensity, device power and network delay factors. The main task of this module is to dynamically adjust the system's working mode according to environmental changes.

[0025] The edge-cloud feedback closed-loop module regularly uploads the denoised image quality index to the cloud, and pushes the new model to the edge through differential update, thereby continuously optimizing the system performance. This module is mainly used to realize feedback and closed-loop optimization between the edge and the cloud. The system will regularly upload the denoised image quality index to the cloud. When the index is lower than the set threshold, the cloud will trigger a retraining operation. In this way, the system can continuously optimize the noise reduction effect and image quality.

[0026] Among them, the present invention realizes efficient image processing in low-light environments through a multi-level processing strategy combined with the advantages of edge computing and cloud computing. Specifically, the system of the present invention solves the image noise problem in low-light conditions and optimizes the image quality through the collaborative work of multiple modules such as a multispectral sensor integration module, an image feature extraction module, an edge computing module, a cloud computing guidance module, a dynamic task scheduling module, and an edge-cloud feedback closed-loop module. The efficiency and flexibility of the system give it broad prospects in practical applications, especially in the fields of night monitoring, security, and intelligent transportation.

[0027] Furthermore, the image feature extraction module includes a high-frequency detail enhancement network, a semantic segmentation network, and a cross-modal attention unit; The high-frequency detail enhancement network is used to enhance the thermal radiation features captured by the infrared sensor (such as the temperature difference at the edge of a metal scratch) to highlight the details in a low-light environment. In a low-light environment, the information provided by the infrared sensor mainly includes the thermal radiation features of the object. Although it can help identify the outline and position of the object, its texture and details are usually not as rich as those of a visible light image. By performing high-frequency detail enhancement processing on the infrared image, the system can highlight its detail information, making the infrared image clearer and helping to identify subtle changes in the scene; The semantic segmentation network is used to retain the color and texture information captured by the visible light sensor and enhance the semantic expression of the scene. In image processing, semantic segmentation is an important technology that can segment an image into different semantic regions to help the system understand the meaning of different parts (such as roads, buildings, people, vehicles, etc.). For visible light images, semantic segmentation can not only retain details, but also effectively enhance the semantic information of the image, especially in the salient areas of the image; The cross-modal attention unit is used to dynamically allocate weights between sensor channels (e.g., infrared sensors in dark areas are prioritized, while visible light sensors in bright areas are prioritized), and automatically adjust the fusion ratio of infrared and visible light data according to the ambient light intensity to ensure that details in low-light conditions are not lost while retaining the color and texture information in bright areas. It can automatically adjust the fusion ratio of infrared and visible light data according to the ambient light intensity (I avg ) changes, automatically adjust the weight distribution of infrared images and visible light images in the fusion process to optimize the overall performance of the image; During the fusion process, the cross-modal attention unit takes into account the strengths and weaknesses of each sensor. For example, at night or in low-light environments, infrared sensors can provide higher object detection capabilities, while visible light sensors can provide richer color and texture details in bright areas. The cross-modal attention unit takes into account the ambient lighting conditions (I avg ) perception, automatically adjusting the weights of infrared and visible light, so that the system can respond flexibly according to specific scenarios and achieve the best fusion effect; For example, in low-light environments, infrared images are given higher weights to highlight details and object outlines; in high-light environments, the weight of visible light images is increased to ensure that the color and texture of the scene are not lost.

[0028] This multi-level image processing technology improves the performance of the system in complex environments, enabling imaging devices in the Internet of Things to adapt to different lighting conditions and provide more accurate and detailed visual information. It is widely used in smart security, autonomous driving, smart monitoring and other fields.

[0029] Furthermore, the fusion formula of the cross-modal attention unit is: ; Among them, F fused represents the fused image feature, which is a weighted combination of the infrared image and the visible light image. IR is the image feature of the infrared channel, F VIS is the image feature of the visible light channel, α is the fusion weight, which is dynamically adjusted according to the ambient light intensity and determines the proportion of infrared image and visible light image in the fusion; The calculation formula of the fusion weight α in the cross-modal attention unit is: ; Among them, I avg is the average light intensity of the scene, which means that when the light is strong, α will become smaller, and when the light is weak, α will increase; I avg It is an important measure of the ambient lighting conditions in the image, usually obtained by statistically calculating the brightness of the image area. This value directly reflects the changes in lighting in the environment, which in turn affects the weighting of different sensor data (such as visible light and infrared) during image fusion; In low-light environments, when the light intensity is low, log(I avg ) will be smaller, which increases the value of α. At this time, the features of the infrared image will be given a greater weight because the infrared sensor can provide more details and object contours in low-light environments, while the weight of the visible light image (1-α) will be reduced accordingly because the visible light sensor performs poorly under such conditions and cannot provide clear image information; In a high light environment, when the light intensity is strong, log(I avg ) is larger, which makes the value of α decrease. At this time, the weight of the visible light image (1-α) increases, because the visible light image can provide richer color and texture information, and the weight of the infrared image decreases, because in a strong light environment, the detail information of the infrared image is usually limited and will not significantly improve the image quality.

[0030] By dynamically adjusting the fusion weights, the cross-modal attention unit can adaptively select appropriate sensor data for fusion according to different lighting conditions to obtain the best imaging effect. This adaptive fusion mechanism makes the image processing system more adaptable and can provide clear, detailed and semantically rich images in different environments.

[0031] Here are some practical application scenarios: Intelligent monitoring system: In the dark night or poorly lit environment, the system can ensure the monitoring effect by enhancing the details of the infrared image.

[0032] Autonomous driving: In low light or extreme weather conditions, infrared images can provide outline information of objects, while in daytime or well-lit conditions, the colors and details provided by visible light images will be more helpful for vehicle navigation and identification.

[0033] Intelligent security: Under complex lighting conditions, the system can adaptively adjust the image fusion strategy to improve the ability to identify intruders, abnormal behaviors, etc.

[0034] Furthermore, the model structure is switched in real time according to the network load. When bandwidth resources are limited, the edge computing module reduces unnecessary computation through a layer pruning strategy and only retains channels that are sensitive to infrared features. The retention rate is greater than 85%, and the channels that are sensitive to infrared features can be retained through the gradient threshold method.

[0035] Layer pruning refers to removing some redundant neurons, channels or layers in a neural network to reduce computing resource consumption and speed up the reasoning process. The specific implementation is: Channel pruning: Determine which channels can be removed based on their contribution to the final output. After pruning, only those channels that have a greater impact on the target task are retained. This method not only reduces the amount of computation, but also saves storage space. Layer pruning: Selectively remove certain layers based on their contribution or importance to reduce the size of the model; generally speaking, pruning these layers does not have a significant impact on the final accuracy, especially when more than 85% model retention is preserved.

[0036] Through layer pruning and channel pruning, the system can dynamically select channels that are sensitive to infrared features to retain, ensuring that the key features of the infrared image are retained. By analyzing the importance of the features of the infrared image, the system can evaluate which channels and layers are more important for the extraction of infrared features, thereby retaining the parts that are sensitive to infrared information during pruning. The system sets the pruning retention rate to be greater than 85%, ensuring that important infrared features are not lost during the pruning process, while greatly improving the reasoning speed and efficiency.

[0037] Through the combination of the above technical means, the edge computing module can maintain good performance in a high-load, resource-constrained environment. The dynamic combination of adaptive compression, inference optimization, model structure switching and layer pruning strategies enables the system to effectively utilize computing resources in different scenarios and achieve optimal performance.

[0038] Furthermore, the cloud computing guidance module includes a noise classification network and a dynamic autoencoder, which can solve the contradiction between edge computing power fluctuation and environmental adaptability, and optimize the global processing efficiency through cloud model pre-training and adaptive parameter push; The noise classification network identifies the noise type in the image through the pre-trained model and generates corresponding noise reduction parameters; the noise classification network is a model based on deep learning, which is responsible for analyzing the noise type in the input image. Through the pre-trained model, the network can learn to identify various noise patterns in the image, such as Gaussian noise, salt and pepper noise, image compression noise, etc. Specifically, the workflow of the noise classification network includes: Pre-training stage: Use a large amount of image data with annotated noise types to train the model to identify different noise characteristics. During the training process, the model gradually learns the laws and manifestations of noise by extracting local and global features of the image; Noise identification: When an image to be processed is input, the noise classification network will analyze the image and output the corresponding noise type label. For example, the system may identify the presence of Gaussian noise or compression artifacts in the image. Generate denoising parameters: Based on the noise type, the network generates appropriate denoising parameters, ready to provide guidance for subsequent denoising operations.

[0039] The dynamic autoencoder is used to generate an optimized denoising kernel function according to the noise type.

[0040] The dynamic autoencoder generates an optimized denoising kernel function (i.e., denoising filter) according to the noise type. The autoencoder is a neural network architecture, usually composed of an encoder and a decoder. It can compress the input data (here, the noise image) into a low-dimensional representation and then reconstruct the input data through the decoder. In the task of noise reduction, the workflow of the dynamic autoencoder is as follows: Noise type input: The dynamic autoencoder receives the noise type information from the noise classification network; Generate kernel function: Based on different noise types, the autoencoder generates an optimized denoising kernel function through the trained network parameters, i.e. a filter that is used to suppress the identified noise type in the image; Adaptive adjustment: The autoencoder can dynamically adjust the parameters of the kernel function according to the noise characteristics of the input image to ensure the best noise suppression effect in different environments.

[0041] Among them, the cloud computing guidance module provides important intelligent support in the image processing system, helping the system to dynamically identify the noise type and optimize the noise reduction process through adaptive algorithms.

[0042] Furthermore, the design of the denoising kernel function is one of the key steps in the image processing system, which directly affects the noise suppression effect of the system. The formula of the denoising kernel function is as follows: ; Among them, N filter Represented as the filter kernel function after denoising, used for noise suppression in images, N base It is represented as the basic denoising kernel function, which is used to deal with general noise conditions. It may be a conventional filter. β represents the dynamic adjustment weight, which depends on the noise type and ambient light intensity. fused Represents the image features after fusion through the cross-modal attention module. These features contain the joint information of visible light and infrared sensors, which not only have rich details but also contain deep semantic information. In the fusion process, the system can organically combine the image information collected by different sensors through the cross-modal attention mechanism, making the image more complete and expressive; In practical applications, visible light images and infrared images often complement each other. Visible light images provide rich colors and details, while infrared images provide temperature information and visibility in low-light environments. Through cross-modal fusion, the quality of images can be improved, especially in complex environments (such as low light, smoke, haze, etc.); β is determined by the noise classification network and adjusted in the dynamic autoencoder. The role of β is mainly reflected in the dynamic adjustment of the fusion weight. According to different lighting conditions, β will affect how the system fuses the feature information of different sensors (such as visible light images and infrared images). It is adjusted specifically through the following formula: ; In strong light environment, I avg When the value of is high, the system usually relies more on the data from the visible light sensor, because the visible light image provides more accurate detail information when there is sufficient light. At this time, the value of β will increase, indicating that more emphasis is placed on the features from the visible light sensor during fusion. In low light environments, I avg When the value of is low, the system will rely more on the data of the infrared sensor, because infrared images can provide clearer object contours and heat source information under low light conditions. At this time, the value of β will decrease, thereby reducing the weight of visible light features in the fusion process.

[0043] Furthermore, the noise classification network is based on the ResNet-18 architecture. ResNet is a deep convolutional neural network architecture. By introducing residual connections, the gradient vanishing problem in deep networks can be effectively avoided. The input is the frequency domain features of low-light images (energy distribution after DCT transformation). For low-light images, the image is first converted from the spatial domain to the frequency domain through discrete cosine transform (DCT). DCT represents the information of the image in the form of frequency, thereby separating the low frequency and high frequency of the image, and outputting the noise type label (Gaussian / pulse / mixed noise). Using the energy distribution after DCT transformation as the input feature can effectively help the network classify different types of noise; The denoising kernel function adjusts the size of the kernel function according to different noise types. For example, for impulse noise, a median filter parameter is generated (the dynamic kernel size ranges from 3×3 to 7×7, a smaller kernel size can accurately retain details, while a larger kernel size can effectively remove noise).

[0044] Dynamically adjusting the size of the denoising kernel can be achieved in the following ways: Adaptive algorithm: Automatically adjust the size of the kernel function according to the distribution of noise. A dynamic filter size range can be set based on factors such as noise intensity and image features. When processing different types of noise, the network can automatically learn the optimal kernel size through forward propagation. Multi-scale analysis: Use a multi-scale convolution filter to perform noise reduction at multiple scales. This method can better preserve the details in the image while removing noise at different scales. The details will not be described here.

[0045] Furthermore, when the light intensity is low (such as: illumination Lux<5), dual-channel full-resolution fusion is enabled, and then the high-performance mode is entered; when the battery is low or the light intensity is high (such as: Lux≥5 or battery <20%), only the key areas of the infrared channel are processed to reduce the computing burden and then enter the basic mode; when the network delay is too high (such as: network delay>100ms), the local cache noise reduction parameters are used, and the cloud backhaul link is closed to save bandwidth and battery consumption, and then enter the emergency power reduction mode.

[0046] In different modes, the adjustments of other modules are as follows: In high performance mode: Multispectral sensor integrated module: In this mode, due to the low light intensity, the sensor will maximize the collaborative work of infrared and visible light sensors to collect more image data. At this time, the system will start dual-channel full-resolution fusion to ensure that details are not lost and improve image quality in low-light environments; The image feature extraction module will give priority to using the high-frequency detail enhancement network and semantic segmentation network to process the image, enhance detail performance and maintain semantic information. The cross-modal attention unit dynamically adjusts the fusion ratio of infrared and visible light according to the ambient light to ensure that the details in the low-light area are enhanced while retaining the color and texture in the visible light area. The edge computing module enables adaptive compression and inference optimization, optimizes computing resources to cope with higher computing needs, and ensures the efficiency and accuracy of image processing when resources are sufficient by switching the model structure; Cloud computing guidance module. In this mode, due to sufficient network transmission bandwidth, the cloud can perform real-time optimization and noise reduction processing, and generate an optimized noise reduction kernel function based on the noise classification network, and push it to the edge for real-time noise reduction optimization; Dynamic task scheduling module. At this time, the system will be in high-performance mode, giving priority to ensuring the quality of image processing, and the requirements for power and network delay are relatively loose.

[0047] In basic mode: Multispectral sensor integration module. At this time, the system will focus on processing the data of the key areas of the infrared sensor, avoiding high-load calculations across the board, focusing only on the characteristics of the infrared image, and reducing the amount of calculations; The image feature extraction module focuses on processing infrared data, enhancing thermal radiation features in low-light environments, reducing complex semantic segmentation and cross-modal attention calculations, and highlighting the details of the infrared channel; The edge computing module optimizes the inference process, adapts to lower computing resources, and uses less computing and a simple model structure to process infrared images; Cloud computing guidance module: Since the system reduces the amount of computational processing on the edge, the cloud can perform more sophisticated noise reduction processing on the image in a timely manner. At this time, the noise reduction strategy will mainly rely on the preset noise reduction kernel function and perform processing based on cloud guidance; Dynamic task scheduling module, the system is in basic mode according to the power or network delay, and the battery consumption and computing burden are relatively low.

[0048] In emergency power reduction mode: Multispectral sensor integrated module. In this mode, in order to reduce the computational burden, the system will only process the key areas of the infrared channel to avoid full data processing under low power and high network latency conditions. The collection of visible light sensors will be limited when necessary; The image feature extraction module, in order to reduce processing complexity, will focus on the key area feature extraction of the infrared channel, and other feature extraction tasks will be delayed or cancelled; Edge computing module: At this time, the edge computing module will greatly reduce the amount of calculation through strategies such as pruning, give priority to retaining channels that are sensitive to infrared features, reduce unnecessary calculations, and enable the most simplified model to maintain the basic operation of the system; Cloud computing guidance module: In the emergency power reduction mode, the cloud will no longer perform real-time noise reduction processing, but will only provide simple guidance to the model through the uploaded image quality indicators when necessary. The push frequency of the noise reduction kernel function will be reduced, and the network burden will be compressed to a minimum. Dynamic task scheduling module: The system will reduce the frequency of data uploads and minimize local computing tasks in emergency power reduction mode based on factors such as ambient light intensity, device power, network latency, etc., while stopping the cloud backhaul link to avoid additional bandwidth and battery consumption.

[0049] It should be noted that no matter which mode is used, the edge-cloud feedback closed-loop module will regularly upload the noise-reduced image quality indicators to the cloud, and push the new model to the edge through differential updates. In different modes, the upload frequency and update mechanism will be adjusted according to power, network status and system operation requirements.

[0050] Furthermore, the edge-cloud feedback closed-loop module uploads the denoised image quality index to the cloud every 10 seconds and triggers a cloud retraining operation when the index is lower than the set threshold.

[0051] In detail, the image quality index can be determined using the structural similarity index (SSIM) or the peak signal-to-noise ratio (PSNR). SSIM is an indicator for measuring the structural similarity between two images, and is mainly used to measure the similarity between the original image and the compressed image or processed image. PSNR is a traditional indicator for measuring image quality, and is mainly used to evaluate the quality of compressed images.

[0052] In this embodiment, the specific triggering conditions are as follows: When SSIM is less than 0.85, it means that the structural similarity of the image has decreased, the denoising effect is not ideal, and retraining needs to be triggered to improve the model; Or when the change in PSNR (ΔPSNR) is greater than 2dB, it means that the image quality has changed significantly. The larger the change in PSNR, the greater the decline in image quality, which also means that the denoising model needs to be optimized.

[0053] It should be noted that after the retraining is triggered, the new model will be pushed to the edge through differential update technology. Differential update technology usually refers to transmitting only the incremental part of the model instead of the entire model. This method can greatly reduce the transmission bandwidth and storage consumption. The above model increment is about 3MB, which means that each update only needs to transmit the updated part of the model instead of the complete model, thereby improving efficiency.

[0054] The system operation process is as follows: Data collection and transmission: The multispectral sensor integration module first collects image data in low-light environments. After preprocessing, the data is transmitted to the image feature extraction module; Feature extraction and optimization: The image feature extraction module processes the transmitted data, extracts key features, and dynamically adjusts the fusion ratio of infrared and visible light data through the cross-modal attention unit to ensure the retention of details in low-light environments; Edge computing: In the edge computing module, image data will be adaptively compressed and optimized for inference based on real-time computing resources to reduce the computing burden and dynamically switch the model structure when needed; Cloud guidance: The cloud computing guidance module generates a denoising kernel function based on the noise type in the image, optimizes it through a noise classification network and a dynamic autoencoder, and pushes the generated denoising parameters to the edge. Task scheduling and mode switching: The dynamic task scheduling module automatically adjusts the system's operating mode according to environmental changes (such as light intensity, power, network latency, etc.) to ensure optimal performance in various scenarios; Feedback loop and continuous optimization: In the edge-cloud feedback loop module, the system uploads the denoised image quality index every 10 seconds. When the index is lower than the set threshold, the cloud triggers a retraining operation to further optimize system performance.

[0055] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing system based on the Internet of Things, characterized in that: It includes a multispectral sensor integration module, an image feature extraction module, an edge computing module, a cloud computing guidance module, a dynamic task scheduling module, and an edge-cloud feedback closed-loop module; The multi-spectral sensor integrated module is used to collect image data in a low-light environment through the collaborative work of a visible light sensor and an infrared light sensor; The image feature extraction module is used to extract features from image data collected by the multispectral sensor; The edge computing module is used to achieve adaptive compression and inference optimization, and optimize the utilization of computing resources by switching the model structure when the network load is high; The cloud computing guidance module classifies the noise types in the image through a noise classification network, generates optimized noise reduction kernel functions according to different types of noise, and pushes these parameters to the edge end, thereby dynamically optimizing the noise reduction process during system operation; The dynamic task scheduling module includes a high-performance mode, a basic mode and an emergency power-saving mode. The dynamic task scheduling module realizes switching between the high-performance mode, the basic mode or the emergency power-saving mode according to the ambient light intensity, the device power and the network delay factors; The edge-cloud feedback closed-loop module regularly uploads the denoised image quality indicators to the cloud, and pushes the new model to the edge through differential updates, thereby continuously optimizing the performance of the system.

2. The image processing system based on the Internet of Things according to claim 1, characterized in that: The image feature extraction module includes a high-frequency detail enhancement network, a semantic segmentation network, and a cross-modal attention unit; The high-frequency detail enhancement network is used to enhance the thermal radiation features captured by the infrared sensor to highlight the details in a low-light environment; The semantic segmentation network is used to retain the color and texture information captured by the visible light sensor and enhance the semantic representation of the scene; The cross-modal attention unit is used to dynamically allocate weights between sensor channels and automatically adjust the fusion ratio of infrared and visible light data according to the ambient light intensity to ensure that details are not lost in low-light conditions while retaining the color and texture information of bright areas.

3. The image processing system based on the Internet of Things according to claim 2, characterized in that: The fusion formula of the cross-modal attention unit is: ； Among them, F fused represents the fused image features, F IR is the image feature of the infrared channel, F VIS is the image feature of the visible light channel, and α is the fusion weight, which is dynamically adjusted according to the ambient light intensity.

4. The image processing system based on the Internet of Things according to claim 3, characterized in that: The calculation formula of the fusion weight α in the cross-modal attention unit is: ； Among them, I avg is the average light intensity of the scene.

5. The image processing system based on the Internet of Things according to claim 1, characterized in that: The edge computing module reduces unnecessary computation through a layer pruning strategy and only retains channels that are sensitive to infrared features, with a retention rate greater than 85%.

6. The image processing system based on the Internet of Things according to claim 4, characterized in that: The cloud computing guidance module includes a noise classification network and a dynamic autoencoder; The noise classification network identifies the noise type in the image through a pre-trained model and generates corresponding noise reduction parameters; The dynamic autoencoder is used to generate an optimized denoising kernel function according to the noise type.

7. The image processing system based on the Internet of Things according to claim 6, characterized in that: The formula of the denoising kernel function is as follows: ； Among them, N filter Represented as the filter kernel function after denoising, used for noise suppression in images, N base It is represented by the basic denoising kernel function, which is used to deal with general noise conditions. β is represented by the dynamic adjustment weight according to the noise type and ambient light intensity. β is determined by the noise classification network and adjusted in the dynamic autoencoder using the following formula: 。 8. The image processing system based on the Internet of Things according to claim 7, characterized in that: The noise classification network is based on the ResNet-18 architecture, with the input being the frequency domain features of low-light images and the output being the noise type label; The denoising kernel function adjusts the size of the kernel function according to different noise types.

9. The image processing system based on the Internet of Things according to claim 1, characterized in that: When the light intensity is low, dual-channel full-resolution fusion is enabled to enter high-performance mode; When the battery is low or the light intensity is high, only the key areas of the infrared channel are processed to reduce the computing burden and enter the basic mode; When the network delay is too high, the local cache noise reduction parameters are used and the cloud backhaul link is closed to save bandwidth and battery consumption, thus entering the emergency power reduction mode.

10. The image processing system based on the Internet of Things according to claim 1, characterized in that: The edge-cloud feedback closed-loop module uploads the denoised image quality index to the cloud every 10 seconds and triggers a cloud retraining operation when the index is lower than the set threshold.

Images