A smart camera control method and control system based on the Internet of Things

Through multimodal sensor data fusion and intelligent processing algorithms, high-accuracy target tracking of intelligent cameras in complex environments is realized, and power consumption is reduced, solving the target tracking accuracy and power consumption problems in the prior art.

CN119767137AActive Publication Date: 2025-04-04SHENZHEN JUXIN IMAGE CO LTD

Patent Information

Application Number
CN202510246172.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-04
Publication Date
2025-04-04
Estimated Expiration
2045-03-04

AI Technical Summary

Technical Problem

Existing smart cameras are difficult to effectively track targets in complex environments (such as lighting changes, occlusion, etc.), and their power consumption is high. The existing methods have limited effects on complex environments.

Method used

Multimodal sensor data fusion acquisition is adopted, scene data is acquired through low light sensors and infrared sensors, combined with dynamic weighted image fusion, Bayesian neural networks and Markov random field models, high-accuracy tracking of the target is achieved, and power consumption is reduced through energy efficiency adaptive regulation.

Benefits of technology

In complex environments, it significantly improves the accuracy and robustness of target tracking, reduces equipment power consumption, and enhances the reliability of the system in harsh environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119767137A_ABST
    Figure CN119767137A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent camera control method and control system based on the Internet of Things, which relates to the field of camera control technology, and comprises the following steps: respectively obtaining polarized reflected light images and thermal imaging data through synchronously working low-light sensors and infrared sensors, and dynamically adjusting the fusion weight coefficients of thermal imaging and visible light images; deploying a Bayesian neural network at an edge computing node, extracting spatiotemporal features through a multi-channel feature map and a convolution module, and optimizing target position prediction by updating weights using an MCMC algorithm; compensating for target position offset when the decrease rate of the intersection-over-union ratio of target bounding boxes between consecutive frames exceeds a threshold; dynamically switching the camera working mode based on the target tracking confidence quantile; generating target trajectories through thermal imaging data in a haze environment, and reducing positioning noise using a time series window; the invention significantly improves the accuracy and robustness of target tracking.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field related to camera control, and in particular to an intelligent camera control method and control system based on the Internet of Things. Background Art

[0002] With the continuous development and popularization of Internet of Things technology, more and more smart devices are beginning to be connected to the Internet of Things, including smart cameras. Smart cameras are gradually developing in the direction of intelligence and networking. As of now, smart cameras can be networked, such as directly connected through PC5 connection, so that multiple smart cameras can cooperate to collect and process videos through networking transmission, such as merging videos collected by multiple smart cameras into one for playback, and the user experience is also better.

[0003] However, traditional target tracking algorithms mostly rely on single sensor data (such as infrared or visual images), and are prone to target loss when the environment changes drastically (such as occlusion, sudden changes in light, etc.). In order to improve tracking accuracy, it is usually necessary to maintain high frame rate and high resolution mode, which leads to a significant increase in power consumption. Existing technologies mostly deal with occlusion problems through some simple filtering or compensation methods, but most methods have limited effects on complex environments (such as haze, strong light reflection, etc.). Summary of the invention

[0004] In order to solve the defects of the prior art, the present invention provides an intelligent camera control method and control system based on the Internet of Things.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: The present invention provides a smart camera control method based on the Internet of Things, comprising the following steps: S1: Multimodal sensor data fusion acquisition, using synchronous low-light sensors and infrared sensors to obtain polarized reflected light images and thermal imaging data of the monitored scene respectively; S2: Dual-modal feature association modeling, divide the thermal imaging data into temperature topological partitions according to the pixel grid, trigger the dynamic target tracking signal when the absolute value of the average temperature difference between adjacent partitions exceeds the threshold, and analyze the polarization angle parameters of the material surface from the polarized reflected light image, use the support vector machine classification algorithm to identify the material type, and establish a temperature-material feature association database; S3: Dynamic weighted image fusion, dynamically adjusts the fusion weight coefficient of thermal imaging and visible light images according to the light intensity change rate parameter monitored in real time by the ambient light sensor; S4: Probabilistic reasoning network processing, deploying Bayesian neural networks on edge computing nodes, extracting spatiotemporal features through multi-channel feature maps and convolution modules, and using MCMC algorithms to update weights to optimize target position prediction; S5: Occlusion compensation and trajectory correction. When the intersection-over-combination ratio of the target bounding box between consecutive frames decreases beyond the threshold, the historical temperature data is used to build a Markov random field model, and the probability distribution of the candidate area is calculated based on the temperature spatial correlation to compensate for the target position offset. S6: Adaptive energy efficiency control, dynamically switching the camera working mode based on the target tracking confidence quantile; S7: Multi-node collaborative verification, in a haze environment, the target trajectory is generated through thermal imaging data, the timing window is used to reduce positioning noise, and the positioning results and parameters are uploaded to the cloud platform through the MQTT protocol.

[0006] As a preferred technical solution of the present invention, the fusion formula of step S3 is: ; Among them, α represents the dynamic adjustment coefficient of ambient light, T thermal (x, y) is the temperature value of the thermal imaging image at position (x, y), I visual (x,y) is the pixel value of the visual image at position (x,y).

[0007] As a preferred technical solution of the present invention, step S4 specifically includes the following steps: S41: A four-layer Bayesian neural network is deployed on the edge computing node, and the input layer is connected to a four-channel feature map, including a thermal imaging gradient map, a material reflection feature map, a motion vector map, and ambient lighting parameters; S42: The residual convolution layer extracts spatiotemporal features, the fully connected layer generates the posterior probability distribution of the target location, and the output layer updates the weight parameters based on Markov chain Monte Carlo sampling; S43: Perform posterior probability distribution inference through edge computing devices. The posterior probability distribution inference formula is as follows: ; Among them, ω represents the network weight, D represents the training data, p(w|D) represents the posterior probability distribution, p(D|w) represents the likelihood function, and p(w) represents the prior distribution.

[0008] As a preferred technical solution of the present invention, the weight update of the Bayesian neural network follows the following formula: ; Among them, ω t is the node weight of the tth iteration, ε is the learning rate, P(D|w t) is the likelihood probability of the training data under the current parameters.

[0009] As a preferred technical solution of the present invention, the fusion result F of step S1 fusion (x, y) is used as the input of the lightweight Bayesian neural network for real-time inference of target tracking. The Bayesian neural network infers the posterior probability distribution and outputs the probability distribution of the target position: ; Among them, W 1 , W 2 , W 3 , W 4 is the weight matrix of the Bayesian neural network, ReLU is the activation function, and softmax is used for probability normalization.

[0010] As a preferred technical solution of the present invention, the weight of the Markov random field model in step S5 is defined as follows: ; Among them, T R is the average temperature of the candidate area, T hist is the mean historical temperature of the target, ρ is the spatial correlation factor, and σ is the standard deviation of the temperature distribution.

[0011] As a preferred technical solution of the present invention, in step S6, when the confidence quantile is lower than 0.9, the high-definition mode is enabled, and when the confidence quantile is higher than 0.9, it is switched to the low-power mode, and the inactivated sensor power supply module is turned off.

[0012] As a preferred technical solution of the present invention, the upload data transmission in step S7 adopts the AES-256 encryption algorithm, and the key pair is regenerated every 15 seconds.

[0013] The present invention also provides an intelligent camera control system, the control system comprising a dual-spectrum sensing module, an edge computing unit, an Internet of Things communication module and a dynamic power management unit; The dual-spectrum sensing module integrates a low-light sensor and an infrared sensor, and has a built-in weighted fusion pre-processing chip; The edge computing unit has a built-in programming logic array for performing Bayesian neural network reasoning, and the storage module includes a temperature-material feature association database; The IoT communication module supports dual-protocol switching modules of MQTT protocol and LoRaWAN protocol, and uploads target positioning data and environmental parameters in real time; The dynamic power management unit dynamically adjusts the sensor power consumption and the main frequency of the computing unit according to the operation mode.

[0014] As a preferred technical solution of the present invention, the edge computing unit is equipped with a hardware acceleration module and supports simultaneous floating-point operations of four-channel feature maps. The Internet of Things communication module includes a data encryption sub-module, and the dynamic power management unit integrates an adaptive voltage regulator.

[0015] The beneficial effects of the present invention are: 1. Even in complex environments (such as lighting changes or occlusions), the present invention significantly improves the accuracy and robustness of target tracking through multimodal data fusion, dynamic weighted image fusion and occlusion compensation.

[0016] 2. The present invention uses intelligent energy efficiency adaptive regulation to dynamically adjust the camera working mode to ensure that the high frame rate / high resolution mode is used only when necessary, thereby significantly reducing the power consumption of the device while ensuring tracking accuracy.

[0017] 3. The present invention enhances the reliability of the system in harsh environments through multi-node collaborative verification and real-time cloud data update, and can effectively cope with different complex situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 It is a schematic diagram of the flow of the intelligent camera control method of the present invention. DETAILED DESCRIPTION

[0019] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0020] Embodiment 1: Figure 1 As shown, a smart camera control method based on the Internet of Things includes the following steps: S1: Multimodal sensor data fusion acquisition, through the synchronous working of low-light sensors (covering visible light to near-infrared bands) and infrared sensors (generally using low-power uncooled long-wave infrared sensors with a wavelength of 8-14μm), respectively obtain polarized reflected light images and thermal imaging data of the monitoring scene; low-light sensors support material reflectivity analysis in low-light scenes, such as distinguishing clothing fiber types or vehicle surface coating characteristics; S2: Dual-modal feature association modeling, divide the thermal imaging data into temperature topological partitions according to (preferably 16×16) pixel grids, trigger the dynamic target tracking signal when the average temperature difference between adjacent partitions exceeds the threshold (such as 3°C), analyze the polarization angle parameters of the material surface from the polarized reflected light image, use the support vector machine classification algorithm to identify the material type, and establish a temperature-material feature association database; by dividing the temperature topological partitions and calculating the temperature difference between adjacent partitions, the moving target can be dynamically identified and tracked; S3: Dynamic weighted image fusion, dynamically adjusts the fusion weight coefficient of thermal imaging and visible light images according to the light intensity change rate parameter monitored in real time by the ambient light sensor; S4: Probabilistic reasoning network processing, deploying Bayesian neural networks on edge computing nodes, extracting spatiotemporal features through multi-channel feature maps and convolution modules, and using MCMC algorithms to update weights to optimize target position prediction; S5: Occlusion compensation and trajectory correction. When the decrease rate of the intersection-over-union ratio of the target bounding box between consecutive frames exceeds a threshold (50-85%, preferably 75%), the historical temperature data is called to construct a Markov random field model, and the probability distribution of the candidate area is calculated based on the temperature spatial correlation to compensate for the target position offset. By calculating the change in the decrease rate of the intersection-over-union ratio of the target bounding box between consecutive frames, the target occlusion phenomenon can be detected and corrected. When the target is occluded, the Markov random field model can use the historical temperature data and spatial correlation to infer the potential position of the target, and compensate for the offset of the target position through the data of multimodal sensors (such as infrared sensors and low-light sensors). In this way, even if the target is temporarily occluded, the system can maintain a high tracking accuracy. S6: Adaptive energy efficiency control, dynamically switching the camera working mode based on the target tracking confidence quantile; S7: Multi-node collaborative verification. In a haze environment, the target trajectory is generated through thermal imaging data, the timing window is used to reduce positioning noise, and the positioning results and parameters are uploaded to the cloud platform through the MQTT protocol. This multi-node collaboration can effectively reduce the error of a single camera in complex scenes and further improve the accuracy and stability of target tracking.

[0021] Among them, the polarized reflected light image and thermal imaging data of the target are obtained through the synchronous working of the low-light sensor and infrared sensor. The weighted adaptive fusion algorithm is used to dynamically adjust the fusion weight of thermal imaging and visual data according to the changes in ambient light, thereby enhancing the target tracking capability in complex environments (such as low light, haze, etc.) and reducing the target loss caused by sudden changes in light. When the ambient light intensity changes, the fusion weight coefficient of thermal imaging and visible light images is dynamically adjusted. This ensures that the camera can always provide the best target recognition capability under different lighting conditions, thereby reducing target tracking failures caused by instantaneous lighting changes; When the target encounters occlusion (for example, when the bounding box intersection-to-union ratio drops below a threshold), occlusion compensation is performed using historical temperature data and the Markov random field model. The model uses the temperature spatial correlation of the target to predict and correct the target position offset, maintain continuous tracking of the target, and avoid tracking loss due to short-term occlusion. The camera working mode is dynamically switched based on the confidence quantile of target tracking. When the confidence of the target is lower than a certain threshold, the system automatically switches to high-definition mode to improve accuracy; when the confidence is high, it switches to low-power mode to effectively reduce device power consumption and extend usage time; In haze environments, thermal imaging data is used to generate target trajectories, and timing windows are used to reduce positioning noise to further improve tracking accuracy in non-ideal environments. This is achieved by updating and adjusting resolution and alarm threshold parameters in real time on the cloud platform to ensure the stability of the system under various environmental conditions.

[0022] Furthermore, the fusion weight of thermal imaging and visual data is dynamically adjusted according to the change of ambient light by weighted averaging. The fusion formula of step S3 is: ; Among them, α is the dynamic adjustment coefficient of ambient light, and its value is adjusted according to the change of ambient light. Generally speaking, the range of α is 0.3~0.7. thermal (x, y) is the temperature value of the thermal imaging image at position (x, y), I visual (x,y) is the pixel value of the visual image at position (x,y).

[0023] For example, when the material in step S2 is identified as metal, the thermal imaging weight α in step S3 is forcibly increased by 0.15 to compensate for the imaging deviation caused by the heat absorption of the metal.

[0024] Further, the step S4 specifically includes the following steps: S41: A four-layer Bayesian neural network is deployed on the edge computing node. The four-layer Bayesian neural network consists of an input layer, a residual convolution layer, a fully connected layer, and an output layer. The input layer is connected to a four-channel feature map, including a thermal imaging gradient map, a material reflection feature map, a motion vector map, and ambient lighting parameters. The thermal imaging gradient map is a composite map of horizontal and vertical gradients extracted from thermal imaging data using the Sobel operator. The material reflection feature map is a grayscale image generated after normalization of the polarization angle data collected by the low-light sensor. The motion vector map is a pixel displacement vector field calculated based on the adjacent frame difference method. The ambient lighting parameter is the real-time reading of the visible light sensor in lux. These feature maps provide the key information required for the camera to track targets in complex environments, including temperature changes, material surface information, target motion status and surrounding lighting conditions. Through multi-channel input, it is possible to integrate multiple information, reduce the error caused by a single data source, and improve the accuracy of target recognition and tracking. S42: The residual convolution layer extracts spatiotemporal features and uses convolution operations to extract spatiotemporal information from multi-channel features, further improving the expressiveness of the feature map. The residual structure can effectively alleviate the problem of gradient disappearance or explosion during training, improving the stability and performance of network training. The fully connected layer generates the posterior probability distribution of the target position. The output layer updates the weight parameters based on Markov chain Monte Carlo sampling. In the output layer of the Bayesian neural network, the posterior probability distribution of the target position is updated through the Markov chain Monte Carlo (MCMC) sampling algorithm. The algorithm infers the probability of the target position under given data based on the Bayesian theorem, thereby avoiding target loss in complex environments. Combined with the posterior probability distribution of Bayesian reasoning, the target position can be dynamically updated according to the current observation data to solve the problem of target loss caused by occlusion, sudden change in illumination, etc. Especially in the case of occlusion and sudden change in illumination, Bayesian reasoning can reasonably infer the possible position of the target, which helps to maintain the stability and accuracy of tracking. S43: Perform posterior probability distribution inference through edge computing devices. The posterior probability distribution inference formula is as follows: ; Among them, ω represents the network weight, D represents the training data, p(w|D) represents the posterior probability distribution, p(D|w) represents the likelihood function, and p(w) represents the prior distribution; The weight update of the Bayesian neural network follows the following formula: ; Among them, ω t is the node weight of the tth iteration, ε is the learning rate, P(D|w t ) is the likelihood probability of the training data under the current parameters; This formula indicates that the network weights are updated using the gradient ascent method. In each iteration, the network weights are adjusted according to the gradient of the loss function to maximize the likelihood function P(D|w t ), in this way, the network gradually learns how to predict the location of the target more accurately, especially when tracking the target in a complex environment, which can reduce the risk of losing the target.

[0025] In practical applications, due to the interference of factors such as ambient light and occlusion, the appearance and motion state of the target may change suddenly. The Bayesian neural network uses the above weight update formula to dynamically adjust the posterior probability distribution of the target in the prediction of each frame of the image, thereby reducing the target positioning error caused by environmental changes. Through repeated updates, the network can adapt to and stabilize the target tracking process under different environmental conditions. In addition, through the adaptive reasoning mechanism of the Bayesian neural network, combined with environmental characteristics (such as lighting, motion status, etc.), the camera's working mode can be dynamically adjusted while ensuring accuracy. For example, under good lighting conditions, the resolution or frame rate can be appropriately reduced to reduce power consumption; in the case of sudden changes in light or the target is obscured, the working accuracy can be improved in a timely manner to ensure the stability of tracking.

[0026] The above scheme combines the advantages of Bayesian neural networks and MCMC algorithms, providing a robust solution for target tracking of smart cameras in complex environments. In particular, its ability to dynamically and adaptively adjust weights and optimize target prediction positions can not only improve target tracking accuracy, but also avoid excessive reliance on high power consumption modes when the environment changes, thereby optimizing energy efficiency.

[0027] Furthermore, the fusion result F of step S1 fusion (x, y) is used as the input of the lightweight Bayesian neural network for real-time inference of target tracking. The Bayesian neural network infers the posterior probability distribution and outputs the probability distribution of the target position: ; Among them, W 1 , W 2 , W 3 , W 4 is the weight matrix of the Bayesian neural network, ReLU is the activation function, and softmax is used for probability normalization.

[0028] The lightweight Bayesian neural network can infer the position of the target through the posterior probability distribution. The input undergoes multiple weight matrix transformations (such as being processed by the ReLU activation function and passed to different layers), and finally outputs the probability distribution of the target position. In this process, the core idea of ​​the Bayesian neural network is to update the probability distribution of the target position through Bayesian inference, which can dynamically adjust the prediction of the target position according to different input features.

[0029] In this way, the camera can maintain high target tracking accuracy even in complex or dynamic environments. Thanks to the fused image input, the Bayesian neural network can quickly adapt and adjust predictions under different environmental conditions (such as occlusion or sudden changes in light), thereby reducing the high frame rate requirements required due to target loss.

[0030] This method not only improves tracking accuracy, but also effectively reduces device power consumption and avoids the waste of resources caused by high frame rate and high resolution.

[0031] Further, the weight of the Markov random field model in step S5 is defined as follows: ; Among them, T R is the current average temperature of the candidate region R, T hist is the target historical temperature mean, ρ is the spatial correlation factor, the range of ρ is 0.5~1.2, σ is the standard deviation of temperature distribution, the temperature standard deviation is dynamically calculated through 30 consecutive frames of data, and is usually used to balance the influence of temperature difference, exp is an exponential function, which is used to adjust the weight according to the size of the temperature difference. The larger the temperature difference, the smaller the index value, indicating that the weight of the location is lower; the smaller the temperature difference, the larger the index value, indicating that the weight of the location is higher; The Markov random field model is used to model the skin temperature distribution of the target. The candidate positions of the target are inferred by considering the temperature changes at different positions on the target surface. In this embodiment, the Markov random field model combines the spatiotemporal correlation to generate a candidate position probability map, and introduces the Pearson correlation coefficient to measure the linear relationship between the temperature at the current position and the target's historical temperature data. For example, if the Pearson correlation coefficient between the temperature feature and the historical data is ≥ 0.85, it means that there is a strong positive correlation between them. R -T hist ∣ (temperature difference) will be smaller, indicating that the temperature change trend in this area is consistent with historical data. At this time, it is necessary to increase the weight of this area to improve the accuracy of the prediction. The current weight W(R) will be multiplied by an additional coefficient of 1.2.

[0032] Furthermore, in step S6, when the confidence quantile is lower than 0.9 (when the target is clear and reliably tracked), the high-definition mode is enabled (a higher frame rate and resolution are enabled) to ensure that high-quality video data can still be obtained in a complex environment and to increase the accuracy of target recognition; When the confidence quantile is higher than 0.9 (when the system detects that the target is lost or uncertain), it switches to low power mode (reducing frame rate or resolution) and turns off inactive sensor power modules to reduce the power consumption of the device. In this mode, the processor operating current does not exceed 1.2A, and the sensor sampling frequency is reduced to 60% of the baseline value.

[0033] With this strategy, the system can adjust the working mode according to actual needs to save battery life or extend the working time of the device, especially when continuous high frame rate / high resolution image is not required.

[0034] It should be noted that in order to further improve the performance of the system, hardware acceleration can be combined, such as using FPGA or GPU to accelerate the calculation of the target tracking algorithm. This is especially important for real-time processing at high frame rate and high resolution, and can significantly improve algorithm processing efficiency and reduce power consumption.

[0035] Furthermore, the upload data transmission in step S7 adopts the AES-256 encryption algorithm, and the key pair is regenerated every 15 seconds.

[0036] To ensure the security of transmitted data, all data uploaded to the cloud via the MQTT protocol uses a strong encryption algorithm - AES-256. AES-256 uses a 256-bit key length, which can effectively prevent data from being eavesdropped or tampered with, ensuring that there are no security loopholes during data transmission, and is suitable for IoT environments with high security requirements.

[0037] Dynamic key update mechanism: In order to further improve the security of the system, the system is designed to regenerate the key pair every 15 seconds. This mechanism ensures that even if a malicious attacker obtains the current key, it can only be cracked in a short time, which greatly improves the security of data transmission. The frequency and intensity of key updates enable the system to resist external attacks and prevent long-term key leakage from causing security risks.

[0038] Embodiment 2: The components identical or corresponding to those in Embodiment 1 are marked with the same reference numerals as those in Embodiment 1. For the sake of simplicity, only the differences from Embodiment 1 are described below. Embodiment 2 differs from Embodiment 1 in that: The present invention also provides an intelligent camera control system, the control system comprising a dual-spectrum sensing module, an edge computing unit, an Internet of Things communication module and a dynamic power management unit; The dual-spectrum sensing module integrates a low-light sensor and an infrared sensor, ensures the time alignment of the two sensor data through a synchronous trigger circuit, and has a built-in weighted fusion pre-processing chip to perform weighted adaptive fusion of thermal imaging and visible light images in real time, reducing the back-end computing burden and improving data processing efficiency. It also has an automatic calibration function to adjust sensor parameters according to ambient temperature and lighting conditions to improve image acquisition quality. The edge computing unit is equipped with a quad-core heterogeneous processor (with integrated GPU and FPGA modules, specifically for efficiently executing Bayesian neural network reasoning tasks) and a built-in programming logic array (supporting customized neural network accelerators and optimizing the real-time processing capabilities of multi-modal data) for executing Bayesian neural network reasoning. The storage module contains a temperature-material feature association database for fast retrieval to optimize target recognition and tracking performance. It also supports localized model training and reasoning, reduces dependence on cloud computing resources, and improves data privacy and system response speed. The IoT communication module supports dual-protocol switching modules of MQTT protocol and LoRaWAN protocol, ensuring data transmission stability and wide-area coverage in different network environments, uploading target positioning data and environmental parameters in real time, and also has end-to-end data encryption function, using AES-256 encryption algorithm to ensure security during data transmission; The dynamic power management unit dynamically adjusts the sensor power consumption and the main frequency of the computing unit according to the operation mode.

[0039] The edge computing unit is equipped with a hardware acceleration module (such as Tensor core and FPGA accelerator), which significantly improves the speed of deep learning reasoning and supports simultaneous floating-point operations of four-channel feature maps, optimizing the parallel processing capabilities of multimodal data, including thermal imaging gradient maps, material reflection feature maps, motion vector maps and ambient lighting parameters. The IoT communication module includes a data encryption submodule, which uses the AES-256 encryption algorithm to encrypt the transmitted data and regenerates the key pair every 15 seconds to ensure communication security. The dynamic power management unit integrates an adaptive voltage regulator, which intelligently adjusts the voltage and current according to the equipment operating load, optimizes power consumption to the greatest extent, and facilitates the system to adjust the working mode (HD mode or low power mode) according to actual needs.

[0040] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A smart camera control method based on the Internet of Things, characterized in that: The following steps are involved: S1: Multimodal sensor data fusion acquisition, using synchronous low-light sensors and infrared sensors to obtain polarized reflected light images and thermal imaging data of the monitored scene respectively; S2: Dual-modal feature association modeling, divide the thermal imaging data into temperature topological partitions according to the pixel grid, trigger the dynamic target tracking signal when the absolute value of the average temperature difference between adjacent partitions exceeds the threshold, and analyze the polarization angle parameters of the material surface from the polarized reflected light image, use the support vector machine classification algorithm to identify the material type, and establish a temperature-material feature association database; S3: Dynamic weighted image fusion, dynamically adjusts the fusion weight coefficient of thermal imaging and visible light images according to the light intensity change rate parameter monitored in real time by the ambient light sensor; S4: Probabilistic reasoning network processing, deploying Bayesian neural networks on edge computing nodes, extracting spatiotemporal features through multi-channel feature maps and convolution modules, and using MCMC algorithms to update weights to optimize target position prediction; S5: Occlusion compensation and trajectory correction. When the intersection-over-combination ratio of the target bounding box between consecutive frames decreases beyond the threshold, the historical temperature data is used to build a Markov random field model, and the probability distribution of the candidate area is calculated based on the temperature spatial correlation to compensate for the target position offset. S6: Adaptive energy efficiency control, dynamically switching the camera working mode based on the target tracking confidence quantile; S7: Multi-node collaborative verification, in a haze environment, the target trajectory is generated through thermal imaging data, the timing window is used to reduce positioning noise, and the positioning results and parameters are uploaded to the cloud platform through the MQTT protocol.

2. The method for controlling an intelligent camera based on the Internet of Things according to claim 1, characterized in that: The fusion formula of step S3 is: ; Among them, α represents the dynamic adjustment coefficient of ambient light, T thermal (x, y) is the temperature value of the thermal imaging image at position (x, y), I visual (x,y) is the pixel value of the visual image at position (x,y).

3. The method for controlling an intelligent camera based on the Internet of Things according to claim 1, characterized in that: The step S4 specifically comprises the following steps: S41: A four-layer Bayesian neural network is deployed on the edge computing node, and the input layer is connected to a four-channel feature map, including a thermal imaging gradient map, a material reflection feature map, a motion vector map, and ambient lighting parameters; S42: The residual convolution layer extracts spatiotemporal features, the fully connected layer generates the posterior probability distribution of the target location, and the output layer updates the weight parameters based on Markov chain Monte Carlo sampling; S43: Perform posterior probability distribution inference through edge computing devices. The posterior probability distribution inference formula is as follows: ; Among them, ω represents the network weight, D represents the training data, p(w|D) represents the posterior probability distribution, p(D|w) represents the likelihood function, and p(w) represents the prior distribution.

4. The method for controlling an intelligent camera based on the Internet of Things according to claim 3, characterized in that: The weight update of the Bayesian neural network follows the following formula: ; Among them, ω t is the node weight of the tth iteration, ε is the learning rate, P(D|w t ) is the likelihood probability of the training data under the current parameters.

5. The method for controlling an intelligent camera based on the Internet of Things according to claim 3, characterized in that: The fusion result F of step S1 fusion (x, y) is used as the input of the lightweight Bayesian neural network for real-time inference of target tracking. The Bayesian neural network infers the posterior probability distribution and outputs the probability distribution of the target position: ; Among them, W1, W2, W3, and W4 are the weight matrices of the Bayesian neural network, ReLU is the activation function, and softmax is used for probability normalization.

6. The method for controlling an intelligent camera based on the Internet of Things according to claim 1, characterized in that: The weights of the Markov random field model in step S5 are defined as follows: ; Among them, T R is the average temperature of the candidate area, T hist is the mean historical temperature of the target, ρ is the spatial correlation factor, and σ is the standard deviation of the temperature distribution.

7. The method for controlling an intelligent camera based on the Internet of Things according to claim 1, characterized in that: In the step S6, when the confidence quantile is lower than 0.9, the high-definition mode is enabled, and when the confidence quantile is higher than 0.9, the mode is switched to the low-power mode, and the inactivated sensor power supply modules are turned off.

8. The method for controlling an intelligent camera based on the Internet of Things according to claim 1, characterized in that: The uploading data transmission in step S7 adopts the AES-256 encryption algorithm, and the key pair is regenerated every 15 seconds.

9. An intelligent camera control system, used to implement the intelligent camera control method according to any one of claims 1 to 8, characterized in that: The control system includes a dual-spectrum sensing module, an edge computing unit, an IoT communication module, and a dynamic power management unit; The dual-spectrum sensing module integrates a low-light sensor and an infrared sensor, and has a built-in weighted fusion pre-processing chip; The edge computing unit is configured with a quad-core heterogeneous processor with a built-in programmable logic array for performing Bayesian neural network reasoning, and the storage module includes a temperature-material feature association database; The IoT communication module supports dual-protocol switching modules of MQTT protocol and LoRaWAN protocol, and uploads target positioning data and environmental parameters in real time; The dynamic power management unit dynamically adjusts the sensor power consumption and the main frequency of the computing unit according to the operation mode.

10. The intelligent camera control system according to claim 9, characterized in that: The edge computing unit is equipped with a hardware acceleration module and supports simultaneous floating-point operations of four-channel feature maps. The Internet of Things communication module includes a data encryption submodule, and the dynamic power management unit integrates an adaptive voltage regulator.

Citation Information

Patent Citations

  • Fusion image generation method, device and system for target object

    CN112163627A

  • Target tracking method and system fusing Bayesian trajectory reasoning

    CN112614163A

  • Method and device for recovering texture information of infrared thermal imaging object image

    CN118037595A

  • Efficient robot vision system based on deep learning and multi-modal fusion

    CN118865042A

  • End tool for Surgical instrument

    KR102679240B1

Cited By

  • Dynamic dimming method and system based on dual-channel camera illumination sensing

    CN120264546A

  • A Dynamic Dimming Method and System Based on Illuminance Sensing of Dual-Channel Cameras

    CN120264546B

  • Fault fusion diagnosis method of distributed photovoltaic system and related device

    CN120279230A

  • Intelligent detection method for agricultural diseases and insect pests

    CN120314329A

  • Data processing method based on artificial intelligence chip

    CN120448095A