A method and device for all-weather humanoid monitoring of low-power monitoring equipment

Abstract

The present invention relates to the field of human shape monitoring technology, and discloses an all-weather human shape monitoring method and device for low-power monitoring equipment. The method comprises: enhancing the original image collected by the monitoring equipment to obtain a preprocessed image; matching the low-power mode of the first device and performing area detection to generate a target candidate area; performing multi-scale feature extraction and feature fusion to obtain a multi-scale fusion feature map; performing human shape detection through a human shape detection model with a dual-head detection network structure to obtain a human shape detection result; using a multi-model fusion adaptive tracking framework to perform motion state prediction tracking and environmental change recognition on the human shape detection result to obtain a target tracking result and environmental change information; according to the target tracking result and environmental change information, the monitoring equipment is prioritized for monitoring areas and the power consumption mode is switched to obtain a second device low-power consumption mode. The present invention can adapt to complex environments and realize all-weather low-power human shape monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of human figure monitoring, and in particular to an all-weather human figure monitoring method and device of low-power consumption monitoring equipment. Background Art

[0002] With the rapid development of the Internet of Things and artificial intelligence technologies, intelligent monitoring systems have been widely used in public safety, home protection and other fields. However, traditional monitoring equipment still faces many challenges in all-weather human detection. Complex and changing environmental conditions, such as changes in lighting and weather, often lead to a decline in image quality and affect detection accuracy. Secondly, there is a contradiction between the need for continuous and uninterrupted monitoring and the power consumption of equipment. How to achieve low-power operation while ensuring detection performance has become an urgent problem to be solved.

[0003] In addition, existing human detection algorithms often perform poorly when dealing with human targets of different scales, postures, and occlusions, especially in long-distance and small target scenarios. At the same time, the balance between real-time requirements and complex algorithms also poses challenges to the design of monitoring equipment. How to achieve efficient and accurate human detection and tracking with limited computing resources has become a research focus. On the other hand, the long-term stable operation of monitoring equipment also faces many problems. The dynamic changes in environmental factors may lead to the performance degradation of the detection model. How to achieve adaptive adjustment and optimization of the model has become a key issue. At the same time, how to flexibly adjust the monitoring strategy according to the target activity characteristics and environmental changes to achieve a reasonable allocation of monitoring resources is also a difficult problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides an all-weather humanoid monitoring method and device for low-power consumption monitoring equipment, which are used to adapt to complex environments and realize all-weather low-power consumption humanoid monitoring.

[0005] In a first aspect, the present invention provides an all-weather humanoid monitoring method for a low-power monitoring device, the all-weather humanoid monitoring method for a low-power monitoring device comprising:

[0006] Perform two-step cross non-local hybrid filtering to reduce noise and enhance contrast on the original image collected by the monitoring equipment to obtain a preprocessed image;

[0007] Performing environment recognition and device power consumption mode matching on the preprocessed image to obtain a first device low power consumption mode of the monitoring device, and performing region detection on the preprocessed image through a region proposal network to generate a target candidate region;

[0008] Through a feature extraction network and a feature pyramid network, multi-scale feature extraction and feature fusion are performed on the target candidate area to obtain a multi-scale fusion feature map;

[0009] Inputting the multi-scale fusion feature map into a human shape detection model with a dual-head detection network structure to perform human shape detection to obtain a human shape detection result;

[0010] Adopting a multi-model fusion adaptive tracking framework, the motion state prediction tracking and environmental change recognition of the human detection result are performed to obtain the target tracking result and environmental change information;

[0011] According to the target tracking result and the environmental change information, the monitoring area priority is divided and the power consumption mode is switched for the monitoring device to obtain a low power consumption mode of the second device.

[0012] In a second aspect, the present invention provides an all-weather humanoid monitoring device for a low-power monitoring device, the all-weather humanoid monitoring device for a low-power monitoring device comprising:

[0013] A preprocessing module is used to perform two-step cross non-local hybrid filtering noise reduction and contrast enhancement on the original image collected by the monitoring device to obtain a preprocessed image;

[0014] A matching module, configured to perform environment recognition and device power consumption mode matching on the pre-processed image to obtain a first device low power consumption mode of the monitoring device, and perform region detection on the pre-processed image through a region proposal network to generate a target candidate region;

[0015] A fusion module is used to perform multi-scale feature extraction and feature fusion on the target candidate area through a feature extraction network and a feature pyramid network to obtain a multi-scale fusion feature map;

[0016] A detection module, used for inputting the multi-scale fusion feature map into a human shape detection model of a dual-head detection network structure to perform human shape detection and obtain a human shape detection result;

[0017] A recognition module is used to adopt a multi-model fusion adaptive tracking framework to perform motion state prediction tracking and environmental change recognition on the human detection result to obtain a target tracking result and environmental change information;

[0018] The switching module is used to prioritize the monitoring areas and switch the power consumption modes of the monitoring device according to the target tracking result and the environmental change information, so as to obtain a low power consumption mode of the second device.

[0019] The third aspect of the present invention provides an all-weather humanoid monitoring device of a low-power monitoring device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory to enable the all-weather humanoid monitoring device of the low-power monitoring device to execute the above-mentioned all-weather humanoid monitoring method of the low-power monitoring device.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned all-weather human monitoring method of the low-power monitoring device.

[0021] In the technical solution provided by the present invention, the quality of the original image is effectively improved through two-step cross non-local hybrid filtering noise reduction and contrast enhancement, and the adaptability of the system under complex lighting and weather conditions is improved. The introduction of environmental recognition and power consumption mode matching mechanism enables the monitoring device to automatically adjust the working mode according to different environmental conditions, and realizes the dynamic balance between device performance and power consumption. The regional proposal network is combined with the residual feature detection network to improve the system's detection ability for human targets of different scales, especially the detection effect in long-distance and small target scenes. Through the combination of feature extraction network and feature pyramid network, the effective extraction and fusion of multi-scale features are realized, and the model's ability to understand complex scenes is enhanced. The dual-head detection network structure is adopted to perform classification and regression tasks respectively, which improves the accuracy of human detection and positioning accuracy. The introduction of a multi-model fusion adaptive tracking framework improves the system's ability to predict the target motion state, and can identify environmental changes at the same time, realizing the dynamic adjustment of the tracker. Through the priority division of monitoring areas and the switching of power consumption modes, the reasonable allocation of monitoring resources is realized, and the overall power consumption is reduced while ensuring the monitoring effect of key areas. By dynamically adjusting the parameters of the image sensor, processor, storage module, and communication module, low-power optimization at the hardware level is achieved, extending the working time of the device. This application achieves a comprehensive improvement in detection accuracy, tracking stability, environmental adaptability, and energy efficiency, providing an effective solution for all-weather low-power humanoid monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0023] Figure 1 A schematic flow chart of an all-weather humanoid monitoring method for a low-power monitoring device provided in an embodiment of the present application;

[0024] Figure 2 A schematic block diagram of the structure of an all-weather humanoid monitoring device of a low-power monitoring device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change based on actual conditions.

[0027] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0030] See also Figure 1 , Figure 1 A flowchart of a method for all-weather humanoid monitoring using a low-power monitoring device provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the all-weather human monitoring method of the low-power monitoring device provided in the embodiment of the present application includes steps S100 to S600.

[0031] Step S100, performing two-step cross non-local hybrid filtering noise reduction and contrast enhancement on the original image collected by the monitoring device to obtain a preprocessed image;

[0032] It is understandable that the execution subject of the present invention can be an all-weather humanoid monitoring device of a low-power monitoring device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0033] Specifically, the grayscale extreme point is located for each pixel of the original image, and the grayscale extreme point set is obtained by detecting the maximum and minimum grayscale values ​​in the image. The grayscale extreme point set helps to identify noise and important details in the image. Based on the grayscale extreme point set, the original image is subjected to non-local median filtering, and the noise is reduced by utilizing the similarity of different regions in the image, while retaining the edges and details in the image as much as possible, thereby obtaining a preliminary denoised image. The difference integral is calculated for the preliminary denoised image, and the difference integral map is constructed by integrating the differences in pixel values ​​in the image. The difference integral map is a tool for enhancing the contrast of image details. Based on the difference integral map, the preliminary denoised image is subjected to accelerated non-mean filtering to quickly reduce the noise in the image while retaining more image details to generate a cross-filtered image. The cross-filtered image is subjected to overall denoising to eliminate residual noise in the image. After denoising is completed, the image histogram is calculated for the denoised image to obtain the grayscale distribution information of the image. The grayscale distribution information reflects the distribution of different brightness levels in the image and is used to optimize the contrast and brightness of the image. Based on the grayscale distribution information obtained, histogram equalization is performed on the denoised image. Histogram equalization is a technology that enhances the global contrast of an image. By redistributing the grayscale levels in the image, more pixels are distributed throughout the grayscale range, thereby improving the overall contrast and detail display of the image. Adaptive contrast stretching is performed on the equalized image. By dynamically adjusting the contrast of different areas in the image, the contrast of the image is made more uniform and natural, and a contrast-enhanced image is generated. Edge-preserving smoothing is performed based on the contrast-enhanced image. While smoothing the image, edge details are retained, and the sharpness and detail expression of the image are enhanced. The image brightness and contrast are finally adjusted on the edge-enhanced image so that the image achieves the best balance in brightness and contrast, and finally a preprocessed image is generated.

[0034] Step S200: Performing environment recognition and device power consumption mode matching on the pre-processed image to obtain a first device low power consumption mode of the monitoring device, and performing region detection on the pre-processed image through a region proposal network to generate a target candidate region;

[0035] Specifically, environmental features are extracted from the preprocessed image, and environmental features in the image are extracted through multiple convolution operations, and an initial environmental feature map is generated. Feature weights are calculated based on the initial environmental feature map, and by analyzing the importance of each feature, more critical features are given higher weights to obtain a weighted environmental feature map. The weighted environmental feature map is input into the environmental classifier, and the current monitoring environment is classified by the environmental classifier to determine the current environment type. The result of the environmental classification is used to select the power consumption mode parameter that best matches the current environment from the preset power consumption mode library, thereby obtaining the first device low power consumption mode of the monitoring device. This mode can effectively reduce the energy consumption of the device in different environments and ensure that monitoring can be performed with minimum power consumption under various environmental conditions. The target candidate region is generated for the preprocessed image through the region proposal network. The region proposal network consists of a residual feature detection network, a region generation network, and a region optimization network. The residual feature detection network processes the preprocessed image in five stages. In the first stage, the image is processed using a 7×7 convolution operation with a step size of 2 and an output channel number of 64, and then passes through a batch normalization layer and a rectified linear unit activation function to obtain a first residual feature map. In the second stage, the first residual feature map is subjected to a maximum pooling operation with a step size of 2 and processed by three residual blocks. Each residual block consists of 1×1, 3×3 and 1×1 convolutional layers, and the number of output channels is 64, 64 and 256 respectively, generating the second residual feature map. In the third stage, the second residual feature map is processed by four residual blocks. The 3×3 convolutional layer in each residual block introduces a deformable convolution module to enhance the feature extraction capability of the model. The number of output channels is 512, and the third residual feature map is obtained. In the fourth stage, the third residual feature map is processed by six residual blocks. Deformable convolution is also used in the 3×3 convolutional layer. The number of output channels is increased to 1024, generating the fourth residual feature map. In the fifth stage, three residual blocks are applied to the fourth residual feature map, and the use of deformable convolution is maintained to obtain the fifth residual feature map with a number of channels of 2048. The region generation network processes the third, fourth and fifth residual feature maps. Perform 1×1 convolution operations on the residual feature maps respectively, and reduce the number of channels of all feature maps to 256. Upsample the fifth residual feature map to make its resolution the same as the fourth residual feature map. Figure 1The fusion result is obtained by element-wise addition fusion. After that, the fusion result is upsampled again and the same addition fusion is performed with the third residual feature map to generate the target residual feature map. The target residual feature map is subjected to a 3×3 convolution operation to generate a feature map for anchor frame generation. At each position of the feature map, anchor frames of 9 scales and 3 ratios are generated, for a total of 27 anchor frames. Two parallel fully connected layers are applied to each anchor frame, one for bounding box regression and the other for binary classification, to obtain the initial candidate region. The region optimization network further optimizes the initial candidate region, removes candidate regions with high overlap, introduces a spatial attention mechanism to weight the features of the candidate region, dynamically adjusts the screening threshold, and selects the top K regions as the final target candidate region. Through the above steps, environment recognition, device power consumption pattern matching, and accurate detection of target candidate regions are efficiently performed under low power conditions, thereby achieving all-weather human monitoring.

[0036] Step S300: performing multi-scale feature extraction and feature fusion on the target candidate area through a feature extraction network and a feature pyramid network to obtain a multi-scale fusion feature map;

[0037] Specifically, the target candidate region is input into the feature extraction network, which contains five stages of processing. In the first stage, a 7×7 convolution operation is performed on the target candidate region, the convolution step size is 2, and the number of output channels is 64. After that, it is processed by a batch normalization layer and a ReLU activation function to generate the first region feature map. In the second stage, the first region feature map is subjected to a maximum pooling operation with a step size of 2, and is processed by three radix 32 group convolution blocks. Each convolution block contains 1×1, 3×3 and 1×1 convolution layers, and the number of output channels is 128, 128 and 256 respectively, and finally the second region feature map is obtained. In the third stage, four radix 32 group convolution blocks are applied to the second region feature map, and a deformable convolution module is introduced in the 3×3 convolution layer of each convolution block. This design enables the network to adapt to geometric transformations in the image more flexibly and capture complex features more effectively. After processing, the number of output channels is 512, and the third region feature map is obtained. In the fourth stage, 23 radix-32 group convolution blocks are applied to process the third region feature map, and deformable convolution is continued to be used in the 3×3 convolution layer of each block. The number of output channels is further increased to 1024, and the fourth region feature map is finally generated. In the fifth stage, the fourth region feature map is processed by three radix-32 group convolution blocks, and the use of deformable convolution is maintained. The number of output channels reaches 2048, and the fifth region feature map is generated. After obtaining the third, fourth, and fifth region feature maps, a feature pyramid network is constructed. A 1×1 convolution operation is performed on the fifth region feature map to reduce the number of channels to 256, and the first pyramid feature map is generated. The first pyramid feature map is upsampled by 2 times and fused with the fourth region feature map after 1×1 convolution dimension reduction by element-level addition to generate the second pyramid feature map. The second pyramid feature map is also upsampled by 2 times and fused with the third region feature map after dimensionality reduction to obtain the third pyramid feature map. In order to ensure the multi-scale nature of the feature, the third pyramid feature map is downsampled by 2 times to obtain the fourth pyramid feature map. The fourth pyramid feature map is subjected to a 3×3 convolution operation with a step size of 2 to generate the fifth pyramid feature map. In order to unify the feature dimension, the third, fourth and fifth pyramid feature maps are subjected to a 3×3 convolution operation respectively, and the number of channels is unified to 256, and finally a multi-scale feature set is formed. On this basis, a bidirectional feature pyramid network is constructed, which contains top-down and bottom-up paths. In the top-down path, starting from the fifth pyramid feature map, it is downsampled step by step and fused with the next level feature map to generate an enhanced feature set. At the same time, in the bottom-up path, starting from the bottom-level feature map of the enhanced feature set, it is upsampled step by step and fused with the previous level feature map to generate a refined feature set. Each fusion operation performs weighted feature fusion. At the same time, the channel attention mechanism analysis and spatial attention mechanism analysis are performed on the refined feature set to obtain an attention-enhanced feature set.The attention enhancement feature set is aggregated across scales, and an aggregated feature map is generated by element-by-element addition. Based on the aggregated feature map, multi-scale attention analysis is performed to obtain the final attention enhancement feature map. Adaptive feature selection is performed on the attention enhancement feature map to screen out the most representative features and obtain a selective feature map. Multi-scale feature reconstruction is performed on the selective feature map to generate a multi-scale fusion feature map. The multi-scale fusion feature map can better capture the features of the target area at different scales, improve the accuracy and reliability of human detection, and achieve efficient monitoring under low power consumption conditions.

[0038] Step S400: input the multi-scale fusion feature map into the human figure detection model of the dual-head detection network structure to perform human figure detection and obtain a human figure detection result;

[0039] Specifically, the candidate region features of different sizes in the multi-scale fusion feature map are uniformly processed, adjusted to a fixed size, and a normalized feature map is generated. The generated normalized feature map is input into the human detection model of the dual-head detection network structure for processing. The dual-head detection network structure includes two parts: the classification head and the regression head. The classification head is responsible for class probability analysis of the normalized feature map. It contains two fully connected layers. The output dimension of the first fully connected layer is 1024, and the output dimension of the second fully connected layer is the number of categories plus 1. This design enables the classification head to effectively distinguish different target categories and output the probability distribution of each category. The regression head is responsible for bounding box regression analysis and also contains two fully connected layers. The output dimension of the first regression fully connected layer is also 1024, and the output dimension of the second regression fully connected layer is 4 times the number of categories. The regression head calculates the precise bounding box parameters of the candidate region to help the model locate the precise position of the target during the detection process. While the classification head generates the class probability distribution, the regression head generates the corresponding bounding box regression parameters. Based on these outputs, the system performs multi-scale context perception and obtains richer multi-scale context features by analyzing context information of different scales. Attention-guided refinement is performed based on multi-scale contextual features. By introducing the attention mechanism, the important areas in the image are focused on to obtain more accurate attention refinement features. Adaptive threshold non-maximum suppression is applied to the attention refinement features to suppress repeated or excessive detection frames that appear during the detection process and generate preliminary screening results. In order to improve the detection accuracy, multi-scale cascade calibration is performed on the preliminary screening results to ensure that the final set of detection frames is more accurate and consistent. Based on the calibrated set of detection frames, context-aware post-processing is performed to optimize the detection results by utilizing global and local context information. Temporal consistency constraints are imposed on the optimized detection results to ensure the continuity and consistency of the detection results in time and generate the final human detection results.

[0040] The normalized feature map is input into the first classification fully connected layer for preliminary classification processing. In this process, the first classification fully connected layer converts the input features into preliminary classification features through a series of calculations. The preliminary classification features are activated by the rectified linear unit (ReLU) activation function to increase the nonlinear expression ability of the network, so that the classification model can better capture complex feature patterns. At the same time, random inactivation is performed to prevent network overfitting and improve the generalization ability of the model. After processing, the intermediate classification features are obtained. The intermediate classification features are input into the second classification fully connected layer for classification score calculation. Through the weighted summation operation of the fully connected layer, the features are mapped to the category space to generate the original category score. The Softmax function is performed on the original category score. The Softmax function can convert the score of any real number into a probability distribution, so that the sum of the scores of each category is 1, and the category probability distribution is obtained. At the same time, the same normalized feature map is input into the first regression fully connected layer for preliminary regression analysis. This process is similar to the classification process, but the output is the features related to the bounding box, that is, the preliminary regression features. In order to ensure the stability and nonlinear expression ability of the regression results, the initial regression features will be activated by the hyperbolic tangent activation function to limit the output value within a certain range, making the regression prediction more stable. The activated features will be batch normalized to reduce the distribution offset of the data, improve the convergence of the model and the prediction accuracy, and obtain the intermediate regression features. The intermediate regression features are input into the second regression fully connected layer to calculate the bounding box parameters. The second regression fully connected layer performs weighted summation on the input regression features and outputs the original bounding box parameters. In order to convert the original parameters into a bounding box position offset that better meets the actual detection requirements, the exponential linear unit activation function (ELU) is applied to the original bounding box parameters. The use of ELU can retain a large output value in the positive interval and alleviate the gradient vanishing problem in the negative interval, making the bounding box regression results more stable. Based on the calculated bounding box offset, the candidate region is transformed to adjust the actual position of the bounding box and generate the adjusted bounding box coordinates. The adjusted coordinates are smoothed to reduce the instability caused by jitter or noise, and finally the smoothed bounding box regression parameters are obtained.

[0041] Step S500: adopting a multi-model fusion adaptive tracking framework to perform motion state prediction tracking and environmental change recognition on the human detection result to obtain the target tracking result and environmental change information;

[0042] Specifically, the human detection results are input into the multi-model interaction filter for state estimation. The multi-model interaction filter estimates different aspects of the detection results by combining multiple models to generate a set of initial state estimates. Each initial state vector is predicted by a cubature Kalman filter. The cubature Kalman filter improves the accuracy of state estimation by processing noise in nonlinear systems to obtain a prediction error covariance matrix. The prediction error covariance matrix is ​​subjected to singular value decomposition to simplify the complex structure of the covariance matrix into more representative components to obtain a reconstructed error covariance matrix. Based on the reconstructed error covariance matrix, the likelihood function values ​​and model probabilities of each model are calculated. These probabilities reflect the applicability of different models in the current scene, thereby obtaining a fusion state estimate. The fusion state estimate combines the advantages of multiple models to make the tracking results more reliable and accurate. The fusion state estimates of the past N frames are input into the long short-term memory network (LSTM). LSTM can effectively capture long-term dependencies in time series and generate a prediction state sequence for the future M frames by analyzing historical data. The predicted state sequence not only takes into account the current observation state, but also combines the temporal change trend, making the prediction of the target's motion state more accurate. On this basis, based on the predicted state sequence and the current observation state, the adaptive weights are calculated. These weights are used to balance the relationship between historical trends and current observations to obtain the fused tracking results. The surrounding area of ​​the fused tracking result is feature extracted, and the type of environmental change is identified through sliding window similarity calculation. The similarity calculation can detect changes in the environment, such as light changes or background interference, and dynamically adjust the key parameters of the tracker according to the identified type of environmental change and the current tracking performance indicators to ensure that the tracker can maintain optimal performance under different environmental conditions and obtain updated tracker parameters. Based on the updated tracker parameters, the fused tracking result is verified. The verification process includes checking the existence and consistency of the target to ensure the accuracy of tracking. If the target is slightly displaced or lost, the system will also execute a local search strategy to relocate the target through further search of the surrounding area, and finally obtain the target tracking result. At the same time, through continuous monitoring of environmental change information, the tracking strategy is adjusted in real time, so as to maintain efficient target tracking performance in various environments, and finally generate reliable target tracking results and detailed environmental change information.

[0043] The human detection results are input into the Singer model, current statistical model and Jerk model in the multi-model interaction filter for processing. The input data is processed according to different motion state assumptions to generate three initial state vectors. Each model makes a preliminary estimate of the target's motion through its specific dynamic assumptions. The three generated initial state vectors are model-interacted through the Markov chain transition probability matrix. The Markov chain transition probability matrix is ​​used to describe the state transition probability between models. The state vectors from different models are mixed through the Markov chain transition probability matrix to generate a more comprehensive mixed initial state. Based on the mixed initial state, each model is predicted to generate a predicted state set. In this process, each model uses its unique state transition equation to predict the future state of the target and obtain the corresponding predicted state vector. The observation equation is solved for the predicted state set, and the predicted state is converted into a predicted observation set that matches the actual observation value. The predicted observation set represents the possible observation results of the target at the next moment under different models. In order to evaluate the accuracy of the prediction, the observation residual is calculated based on the comparison between the predicted observation set and the actual observation value to obtain the residual sequence. The residual sequence reflects the difference between the predicted value and the actual value, and is an important basis for correcting the model prediction. By performing adaptive factor analysis on the residual sequence, the noise in the system can be identified and corrected to obtain the corrected observation noise covariance matrix. Based on the corrected observation noise covariance matrix, the Kalman gain is calculated. The Kalman gain is a key parameter in the Kalman filter, which is used to optimally fuse the predicted state with the actual observation to minimize the estimation error. The state update matrix is ​​calculated using the Kalman gain, and a volume transformation is performed on it. The volume transformation enhances the filter's processing ability for nonlinear features by mapping the state vector to a high-dimensional space, and obtains a transformed state estimate set. Based on the transformed state estimate set, the weighted mean and covariance are calculated to obtain a fused state estimate. The fused state estimate is inversely transformed to map it from the high-dimensional space back to the original space to generate the final prediction error covariance matrix.

[0044] Step S600: According to the target tracking result and the environmental change information, the monitoring area priority is divided and the power consumption mode is switched for the monitoring device to obtain a low power consumption mode of the second device.

[0045] Specifically, according to the target tracking results, a target activity hotspot area is generated to reflect the main activity area of ​​the target within the monitoring range. Based on this information, the entire monitoring area is grid-divided to generate a monitoring grid map. Grid division subdivides the monitoring area into several small units, each of which represents a basic area that can be independently managed by the monitoring device. Priority scoring is performed on each grid unit in the monitoring grid map. The criteria for priority scoring may include factors such as the frequency of target appearance, residence time, and importance of the area. The scoring results will be summarized into a grid priority matrix, which reflects the relative importance of each grid in the entire monitoring task. Based on the grid priority matrix, adjacent grid units with similar priorities are merged to generate a preliminary monitoring area division result. The boundary of the preliminary monitoring area division result is optimized, and the boundaries of each monitoring area are adjusted to make it more reasonably allocate monitoring resources to avoid resource waste or monitoring blind spots. The optimized monitoring area division result can more accurately reflect the actual monitoring needs. The optimized monitoring area division result is combined with the environmental change information to construct a regional-environmental feature vector. Based on the regional-environmental feature vector, power consumption mode clustering analysis is performed, and the optimal power consumption mode in different areas is identified through the clustering results. According to the power consumption mode clustering results, the power consumption mode of each monitoring area is classified, and a regional power consumption mode mapping table is generated to list the power consumption modes applicable to each monitoring area, so as to ensure that the monitoring equipment can adopt the optimal energy consumption scheme in different areas. On this basis, global power consumption optimization is carried out, and the global optimal power consumption configuration scheme is found by analyzing the power consumption mode mapping table of each area. The optimization process not only focuses on the energy consumption performance of a single area, but also considers the coordination and resource utilization efficiency of the overall monitoring system. The optimized power consumption mode configuration is obtained, and based on this configuration, the parameters of each hardware module of the monitoring device are adjusted to achieve the goal of the second device low power consumption mode. Parameter adjustment includes multiple aspects: adjusting the exposure time and gain of the image sensor to adapt to different lighting conditions and monitoring requirements; adjusting the operating voltage and frequency of the processor to maintain the best energy efficiency ratio under different workloads; adjusting the data cache and persistence scheme of the storage module to ensure that data processing is both efficient and energy-saving; adjusting the data transmission strategy of the communication module to reduce unnecessary energy consumption and ensure that key data can still be effectively transmitted in low bandwidth and low power consumption mode.

[0046] In the embodiment of the present invention, the quality of the original image is effectively improved by two-step cross non-local hybrid filtering noise reduction and contrast enhancement, and the adaptability of the system under complex lighting and weather conditions is improved. The introduction of environmental recognition and power consumption mode matching mechanism enables the monitoring device to automatically adjust the working mode according to different environmental conditions, and realizes the dynamic balance between device performance and power consumption. The regional proposal network is combined with the residual feature detection network to improve the system's detection ability for human targets of different scales, especially the detection effect in long-distance and small target scenes. Through the combination of feature extraction network and feature pyramid network, the effective extraction and fusion of multi-scale features are realized, and the model's ability to understand complex scenes is enhanced. The dual-head detection network structure is adopted to perform classification and regression tasks respectively, which improves the accuracy of human detection and positioning accuracy. The introduction of the adaptive tracking framework of multi-model fusion improves the system's ability to predict the target motion state, and can identify environmental changes at the same time, realizing the dynamic adjustment of the tracker. Through the priority division of monitoring areas and the switching of power consumption modes, the reasonable allocation of monitoring resources is realized, and the overall power consumption is reduced while ensuring the monitoring effect of key areas. By dynamically adjusting the parameters of the image sensor, processor, storage module, and communication module, low-power optimization at the hardware level is achieved, extending the working time of the device. This application achieves a comprehensive improvement in detection accuracy, tracking stability, environmental adaptability, and energy efficiency, providing an effective solution for all-weather low-power humanoid monitoring.

[0047] In a specific embodiment, the process of executing step S100 may specifically include the following steps:

[0048] The pixel grayscale extreme points of the original image are located to obtain a grayscale extreme point set, and based on the grayscale extreme point set, the original image is subjected to non-local median filtering to obtain a preliminary denoised image;

[0049] Calculating the difference integral of the preliminary denoised image to obtain a difference integral map, and based on the difference integral map, performing accelerated non-mean filtering on the preliminary denoised image to obtain a cross-filtered image;

[0050] Performing overall noise reduction processing on the cross-filtered image to obtain a noise-reduced image, and performing image histogram calculation based on the noise-reduced image to obtain grayscale distribution information;

[0051] According to the grayscale distribution information, the denoised image is subjected to histogram equalization processing to obtain an equalized image, and the equalized image is subjected to adaptive contrast stretching to obtain a contrast enhanced image;

[0052] An edge-preserving smoothing process is performed on the contrast-enhanced image to obtain an edge-enhanced image, and image brightness and contrast are adjusted on the edge-enhanced image to obtain a preprocessed image.

[0053] Specifically, the original image is subjected to the location of the pixel grayscale extreme points. Grayscale extreme points refer to the local maximum or minimum values ​​of the pixel grayscale values ​​in the image. These extreme points usually correspond to significant features in the image, such as edges or corners. By locating these grayscale extreme points, a grayscale extreme point set is generated, which contains all the key grayscale change points in the image. The local extreme point detection algorithm is used to find local extreme points in the neighborhood around each pixel. If the grayscale value of a pixel is the maximum or minimum in its neighborhood, the pixel is considered to be an extreme point and is added to the grayscale extreme point set. Based on the grayscale extreme point set, the original image is subjected to non-local median filtering. Non-local median filtering is a denoising method based on similar regions in an image. Unlike traditional local median filtering, it determines the median by searching for similar pixels in a larger range, thereby retaining more image details. For each pixel in the image, its value will be replaced by the median of a group of similar pixels with the largest weight within a certain search range. This process can effectively reduce noise while retaining the detail information in the image to obtain a preliminary denoised image. Calculate the difference integral map for the preliminary denoised image. The difference integral map is a technique for enhancing image contrast and details. By accumulating the gray value differences between adjacent pixels, the areas with drastic changes in the image are highlighted. The calculation of the difference integral map can be expressed as:

[0054] ;

[0055] in, Indicates the position in the difference integral map The pixel value of Indicates the original image with Adjacent pixel values, and is the size of the integration region. This calculation will enhance the areas in the image where the grayscale value changes significantly, making these areas more prominent in the subsequent processing process. Based on the calculated difference integral map, the preliminary denoised image is subjected to accelerated non-mean filtering. Accelerated non-mean filtering is an improved filtering technique that reduces noise and improves the smoothness of the image by using non-mean calculation. In this process, the filter not only considers the pixel value itself, but also refers to the value of the corresponding position in the difference integral map, so as to perform more accurate smoothing on the image. The resulting cross-filtered image will have lower noise and higher detail retention. The cross-filtered image is subjected to overall denoising to eliminate the noise that may remain in the image and obtain a purer denoised image. The image histogram is calculated based on the denoised image. The histogram reflects the distribution of different grayscale levels in the image. The grayscale distribution information can be used for subsequent contrast adjustment and enhancement. Based on the calculated grayscale distribution information, the denoised image is subjected to histogram equalization. Histogram equalization is a commonly used image enhancement technique that stretches the grayscale distribution of the image so that more pixel values ​​are distributed in the entire grayscale range, thereby improving the global contrast of the image. The equalized image usually has higher visual clarity and contrast. Based on the equalized image, adaptive contrast stretching is performed. Adaptive contrast stretching dynamically adjusts the contrast of different areas according to the local characteristics of the image, so that the image can obtain the best visual effect in each area. Edge-preserving smoothing is performed based on the contrast-enhanced image. While smoothing the image, the sharpness of the edge area is retained. Edge-preserving smoothing usually introduces an edge detection operator to protect the edge area during the smoothing process to avoid edge blurring and obtain an edge-enhanced image. The edge-enhanced image is subjected to the final adjustment of image brightness and contrast. According to the global characteristics of the image, the brightness distribution is balanced and the contrast is optimized so that the image reaches the best visual state. The preprocessed image is generated.

[0056] In a specific embodiment, the process of executing step S200 may specifically include the following steps:

[0057] Extracting environmental features from the preprocessed image to obtain an initial environmental feature map, and calculating feature weights based on the initial environmental feature map to obtain a weighted environmental feature map;

[0058] Input the weighted environment feature map into the environment classifier, classify the current environment, obtain the environment type, and select matching power consumption mode parameters from a preset power consumption mode library according to the environment type to obtain the first device low power consumption mode of the monitoring device;

[0059] Generate target candidate regions for the preprocessed image through a region proposal network, which includes a residual feature detection network, a region generation network, and a region optimization network;

[0060] The residual feature detection network consists of 5 stages, and the processing process of each stage is as follows: Stage 1: Perform a 7×7 convolution operation on the preprocessed image with a step size of 2 and an output channel number of 64, and then pass through a batch normalization layer and a rectified linear unit activation function to obtain the first residual feature map; Stage 2: Perform a maximum pooling operation on the first residual feature map with a step size of 2, and then pass through 3 residual blocks, each of which contains three convolution layers of 1×1, 3×3, and 1×1, and the number of output channels is respectively 64, 64, 256, and the second residual feature map is obtained; the third stage: 4 residual blocks are applied to the second residual feature map, and a deformable convolution module is introduced in the 3×3 convolution layer of each residual block, and the number of output channels is 512, and the third residual feature map is obtained; the fourth stage: 6 residual blocks are applied to the third residual feature map, and deformable convolution is also used in the 3×3 convolution layer, and the number of output channels is 1024, and the fourth residual feature map is obtained; the fifth stage: 3 is applied to the fourth residual feature map residual blocks, keep the use of deformable convolution, the output channel number is 2048, and the fifth residual feature map is obtained; the region generation network processes the third residual feature map, the fourth residual feature map and the fifth residual feature map, and the specific steps include: performing 1×1 convolution operations on the third residual feature map, the fourth residual feature map and the fifth residual feature map respectively, and uniformly reducing the number of channels to 256; upsampling the fifth residual feature map once, and performing element-level addition fusion with the fourth residual feature map to obtain a fusion result; upsampling the fusion result again, and performing element-level addition fusion with the third residual feature map to obtain a target residual feature map; performing a 3×3 volume operation on the target residual feature map to obtain a feature map for generating an anchor frame; at each position of the feature map for generating an anchor frame, generating anchor frames of 9 scales and 3 ratios, a total of 27 anchor frames; applying two parallel fully connected layers to each anchor frame, one for bounding box regression and the other for binary classification, to obtain an initial candidate region;

[0061] The region optimization network optimizes the initial candidate regions, eliminates candidate regions with high overlap, introduces a spatial attention mechanism to weight the features of the candidate regions, dynamically adjusts the screening threshold and selects the top K as the final target candidate regions.

[0062] Specifically, environmental feature extraction is performed on the preprocessed image to extract features that can represent environmental information from the image. The spatial features of the image are captured through convolution operations, and the layer-by-layer abstraction capability of the multi-layer convolutional network is used to obtain the initial environmental feature map. Feature weights are calculated for the initial environmental feature map. Each feature in the initial environmental feature map is weighted to highlight more important features. Weighted processing can be achieved by calculating the importance of each feature, and the importance can be measured based on the distinguishability of the feature in different environments. For example, the following formula can be used to calculate feature weights:

[0063] ;

[0064] in, Indicates The weight of the feature, Indicates The value of the feature in the initial environment feature map, is an adjustment parameter used to control the sensitivity of weight calculation. In this way, a weighted environmental feature map is obtained to reflect the important information in the current environment. After the weighted environmental feature map is input into the environmental classifier, the current environment is classified. The environmental classifier determines the type of environment in which the current monitoring device is located by analyzing the information in the weighted environmental feature map, such as daytime, nighttime, sunny day, rainy day, etc. According to the classification result, the most matching power consumption mode parameter is selected from the preset power consumption mode library to obtain the first device low power consumption mode of the monitoring device, ensuring that the monitoring device can work with optimal energy efficiency in different environments. After determining the low power consumption mode, the target candidate region is generated for the preprocessed image through the region proposal network. The region proposal network consists of a residual feature detection network, a region generation network, and a region optimization network. The residual feature detection network is used to extract multi-scale feature representations from the preprocessed image. In the first stage, the preprocessed image is subjected to a 7×7 convolution operation with a step size of 2 to reduce the resolution of the feature map and increase the number of channels to 64. The convolved feature map is processed by batch normalization and rectified linear unit (ReLU) activation function to obtain the first residual feature map. The first residual feature map enters the second stage of processing and performs a maximum pooling operation with a step size of 2 to further reduce the size of the feature map. The feature map is processed by three residual blocks, each of which contains 1×1, 3×3 and 1×1 convolution layers, and the number of output channels is 64, 64 and 256 respectively. After processing, the second residual feature map is generated. In the third stage, four residual blocks are applied to the second residual feature map, and a deformable convolution module is introduced in the 3×3 convolution layer of each residual block. The deformable convolution module can better capture the geometric change characteristics in the image by adjusting the shape and size of the convolution kernel. The final output channel number is 512, and the third residual feature map is obtained. In the fourth stage, the third residual feature map is processed by six residual blocks, and deformable convolution is also used in the 3×3 convolution layer of each residual block. The number of output channels increases to 1024, and the fourth residual feature map is generated. In the fifth stage, the fourth residual feature map is processed by three residual blocks, and the deformable convolution is maintained. The final output channel number is 2048, and the fifth residual feature map is obtained. The region generation network processes the third, fourth and fifth residual feature maps. The three feature maps are respectively subjected to 1×1 convolution operations to uniformly reduce the number of channels to 256 for subsequent feature fusion and processing. The fifth residual feature map is upsampled once to match its resolution with the fourth residual feature map, and the two feature maps are fused together by element-level addition to obtain a preliminary fusion result. The fusion result is upsampled again and the same fusion operation is performed with the third residual feature map to finally generate the target residual feature map. The target residual feature map is subjected to a 3×3 convolution operation to obtain a feature map for generating anchor frames. On the feature map for generating anchor frames, anchor frames of 9 scales and 3 ratios are generated at each position, for a total of 27 anchor frames.These anchor boxes are processed by two parallel fully connected layers, one for bounding box regression to predict the precise location of the target, and the other for binary classification to determine whether the anchor box contains the target, and finally obtain the initial candidate region. The region optimization network optimizes the initial candidate region on this basis. Candidate regions with high overlap are eliminated to reduce the interference caused by repeated detection. The spatial attention mechanism is introduced to weight the features of the candidate regions and increase the weights of important regions. At the same time, by dynamically adjusting the screening threshold, the system can adaptively select the top K regions that are most likely to contain the target, and finally generate optimized target candidate regions.

[0065] In a specific embodiment, the process of executing step S300 may specifically include the following steps:

[0066] The target candidate region is input into the feature extraction network. The feature extraction network contains 5 stages. The processing process of each stage is as follows: Stage 1: Perform a 7×7 convolution operation on the target candidate region with a step size of 2 and an output channel number of 64, and then pass through a batch normalization layer and a ReLU activation function to obtain the first region feature map; Stage 2: Perform a maximum pooling operation on the first region feature map with a step size of 2, and then pass through 3 group convolution blocks with a base of 32. Each block contains three convolution layers of 1×1, 3×3, and 1×1, and the output channel numbers are 128, 128, and 256 respectively, to obtain the second region feature map ; Stage 3: Apply 4 radix-32 group convolution blocks to the second regional feature map, introduce a deformable convolution module in the 3×3 convolution layer of each block, and the number of output channels is 512 to obtain the third regional feature map; Stage 4: Apply 23 radix-32 group convolution blocks to the third regional feature map, and also use deformable convolution in the 3×3 convolution layer, and the number of output channels is 1024 to obtain the fourth regional feature map; Stage 5: Apply 3 radix-32 group convolution blocks to the fourth regional feature map, keep the use of deformable convolution, and the number of output channels is 2048 to obtain the fifth regional feature map;

[0067] A feature pyramid network is constructed based on the third region feature map, the fourth region feature map, and the fifth region feature map, and the specific steps include: performing a 1×1 convolution operation on the fifth region feature map to reduce the number of channels to 256 to obtain a first pyramid feature map; performing a 2-fold upsampling on the first pyramid feature map, and performing element-level addition fusion with the fourth region feature map after 1×1 convolution dimension reduction to obtain a second pyramid feature map; performing a 2-fold upsampling on the second pyramid feature map, and performing element-level addition fusion with the third region feature map after 1×1 convolution dimension reduction to obtain a third pyramid feature map; performing a 2-fold downsampling on the third pyramid feature map to obtain a fourth pyramid feature map; performing a 3×3 convolution operation on the fourth pyramid feature map with a step size of 2 to obtain a fifth pyramid feature map; performing a 3×3 convolution operation on the third pyramid feature map, the fourth pyramid feature map, and the fifth pyramid feature map, respectively, unifying the number of channels to 256, and obtaining a multi-scale feature set;

[0068] Based on the feature maps in the multi-scale feature set, a bidirectional feature pyramid network is constructed, which includes the following steps: Top-down path: starting from the fifth pyramid feature map, downsampling step by step and merging with the next-level feature map to obtain an enhanced feature set; Bottom-up path: starting from the bottom-level feature map of the enhanced feature set, upsampling step by step and merging with the previous-level feature map to obtain a refined feature set; weighted feature fusion is performed on each fusion operation, and at the same time, channel attention mechanism analysis and spatial attention mechanism analysis are performed on the refined feature set to obtain an attention-enhanced feature set;

[0069] Perform element-by-element addition of cross-scale feature aggregation on the attention enhancement feature set to obtain an aggregated feature map, and perform multi-scale attention analysis based on the aggregated feature map to obtain an attention enhancement feature map;

[0070] Adaptive feature selection is performed on the attention enhanced feature map to obtain a selective feature map, and multi-scale feature reconstruction is performed on the selective feature map to obtain a multi-scale fusion feature map.

[0071] Specifically, a convolution operation is performed on the target candidate region. Assume that the input target candidate region is , then in the first stage, a 7×7 convolution kernel is used Perform a convolution operation, the output of which is the first regional feature map , which can be expressed as:

[0072] ;

[0073] in, Represents the weight matrix of the convolution kernel, * represents the convolution operation, is the bias, BN represents the batch normalization operation, and ReLU is the rectified linear unit activation function. Through this step, The number of output channels of is 64, and the size is reduced due to the stride of 2. In the second stage, The size is further reduced by the maximum pooling operation and then processed by three group convolution blocks with a base of 32. Each group convolution block includes 1×1, 3×3 and 1×1 convolution layers, and its output can be expressed as:

[0074] ;

[0075] in, is the second regional feature map, and are the convolution kernel weights and biases in the second stage. The number of output channels is 128, 128, and 256 respectively through convolution operations. In the third stage, When processing, four radix 32 group convolution blocks are applied. A deformable convolution module is introduced in each 3×3 convolution layer. The function of this module is to dynamically adjust the position of the convolution kernel to adapt to the complex morphology of image features. Output of the deformable convolution layer It can be expressed as:

[0076] ;

[0077] Among them, DC represents the deformable convolution operation, and is the weight and bias of the convolution kernel. The number of output channels at this stage is 512, generating the third region feature map The fourth phase continues 23 base-32 group convolution blocks are applied to the same network, and the output is the feature map of the fourth region. , the number of channels is 1024. Similar processing is performed in the fifth stage, and the fifth regional feature map is finally generated , whose number of channels is 2048. In order to construct the feature pyramid network, Perform a 1×1 convolution operation to reduce the number of channels to 256 and obtain the first pyramid feature map

[0078] ;

[0079] right Perform 2 times upsampling and compare it with the feature map of the fourth region Perform element-level addition fusion to obtain the second pyramid feature map

[0080] ;

[0081] Similarly, Upsample again by a factor of 2 and compare Fusion is performed to generate the third pyramid feature map In the fused multi-scale feature set, the number of channels is unified to 256. These feature maps are downsampled step by step in the top-down path and upsampled step by step in the bottom-up path, and fused to obtain enhanced feature sets and refined feature sets. Cross-scale feature aggregation is performed on the attention-enhanced feature set, and the aggregated feature map is obtained by element-by-element addition.

[0082] ;

[0083] right Perform multi-scale attention analysis and adaptive feature selection to obtain the final multi-scale fusion feature map Through this process, rich multi-scale and multi-level features are extracted from the image.

[0084] In a specific embodiment, the process of executing step S400 may specifically include the following steps:

[0085] Unify the candidate region features of different sizes in the multi-scale fusion feature map to a fixed size to obtain a normalized feature map;

[0086] The normalized feature map is input into a human shape detection model of a dual-head detection network structure. The human shape detection model of the dual-head detection network structure includes a classification head and a regression head, wherein: the classification head includes two fully connected layers, the output dimension of the first classification fully connected layer is 1024, and the output dimension of the second classification fully connected layer is the number of categories plus 1; the regression head includes two fully connected layers, the output dimension of the first regression fully connected layer is 1024, and the output dimension of the second regression fully connected layer is 4 times the number of categories;

[0087] The classification head performs category probability analysis on the normalized feature map to obtain the category probability distribution, and the regression head performs bounding box regression analysis on the normalized feature map to obtain the bounding box regression parameters;

[0088] Multi-scale context perception is performed based on category probability distribution and bounding box regression parameters to obtain multi-scale context features, and attention-guided refinement processing is performed based on the multi-scale context features to obtain attention refinement features;

[0089] Adaptive threshold non-maximum suppression is performed on the attention refinement features to obtain preliminary screening results, and multi-scale cascade calibration is performed on the preliminary screening results to obtain a calibrated detection frame set;

[0090] Context-aware post-processing is performed based on the calibrated detection frame set to obtain an optimized detection result, and temporal consistency constraints are performed on the optimized detection result to obtain a human detection result.

[0091] Specifically, the multi-scale fusion feature map is processed. The multi-scale fusion feature map usually contains candidate region features of different scales and positions. These features need to be unified to a fixed size before being input into the detection network to ensure that the model can process them effectively. Through the scale normalization operation, all candidate region features are adjusted to the same size to generate a normalized feature map. Assume that each candidate region feature map With different widths and heights, they are transformed to the same size through interpolation or pooling operations. ,Right now:

[0092] ;

[0093] in, Represents the normalized feature map, Resize is the image resizing operation, and is a uniform height and width. Through this processing, all candidate region feature maps are normalized to the same size to form a normalized feature map. The normalized feature map is input into the human detection model of the dual-head detection network structure for human detection. The dual-head detection network structure includes two main branches: a classification head and a regression head. The function of the classification head is to classify the objects in the normalized feature map and determine which category the target belongs to. The classification head consists of two fully connected layers. The output dimension of the first fully connected layer is 1024, which is used to extract high-dimensional features; the output dimension of the second fully connected layer is the number of categories plus 1 (where +1 is usually used to represent the background class), which is used to generate the probability distribution of each category. Assume that the number of categories is , then the output dimension of the second classification fully connected layer is . In terms of formula, it can be expressed as:

[0094] ;

[0095] in, represents the category probability distribution, and are the weight matrices of the first and second fully connected layers in the classification head, and is the corresponding bias term, ReLU is the rectified linear unit activation function, and the Softmax function is used to convert the output into a probability distribution. At the same time, the regression head is used to perform accurate bounding box regression analysis on the location of the target. The regression head also contains two fully connected layers. The output dimension of the first fully connected layer is 1024, and the output dimension of the second fully connected layer is (4 represents the four parameters of the bounding box, Represents the number of categories). The bounding box parameters of each candidate region are generated by the regression head:

[0096] ;

[0097] in, represents the bounding box regression parameters, and are the weight matrices of the first and second fully connected layers in the regression head, and is the corresponding bias term. The category probability distribution obtained by the classification head and the bounding box regression parameters generated by the regression head , and perform multi-scale context-aware processing. Multi-scale context-awareness combines context information at different scales to obtain more accurate and comprehensive target features. This process can be achieved by fusing feature maps at multiple scales. To achieve this, we finally generate multi-scale context features

[0098] ;

[0099] in, is the weight coefficient, indicating the importance of each scale feature. The multi-scale context features are further enhanced through attention-guided refinement. The attention mechanism refines the feature representation by focusing on the most significant part of the image and generating attention-refined features. . Refine the features of attention Adaptive threshold non-maximum suppression is performed to remove redundant overlapping frames and retain only the most likely target candidate frames to obtain preliminary screening results. The process of adaptive threshold non-maximum suppression can be expressed as:

[0100] ;

[0101] Among them, IoU represents the intersection over union ratio of two bounding boxes, and threshold is the suppression threshold. The preliminary screening results are calibrated in a multi-scale cascade, and the accuracy and consistency of the candidate boxes are ensured through layer-by-layer feature fusion and adjustment to obtain a set of calibrated detection boxes. On this basis, context-aware post-processing is performed to optimize the detection results by combining the environmental information around the target to make them more consistent with the actual scene, and generate optimized detection results. Temporal consistency constraints are imposed on the optimized detection results to ensure the continuity and consistency of the detection results in time in the case of videos or continuous frames, so as to obtain reliable human detection results.

[0102] In a specific embodiment, the execution step performs a category probability analysis on the normalized feature map through a classification head to obtain a category probability distribution, and performs a bounding box regression analysis on the normalized feature map through a regression head to obtain a bounding box regression parameter. The process may specifically include the following steps:

[0103] The normalized feature map is input into the first classification fully connected layer for classification processing to obtain preliminary classification features, and the preliminary classification features are subjected to modified linear unit activation function and random inactivation processing to obtain intermediate classification features;

[0104] The intermediate classification features are input into the second classification fully connected layer to calculate the classification score to obtain the original category score, and the Softmax function is performed on the original category score to obtain the category probability distribution;

[0105] The normalized feature map is input into the first regression fully connected layer for regression analysis to obtain preliminary regression features, and the preliminary regression features are processed with hyperbolic tangent activation function and batch normalization to obtain intermediate regression features;

[0106] The intermediate regression features are input into the second regression fully connected layer to calculate the bounding box parameters to obtain the original bounding box parameters, and the exponential linear unit activation function is performed on the original bounding box parameters to obtain the bounding box offset;

[0107] The candidate region is transformed in coordinates based on the bounding box offset to obtain adjusted bounding box coordinates, and the adjusted bounding box coordinates are smoothed to obtain bounding box regression parameters.

[0108] Specifically, the normalized feature map is input into the first classification fully connected layer for processing. In this process, the normalized feature map is converted into a high-dimensional feature representation, namely, the preliminary classification feature, through the first classification fully connected layer. The first classification fully connected layer performs a linear transformation on the input features, that is, maps the input features into a high-dimensional space through matrix multiplication and addition. The formula is as follows:

[0109] ;

[0110] in, is the preliminary classification feature, is the weight matrix of the first classification fully connected layer, is the normalized feature map, is the bias term. The preliminary classification features are transformed nonlinearly through the rectified linear unit (ReLU) activation function. This operation can enhance the expressiveness of the model and increase the model's ability to capture complex patterns. The function of ReLU is to set values ​​less than zero in the input to zero and retain values ​​greater than zero. After ReLU processing, the activated features are randomly inactivated (Dropout), which is a regularization technique that reduces the possibility of model overfitting by randomly setting the output of a part of the neurons to zero. After these processes, the intermediate classification features are obtained. The intermediate classification features are input into the second classification fully connected layer to calculate the classification scores. The second classification fully connected layer again linearly transforms the input features to generate the original category scores, which represent the model's confidence in each category. The original category scores are operated through the Softmax function to convert the scores into probability distributions:

[0111] ;

[0112] in, is the output feature of the second classification fully connected layer, and are the weight matrix and bias term of this layer respectively, is the category probability distribution. The Softmax function can convert any real number score into a probability value between 0 and 1, and ensure that the sum of the probabilities of all categories is 1, and obtain the final category probability distribution. At the same time, the normalized feature map is input to the first regression fully connected layer for regression analysis. Similar to the processing process of the classification network, the first regression fully connected layer performs a linear transformation on the input features to generate preliminary regression features:

[0113] ;

[0114] in, is the initial regression feature, is the weight matrix of the first regression fully connected layer, is the bias term. The preliminary regression features are then nonlinearly transformed through the hyperbolic tangent activation function (Tanh). The Tanh function can map the input features to the interval [-1, 1], making the distribution of feature values ​​more uniform. The features after Tanh activation are batch normalized. By standardizing the mean and variance of each batch of inputs, the distribution differences between features are reduced, the network training process is accelerated, and the stability of the model is improved. After these processes, the intermediate regression features are obtained. The intermediate regression features are input to the second regression fully connected layer to calculate the bounding box parameters. The second regression fully connected layer performs a linear transformation on the input features to generate the original bounding box parameters

[0115] ;

[0116] in, and is the weight matrix and bias term of the second regression fully connected layer. In order to make the bounding box parameters more in line with the needs of actual application scenarios, the original bounding box parameters are activated by the exponential linear unit (ELU). The ELU activation function can alleviate the gradient vanishing problem while maintaining the nonlinearity of the input features and limit the output value to a certain range. After the ELU operation, the bounding box offset is generated. The candidate region is transformed based on the bounding box offset. The adjustment of the bounding box coordinates usually uses the following formula:

[0117] ;

[0118] in, are the adjusted bounding box coordinates, are the bounding box coordinates of the original candidate region, It is the offset calculated by the regression network. In order to make the adjusted bounding box coordinates smoother and more stable, it is usually smoothed to reduce the instability caused by noise or error and obtain the final bounding box regression parameters.

[0119] In a specific embodiment, the process of executing step S500 may specifically include the following steps:

[0120] The human detection result is input into the multi-model interactive filter for state estimation to obtain an initial state estimation set, and each initial state vector in the initial state estimation set is predicted by the cubature Kalman filter to obtain a prediction error covariance matrix;

[0121] Perform singular value decomposition on the prediction error covariance matrix to obtain the reconstructed error covariance matrix, and calculate the likelihood function value and model probability of each model based on the reconstructed error covariance matrix to obtain the fusion state estimation;

[0122] The fusion state estimation of the past N frames is input into the long short-term memory network to obtain the predicted state sequence of the future M frames, and the adaptive weight is calculated based on the predicted state sequence and the current observed state to obtain the fusion tracking result;

[0123] Perform feature extraction and sliding window similarity calculation on the surrounding area of ​​the fused tracking result to obtain the type of environmental change, and adjust the key parameters of the tracker according to the type of environmental change and tracking performance indicators to obtain updated tracker parameters;

[0124] Based on the updated tracker parameters, target verification and local search strategy are performed on the fused tracking results to obtain target tracking results and environment change information.

[0125] Specifically, the human detection results are processed by multi-model interactive filtering. The function of the multi-model interactive filter is to estimate the target state by combining multiple dynamic models to improve the robustness and accuracy of tracking. The human detection results are used as input and enter the multi-model interactive filter, which includes multiple models that may describe the target motion, such as the Singer model, the current statistical model, and the Jerk model. These models perform state estimation on the detection results respectively to obtain an initial state estimation set, which contains preliminary predictions of different models on state variables such as the target position and speed. Each initial state vector in the initial state estimation set is predicted by a cubature Kalman filter. The cubature Kalman filter is a nonlinear filtering method that updates the state and covariance by selecting a certain number of sampling points in the state space and calculating the propagation of these points. The cubature Kalman filter can handle nonlinear systems without significantly increasing the computational complexity, and is superior to the traditional extended Kalman filter in terms of state estimation accuracy. After the cubature Kalman filter prediction, the prediction error covariance matrix is ​​obtained, which describes the uncertainty in the state estimation. The prediction error covariance matrix is ​​decomposed into a series of eigenvalues ​​and eigenvectors by singular value decomposition to obtain the reconstructed error covariance matrix. The reconstruction process aims to remove or reduce the noise and error in the covariance matrix and retain the most important characteristic components to enhance the stability and accuracy of subsequent calculations. Based on the reconstructed error covariance matrix, the likelihood function value and model probability of each model are calculated. The likelihood function value reflects the ability of each model to explain the current observation data, while the model probability represents the weight of each model in the multi-model fusion. By combining these model probabilities, a more robust fusion state estimate is obtained. The fusion state estimate of the past N frames is input into the long short-term memory network (LSTM). LSTM is a neural network that can capture long-term dependencies in time series and is suitable for handling target tracking problems with time series dependencies. Under the processing of LSTM, the movement trend of the target is learned from the historical state, and the state sequence of the next M frames is predicted. Based on the predicted state sequence and the current observed state, adaptive weights are calculated. These weights are used to balance between the current observed data and the predicted data to obtain more accurate fusion tracking results. Perform feature extraction and sliding window similarity calculation on the area around the fused tracking result to detect the type of environmental change. Feature extraction can use convolutional neural networks or other image processing technologies to extract information features such as color, texture, and edges from the image. The sliding window similarity calculation is to slide and compare in different areas, calculate the similarity score, and determine whether the environment has changed. Based on the similarity calculation, identify the type of change in the current environment, such as light changes, background changes, etc. Automatically adjust the key parameters of the tracker based on the identified type of environmental change and tracking performance indicators (such as tracking accuracy, tracking stability, etc.).For example, when the light changes significantly, adjust the contrast and brightness parameters of the image processing, or change the weight distribution of the filter to adapt to the new environmental conditions. By adjusting and updating the tracker parameters, efficient target tracking is maintained in the new environment. Based on the updated tracker parameters, the fusion tracking results are subjected to target verification and local search strategy. The purpose of target verification is to check the validity of the current tracking results and ensure that the tracked target is consistent with the detection results; while the local search strategy is to relocate the target by searching in a small area around the target when the target may be slightly displaced or temporarily lost. After this series of processing, the target tracking results and environmental change information are finally obtained.

[0126] In a specific embodiment, the execution step inputs the human detection result into the multi-model interactive filter for state estimation to obtain an initial state estimation set, and performs cubature Kalman filter prediction on each initial state vector in the initial state estimation set to obtain a prediction error covariance matrix, which may specifically include the following steps:

[0127] The human detection results are respectively input into the Singer model, the current statistical model and the Jerk model in the multi-model interaction filter for processing to obtain three initial state vectors;

[0128] The three initial state vectors are modeled to interact and a mixed initial state is obtained. The model interaction process uses a Markov chain transfer probability matrix.

[0129] Based on the mixed initial state, the state of each model is predicted to obtain a predicted state set, and the observation equation is solved for the predicted state set to obtain a predicted observation set;

[0130] Based on the predicted observation set and the actual observation value, the observation residual is calculated to obtain the residual sequence, and the residual sequence is subjected to adaptive factor analysis to obtain the corrected observation noise covariance matrix;

[0131] Based on the modified observation noise covariance matrix, the Kalman gain is calculated to obtain the state update matrix, and the state update matrix is ​​volume transformed to obtain a transformed state estimation set;

[0132] Based on the transformed state estimation set, the weighted mean and covariance are calculated to obtain the fused state estimation, and the fused state estimation is inversely volumetrically transformed to obtain the prediction error covariance matrix.

[0133] Specifically, the state information (such as position, speed, etc.) of the detected humanoid target is input into multiple models in the multi-model interaction filter. Commonly used models include the Singer model, the current statistical model, and the Jerk model, which are suitable for different types of motion trajectories. The Singer model is suitable for situations where the target accelerates and decelerates, the current statistical model is suitable for uniform motion, and the Jerk model can handle sudden acceleration or deceleration in the target's motion. In the first step of filtering, each model will independently process the input detection results to generate its own initial state vector. Assume that the detection result is , then for each model ( , corresponding to the Singer, current statistics and Jerk models respectively), will generate the initial state vector :

[0134] ;

[0135] in, It is The state prediction function of the model, is the initial state vector generated by the model. , and Model interaction is performed to obtain a mixed initial state. The model interaction process usually uses a Markov chain transition probability matrix, which describes the possibility of state transition between different models. Assume that the transition probability matrix of the Markov chain is ,in Representation Model Transfer to Model The probability of mixing the initial state It can be expressed as:

[0136] ;

[0137] The mixed initial state combines the advantages of multiple models, making the filter more robust and accurate when processing complex motion patterns. Based on the mixed initial state, each model is predicted again to obtain a set of predicted states. Each model predicts the mixed initial state according to its unique state transfer equation. Assume that the state transfer equation is ,in It is a model The state transition matrix, is process noise, then for each model, the predicted state is:

[0138] ;

[0139] Through this prediction process, we get the predicted state set under different models. We solve the observation equation for the predicted state set to get the predicted observation set. The observation equation maps the predicted state to the observation space and converts the internal motion state into an observable quantity. Assume that the observation equation is ,in is the observation matrix, is the observation noise, then the predicted observation is:

[0140] ;

[0141] At this time, the predicted observation set includes the predicted observation values ​​under each model. Calculate the observed residual. The observed residual is the difference between the predicted observation and the actual observation, which is a measure of the accuracy of the forecast. It can be expressed as:

[0142] ;

[0143] By calculating the residuals of each model, the residual sequence is obtained. In order to improve the robustness of the model, adaptive factor analysis is performed on the residual sequence to identify and correct the observation noise, and the corrected observation noise covariance matrix is ​​obtained. The corrected covariance matrix can more accurately reflect the current observation noise level and is used for subsequent filtering and state updates. Based on the corrected observation noise covariance matrix, the Kalman gain is calculated. The Kalman gain is used to find the best balance between the predicted state and the actual observation, thereby updating the state estimate. The calculation formula of the Kalman gain is:

[0144] ;

[0145] in, is the covariance matrix of the predicted state, is the observation matrix, is the corrected observation noise covariance matrix. Through the Kalman gain, the state update matrix is ​​obtained. The state update matrix is ​​volumetrically transformed. The volumetric transformation is used to perform geometric transformations in the state space, so that the filter can handle more complex nonlinear situations. The transformed state estimate set can better capture the changes in the state space. Based on the transformed state estimate set, the weighted mean and covariance are calculated to obtain the fused state estimate. The weighted mean combines the state estimates of different models, and the covariance reflects the uncertainty in the state estimate. The fused state estimate combines the information of multiple models and is the best estimate of the target state. The fused state estimate is inversely volumetrically transformed, and it is regressed from the transformed space to the original state space, and finally the prediction error covariance matrix is ​​obtained, which reflects the uncertainty of the fused state estimate.

[0146] In a specific embodiment, the process of executing step S600 may specifically include the following steps:

[0147] Generate a target activity hotspot area according to the target tracking result, and divide the monitoring area into grids based on the target activity hotspot area to obtain a monitoring grid map;

[0148] Priority scoring is performed on each grid unit in the monitoring grid map to obtain a grid priority matrix, and based on the grid priority matrix, adjacent grids are merged to obtain a preliminary monitoring area division result;

[0149] Optimize the boundaries of the preliminary monitoring area division results to obtain optimized monitoring area division results, and construct a regional-environmental feature vector based on the optimized monitoring area division results and environmental change information;

[0150] Generate a power consumption mode clustering result according to the region-environment feature vector, and classify the power consumption mode of each monitoring area based on the power consumption mode clustering result to obtain a regional power consumption mode mapping table;

[0151] Perform global power consumption optimization on the regional power consumption mode mapping table to obtain an optimized power consumption mode configuration, and based on the optimized power consumption mode configuration, perform parameter adjustment on the hardware module of the monitoring device to obtain a second device low power consumption mode, wherein the parameter adjustment includes: adjusting the exposure time and gain of the image sensor, adjusting the operating voltage and frequency of the processor, adjusting the data cache and persistence scheme of the storage module, and adjusting the data transmission strategy of the communication module.

[0152] Specifically, the target's activity trajectory and frequency information are extracted from the target tracking results. This information can reflect the target's activity hotspots in the monitoring area. Activity hotspots refer to areas where targets often appear or move frequently. By analyzing these areas, it can be determined which areas need to be monitored first. By mapping the target's movement trajectory to the monitoring area and counting the number of times the target appears at each location, the target activity hotspot is generated. Assume that the trajectory of the target in a certain period of time is , where each Indicates that the target is at time The target activity hotspot area can be obtained by density estimation of the trajectory. Based on the generated target activity hotspot area, the entire monitoring area is gridded to obtain a monitoring grid map. Gridding is to divide the entire monitoring area into several small units (grid units) of equal size, and each grid unit represents a sub-area of ​​the monitoring area. Assume that the size of the monitoring area is , divided into After the gridding is completed, each grid cell in the monitoring grid map is given a priority score. The priority score is determined based on the activity frequency of the target in each grid cell. For each grid cell , priority score It can be expressed as:

[0153] ;

[0154] Among them, Density Indicates that the target is in the grid cell The activity density in is a mapping function used to convert activity density into priority scores. By scoring all grid cells, a grid priority matrix is ​​obtained , which reflects the importance of each grid cell. Based on the grid priority matrix, adjacent grids are merged to form a larger monitoring area. The merging process usually combines adjacent grid cells with similar priorities, thereby reducing the complexity of monitoring and concentrating resources in important areas. The merged area can be regarded as the preliminary monitoring area division result. After obtaining the preliminary monitoring area division result, boundary optimization is performed to determine a more accurate monitoring area. The goal of boundary optimization is to ensure that the monitoring area is more in line with actual monitoring needs by adjusting the boundaries of the area. For example, boundary optimization can be achieved through morphological operations or algorithms such as minimum enclosing rectangles, so that the optimized monitoring area division results are more reasonable and compact. Based on the optimized monitoring area division results, combined with environmental change information, a regional-environmental feature vector is constructed. Environmental change information includes lighting conditions, weather changes, etc., which can be obtained through sensors or external data sources. Regional-environmental feature vector It can be expressed as:

[0155] ;

[0156] in, Indicates the region-related Features, such as regional priority, ambient light intensity, etc. According to the regional-environmental feature vector, the power consumption mode of the monitoring area is clustered and analyzed to generate the power consumption mode clustering results. Through clustering algorithms, such as K-means, monitoring areas with similar regional-environmental characteristics are classified into the same category, so that corresponding power consumption mode strategies are formulated for each category of areas. A regional power consumption mode mapping table is obtained, which lists the power consumption modes applicable to each monitoring area. Based on the regional power consumption mode mapping table, the global power consumption of the monitoring device is optimized. The goal of global power consumption optimization is to reduce the energy consumption of the device as much as possible while ensuring the quality of monitoring. The optimization process can be achieved through algorithms such as linear programming or dynamic programming. The optimized power consumption mode configuration ensures that the monitoring equipment in each area operates in a suitable power consumption mode, thereby improving the overall energy efficiency. The hardware modules of the monitoring device are adjusted based on the optimized power consumption mode configuration. Parameter adjustment includes adjusting the exposure time and gain of the image sensor to adapt to different lighting conditions; adjusting the operating voltage and frequency of the processor to adapt to different computing loads; adjusting the data cache and persistence scheme of the storage module to reduce unnecessary storage operations; and adjusting the data transmission strategy of the communication module to reduce the energy consumption of data transmission. Through these adjustments, the monitoring device can operate with the lowest energy consumption in different environments, obtaining a second device low power consumption mode.

[0157] See also Figure 2 , Figure 2 A schematic block diagram of the structure of the all-weather humanoid monitoring device 200 of the low-power monitoring device provided in the embodiment of the present application, as shown in FIG. Figure 2 As shown, the all-weather humanoid monitoring device 200 of the low-power monitoring device includes:

[0158] A preprocessing module 210 is used to perform two-step cross non-local hybrid filtering noise reduction and contrast enhancement on the original image collected by the monitoring device to obtain a preprocessed image;

[0159] A matching module 220 is used to perform environment recognition and device power consumption mode matching on the pre-processed image to obtain a first device low power consumption mode of the monitoring device, and perform region detection on the pre-processed image through a region proposal network to generate a target candidate region;

[0160] A fusion module 230 is used to perform multi-scale feature extraction and feature fusion on the target candidate area through a feature extraction network and a feature pyramid network to obtain a multi-scale fusion feature map;

[0161] A detection module 240 is used to input the multi-scale fusion feature map into a human shape detection model of a dual-head detection network structure to perform human shape detection and obtain a human shape detection result;

[0162] The recognition module 250 is used to adopt a multi-model fusion adaptive tracking framework to perform motion state prediction tracking and environmental change recognition on the human detection results to obtain target tracking results and environmental change information;

[0163] The switching module 260 is used to prioritize the monitoring areas and switch the power consumption modes of the monitoring devices according to the target tracking results and the environmental change information, so as to obtain the low power consumption mode of the second device.

[0164] Through the synergy of the above components, the two-step cross non-local hybrid filtering noise reduction and contrast enhancement are used to effectively improve the quality of the original image and improve the adaptability of the system under complex lighting and weather conditions. The introduction of environmental recognition and power consumption mode matching mechanism enables the monitoring equipment to automatically adjust the working mode according to different environmental conditions, achieving a dynamic balance between equipment performance and power consumption. The use of region proposal network combined with residual feature detection network improves the system's detection ability for human targets of different scales, especially in long-distance and small target scenes. Through the combination of feature extraction network and feature pyramid network, the effective extraction and fusion of multi-scale features are achieved, and the model's ability to understand complex scenes is enhanced. The dual-head detection network structure is used to perform classification and regression tasks respectively, which improves the accuracy of human detection and positioning accuracy. The introduction of multi-model fusion adaptive tracking framework improves the system's ability to predict the target's motion state, and can also identify environmental changes, realizing dynamic adjustment of the tracker. Through the priority division of monitoring areas and power consumption mode switching, the reasonable allocation of monitoring resources is achieved, while ensuring the monitoring effect of key areas and reducing the overall power consumption. By dynamically adjusting the parameters of the image sensor, processor, storage module, and communication module, low-power optimization at the hardware level is achieved, extending the working time of the device. This application achieves a comprehensive improvement in detection accuracy, tracking stability, environmental adaptability, and energy efficiency, providing an effective solution for all-weather low-power humanoid monitoring.

[0165] The present application also provides an all-weather humanoid monitoring device of a low-power monitoring device, wherein the all-weather humanoid monitoring device of the low-power monitoring device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the all-weather humanoid monitoring method of the low-power monitoring device in the above-mentioned embodiments.

[0166] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the all-weather humanoid monitoring method of the low-power monitoring device.

[0167] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0169] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for all-weather humanoid monitoring using a low-power monitoring device, characterized in that: include: Perform two-step cross non-local hybrid filtering to reduce noise and enhance contrast on the original image collected by the monitoring equipment to obtain a preprocessed image; Performing environment recognition and device power consumption mode matching on the preprocessed image to obtain a first device low power consumption mode of the monitoring device, and performing region detection on the preprocessed image through a region proposal network to generate a target candidate region; Through a feature extraction network and a feature pyramid network, multi-scale feature extraction and feature fusion are performed on the target candidate area to obtain a multi-scale fusion feature map; Inputting the multi-scale fusion feature map into a human shape detection model with a dual-head detection network structure to perform human shape detection to obtain a human shape detection result; Adopting a multi-model fusion adaptive tracking framework, the motion state prediction tracking and environmental change recognition of the human detection result are performed to obtain the target tracking result and environmental change information; According to the target tracking result and the environmental change information, the monitoring area priority is divided and the power consumption mode is switched for the monitoring device to obtain a low power consumption mode of the second device.

2. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 1, characterized in that: The method of performing two-step cross non-local hybrid filtering noise reduction and contrast enhancement on the original image collected by the monitoring device to obtain a pre-processed image includes: Positioning pixel grayscale extreme points of the original image to obtain a grayscale extreme point set, and performing non-local median filtering on the original image based on the grayscale extreme point set to obtain a preliminary denoised image; Calculating a difference integral for the preliminary denoised image to obtain a difference integral map, and based on the difference integral map, performing accelerated non-mean filtering on the preliminary denoised image to obtain a cross-filtered image; Performing overall noise reduction processing on the cross-filtered image to obtain a noise-reduced image, and performing image histogram calculation based on the noise-reduced image to obtain grayscale distribution information; According to the grayscale distribution information, the denoised image is subjected to histogram equalization processing to obtain an equalized image, and the equalized image is subjected to adaptive contrast stretching to obtain a contrast enhanced image; An edge-preserving smoothing process is performed on the contrast-enhanced image to obtain an edge-enhanced image, and image brightness and contrast are adjusted on the edge-enhanced image to obtain a pre-processed image.

3. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 1, characterized in that: The performing environment recognition and device power consumption mode matching on the pre-processed image to obtain a first device low power consumption mode of the monitoring device, and performing region detection on the pre-processed image through a region proposal network to generate a target candidate region, including: Extracting environmental features from the preprocessed image to obtain an initial environmental feature map, and calculating feature weights based on the initial environmental feature map to obtain a weighted environmental feature map; Inputting the weighted environment feature map into an environment classifier, classifying the current environment to obtain an environment type, and selecting matching power consumption mode parameters from a preset power consumption mode library according to the environment type to obtain a first device low power consumption mode of the monitoring device; Generate a target candidate region for the preprocessed image through a region proposal network, wherein the region proposal network includes a residual feature detection network, a region generation network and a region optimization network; The residual feature detection network includes 5 stages, and the processing process of each stage is as follows: Stage 1: Perform a 7×7 convolution operation on the preprocessed image with a step size of 2 and an output channel number of 64, and then pass through a batch normalization layer and a rectified linear unit activation function to obtain a first residual feature map; Stage 2: Perform a maximum pooling operation on the first residual feature map with a step size of 2, and then pass through 3 residual blocks, each residual block contains three convolution layers of 1×1, 3×3, and 1×1, and the output channel numbers are 64, 64, and 256 respectively, to obtain a second residual feature map ; Stage 3: Apply 4 residual blocks to the second residual feature map, introduce a deformable convolution module in the 3×3 convolution layer of each residual block, and the number of output channels is 512 to obtain a third residual feature map; Stage 4: Apply 6 residual blocks to the third residual feature map, and also use deformable convolution in the 3×3 convolution layer, and the number of output channels is 1024 to obtain a fourth residual feature map; Stage 5: Apply 3 residual blocks to the fourth residual feature map, keep the use of deformable convolution, and the number of output channels is 2048 to obtain a fifth residual feature map; The region generation network processes the third residual feature map, the fourth residual feature map and the fifth residual feature map, and the specific steps include: performing a 1×1 convolution operation on the third residual feature map, the fourth residual feature map and the fifth residual feature map respectively, and uniformly reducing the number of channels to 256; performing one upsampling on the fifth residual feature map, and performing element-level addition fusion with the fourth residual feature map to obtain a fusion result; upsampling the fusion result again, and performing element-level addition fusion with the third residual feature map to obtain a target residual feature map; performing a 3×3 convolution operation on the target residual feature map to obtain a feature map for generating an anchor frame; generating anchor frames of 9 scales and 3 ratios at each position of the feature map for generating an anchor frame, a total of 27 anchor frames; applying two parallel fully connected layers to each anchor frame, one for bounding box regression and the other for binary classification, to obtain an initial candidate region; The region optimization network optimizes the initial candidate regions, removes candidate regions with high overlap, introduces a spatial attention mechanism to weight the features of the candidate regions, dynamically adjusts the screening threshold and selects the top K as the final target candidate regions.

4. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 1, characterized in that: The method of performing multi-scale feature extraction and feature fusion on the target candidate region through the feature extraction network and the feature pyramid network to obtain a multi-scale fusion feature map includes: The target candidate region is input into the feature extraction network, which includes 5 stages. The processing process of each stage is as follows: Stage 1: Perform a 7×7 convolution operation on the target candidate region with a step size of 2 and an output channel number of 64, and then pass through a batch normalization layer and a ReLU activation function to obtain a first region feature map; Stage 2: Perform a maximum pooling operation on the first region feature map with a step size of 2, and then pass through 3 group convolution blocks with a base of 32, each block contains three convolution layers of 1×1, 3×3, and 1×1, and the output channel numbers are 128, 128, and 256 respectively, to obtain a second region feature map ; Stage 3: Apply 4 group convolution blocks with a base number of 32 to the second regional feature map, introduce a deformable convolution module in the 3×3 convolution layer of each block, and the number of output channels is 512 to obtain a third regional feature map; Stage 4: Apply 23 group convolution blocks with a base number of 32 to the third regional feature map, and also use deformable convolution in the 3×3 convolution layer, and the number of output channels is 1024 to obtain a fourth regional feature map; Stage 5: Apply 3 group convolution blocks with a base number of 32 to the fourth regional feature map, keep using deformable convolution, and the number of output channels is 2048 to obtain a fifth regional feature map; A feature pyramid network is constructed based on the third region feature map, the fourth region feature map, and the fifth region feature map, and the specific steps include: performing a 1×1 convolution operation on the fifth region feature map to reduce the number of channels to 256 to obtain a first pyramid feature map; performing 2-fold upsampling on the first pyramid feature map, performing element-level addition fusion with the fourth region feature map after 1×1 convolution dimension reduction to obtain a second pyramid feature map; performing 2-fold upsampling on the second pyramid feature map, performing element-level addition fusion with the third region feature map after 1×1 convolution dimension reduction to obtain a third pyramid feature map; performing 2-fold downsampling on the third pyramid feature map to obtain a fourth pyramid feature map; performing a 3×3 convolution operation on the fourth pyramid feature map with a step size of 2 to obtain a fifth pyramid feature map; performing 3×3 convolution operations on the third pyramid feature map, the fourth pyramid feature map, and the fifth pyramid feature map respectively, unifying the number of channels to 256, and obtaining a multi-scale feature set; Based on the feature maps in the multi-scale feature set, a bidirectional feature pyramid network is constructed, comprising the following steps: a top-down path: starting from the fifth pyramid feature map, downsampling step by step and fusing with the next-level feature map to obtain an enhanced feature set; a bottom-up path: starting from the bottom-level feature map of the enhanced feature set, upsampling step by step and fusing with the previous-level feature map to obtain a refined feature set; performing weighted feature fusion on each fusion operation, and at the same time, performing channel attention mechanism analysis and spatial attention mechanism analysis on the refined feature set to obtain an attention-enhanced feature set; Performing element-by-element addition of cross-scale feature aggregation on the attention enhancement feature set to obtain an aggregated feature map, and performing multi-scale attention analysis based on the aggregated feature map to obtain an attention enhancement feature map; Adaptive feature selection is performed on the attention enhancement feature map to obtain a selective feature map, and multi-scale feature reconstruction is performed on the selective feature map to obtain a multi-scale fusion feature map.

5. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 1, characterized in that: The step of inputting the multi-scale fusion feature map into a human shape detection model of a dual-head detection network structure to perform human shape detection and obtain a human shape detection result includes: Unifying the candidate region features of different sizes in the multi-scale fusion feature map to a fixed size to obtain a normalized feature map; Input the normalized feature map into a human figure detection model of a dual-head detection network structure, wherein the human figure detection model of the dual-head detection network structure includes a classification head and a regression head, wherein: the classification head includes two fully connected layers, the output dimension of the first classification fully connected layer is 1024, and the output dimension of the second classification fully connected layer is the number of categories plus 1; the regression head includes two fully connected layers, the output dimension of the first regression fully connected layer is 1024, and the output dimension of the second regression fully connected layer is 4 times the number of categories; Performing a category probability analysis on the normalized feature map through the classification head to obtain a category probability distribution, and performing a bounding box regression analysis on the normalized feature map through the regression head to obtain a bounding box regression parameter; Performing multi-scale context perception based on the category probability distribution and the bounding box regression parameters to obtain multi-scale context features, and performing attention-guided refinement processing based on the multi-scale context features to obtain attention refinement features; Performing adaptive threshold non-maximum suppression on the attention refinement features to obtain a preliminary screening result, and performing multi-scale cascade calibration on the preliminary screening result to obtain a calibrated detection frame set; Context-aware post-processing is performed based on the calibrated detection frame set to obtain an optimized detection result, and temporal consistency constraints are performed on the optimized detection result to obtain a human detection result.

6. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 5, characterized in that: The performing category probability analysis on the normalized feature map by the classification head to obtain category probability distribution, and performing bounding box regression analysis on the normalized feature map by the regression head to obtain bounding box regression parameters, includes: Inputting the normalized feature map into the first classification fully connected layer for classification processing to obtain preliminary classification features, and performing modified linear unit activation function and random dropout processing on the preliminary classification features to obtain intermediate classification features; Inputting the intermediate classification features into the second classification fully connected layer to calculate the classification score to obtain the original category score, and performing a Softmax function operation on the original category score to obtain a category probability distribution; Inputting the normalized feature map into the first regression fully connected layer for regression analysis to obtain preliminary regression features, and performing hyperbolic tangent activation function and batch normalization processing on the preliminary regression features to obtain intermediate regression features; Inputting the intermediate regression features into the second regression fully connected layer to calculate bounding box parameters to obtain original bounding box parameters, and performing an exponential linear unit activation function operation on the original bounding box parameters to obtain a bounding box offset; The candidate region is subjected to coordinate transformation based on the bounding box offset to obtain adjusted bounding box coordinates, and the adjusted bounding box coordinates are subjected to smoothing to obtain bounding box regression parameters.

7. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 1, characterized in that: The adaptive tracking framework using multi-model fusion performs motion state prediction tracking and environmental change recognition on the human detection result to obtain the target tracking result and environmental change information, including: Inputting the human detection result into a multi-model interactive filter for state estimation to obtain an initial state estimation set, and performing cubature Kalman filter prediction on each initial state vector in the initial state estimation set to obtain a prediction error covariance matrix; Performing singular value decomposition on the prediction error covariance matrix to obtain a reconstructed error covariance matrix, and calculating the likelihood function value and model probability of each model based on the reconstructed error covariance matrix to obtain a fusion state estimate; Input the fusion state estimation of the past N frames into the long short-term memory network to obtain the predicted state sequence of the future M frames, and calculate the adaptive weight based on the predicted state sequence and the current observation state to obtain the fusion tracking result; Performing feature extraction and sliding window similarity calculation on the area around the fused tracking result to obtain the type of environmental change, and adjusting the key parameters of the tracker according to the type of environmental change and the tracking performance index to obtain updated tracker parameters; Based on the updated tracker parameters, target verification and local search strategy are performed on the fused tracking result to obtain the target tracking result and environment change information.

8. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 7, characterized in that: The human detection result is input into a multi-model interactive filter for state estimation to obtain an initial state estimation set, and each initial state vector in the initial state estimation set is predicted by a volumetric Kalman filter to obtain a prediction error covariance matrix, including: The human detection results are respectively input into the Singer model, the current statistical model and the Jerk model in the multi-model interaction filter for processing to obtain three initial state vectors; Performing model interaction on the three initial state vectors to obtain a mixed initial state, wherein the model interaction process uses a Markov chain transfer probability matrix; Based on the mixed initial state, each model is predicted to obtain a predicted state set, and the observation equation is solved for the predicted state set to obtain a predicted observation set; Based on the predicted observation set and the actual observation value, the observation residual is calculated to obtain a residual sequence, and the residual sequence is subjected to adaptive factor analysis to obtain a corrected observation noise covariance matrix; Based on the modified observation noise covariance matrix, the Kalman gain is calculated to obtain a state update matrix, and the state update matrix is ​​volume transformed to obtain a transformed state estimation set; Based on the transformed state estimation set, a weighted mean and a covariance are calculated to obtain a fused state estimation, and an inverse volume transformation is performed on the fused state estimation to obtain a prediction error covariance matrix.

9. The all-weather humanoid monitoring method of low-power monitoring equipment according to claim 1, characterized in that: The step of performing monitoring area priority division and power consumption mode switching on the monitoring device according to the target tracking result and the environmental change information to obtain a low power consumption mode of the second device includes: Generate a target activity hotspot area according to the target tracking result, and divide the monitoring area into grids based on the target activity hotspot area to obtain a monitoring grid map; Prioritizing each grid unit in the monitoring grid map to obtain a grid priority matrix, and merging adjacent grids based on the grid priority matrix to obtain a preliminary monitoring area division result; Performing boundary optimization on the preliminary monitoring area division result to obtain an optimized monitoring area division result, and constructing a region-environment feature vector based on the optimized monitoring area division result and the environmental change information; Generate a power consumption mode clustering result according to the region-environment feature vector, and classify the power consumption mode of each monitoring area based on the power consumption mode clustering result to obtain a regional power consumption mode mapping table; The regional power consumption mode mapping table is globally optimized to obtain an optimized power consumption mode configuration, and based on the optimized power consumption mode configuration, the hardware module of the monitoring device is parameter adjusted to obtain a second device low power consumption mode, wherein the parameter adjustment includes: adjusting the exposure time and gain of the image sensor, adjusting the operating voltage and frequency of the processor, adjusting the data cache and persistence scheme of the storage module, and adjusting the data transmission strategy of the communication module.

10. An all-weather humanoid monitoring device for low-power monitoring equipment, characterized in that: The all-weather humanoid monitoring method for executing the low-power monitoring device according to any one of claims 1 to 9 comprises: A preprocessing module is used to perform two-step cross non-local hybrid filtering noise reduction and contrast enhancement on the original image collected by the monitoring device to obtain a preprocessed image; A matching module, configured to perform environment recognition and device power consumption mode matching on the pre-processed image to obtain a first device low power consumption mode of the monitoring device, and perform region detection on the pre-processed image through a region proposal network to generate a target candidate region; A fusion module is used to perform multi-scale feature extraction and feature fusion on the target candidate area through a feature extraction network and a feature pyramid network to obtain a multi-scale fusion feature map; A detection module, used for inputting the multi-scale fusion feature map into a human shape detection model of a dual-head detection network structure to perform human shape detection and obtain a human shape detection result; A recognition module is used to adopt a multi-model fusion adaptive tracking framework to perform motion state prediction tracking and environmental change recognition on the human detection result to obtain a target tracking result and environmental change information; The switching module is used to prioritize the monitoring areas and switch the power consumption modes of the monitoring device according to the target tracking result and the environmental change information, so as to obtain a low power consumption mode of the second device.

Images