Low-power-consumption monitoring awakening method, device and equipment based on human shape recognition

By downsampling and grayscale preprocessing of low-power monitoring devices, humanoid features are extracted and recognition models are trained. Combined with multi-mode adaptive strategies and incremental learning, traditional equipment has solved the accuracy and energy efficiency of humanoid recognition. The problem is to achieve efficient and accurate humanoid recognition and low power consumption operation.

CN118982847AActive Publication Date: 2024-11-19SHENZHEN ANKED SHITONG ELECTRONICS CO LTD

Patent Information

Application Number
CN202411220085.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-19
Estimated Expiration
2044-09-02

AI Technical Summary

Technical Problem

Traditional low-power monitoring equipment is difficult to dynamically adjust its working state, resulting in low energy utilization efficiency and insufficient accuracy and real-time performance in humanoid target recognition, which is prone to false alarms and missed alarms, affecting the monitoring effect.

Method used

By downsampling and grayscale preprocessing of the original image sequence, combined with background differential technology, humanoid features are extracted and initial humanoid recognition model is trained. Multi-scale feature extraction and weighted voting mechanism are adopted to generate multi-mode adaptive monitoring strategies and dynamically adjust The device's working mode and the model is updated through incremental learning to realize multi-level wake-up control strategy.

Benefits of technology

It improves the energy utilization efficiency of monitoring equipment, improves the accuracy and real-timeness of humanoid recognition, and provides a more reliable security solution that can achieve efficient and accurate humanoid recognition and low-power operation in complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982847A_ABST
    Figure CN118982847A_ABST
Patent Text Reader

Abstract

The invention relates to a low-power-consumption monitoring awakening method, device and equipment based on human shape recognition. The method comprises the steps of performing downsampling and background differencing on an original image sequence acquired by low-power-consumption monitoring equipment to obtain a preliminary target mask; carrying out human shape feature extraction to obtain a human shape region false label; training to obtain an initial human shape recognition model and carrying out multi-scale feature extraction to obtain a target feature vector; performing classification to obtain a preliminary human shape recognition result, and performing optimization to obtain a human shape target tracking sequence; a scene activeness index is calculated, a multi-mode adaptive monitoring strategy is generated, and a low-power-consumption working scheme is adjusted; and performing incremental learning to obtain a target human shape recognition model, and generating a multi-stage wake-up control strategy. According to the embodiment of the invention, the energy utilization efficiency of the monitoring equipment is improved, the service life of a battery is prolonged, the accuracy and real-time performance of monitoring can be improved, and a more reliable and intelligent security solution is provided for a user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of human figure recognition, and in particular to a low-power monitoring wake-up method, device and equipment based on human figure recognition. Background Art

[0002] With the rapid development of the Internet of Things and smart home technologies, low-power monitoring devices have been widely used in security, home monitoring and other fields. However, traditional low-power monitoring devices often adopt fixed working modes, which makes it difficult to dynamically adjust the working state according to the actual scene requirements, resulting in low energy efficiency. At the same time, these devices are also insufficient in accuracy and real-time performance in human target recognition, and often have false alarms and missed alarms, affecting the monitoring effect.

[0003] In addition, most existing monitoring wake-up methods rely on simple motion detection or fixed threshold triggers, which cannot effectively distinguish between human targets and other moving objects. They are easily disturbed by the environment and wake up frequently, increasing unnecessary energy consumption. In complex and changeable actual monitoring scenarios, how to ensure recognition accuracy while achieving low power consumption has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The objective of the present invention is to provide a low-power monitoring wake-up method, device and equipment based on human figure recognition, so as to improve the energy utilization efficiency of the monitoring equipment, extend the battery life, and improve the accuracy and real-time performance of monitoring, so as to provide users with a more reliable and intelligent security solution.

[0005] To achieve the above object, the present invention provides a low-power monitoring wake-up method based on human figure recognition, comprising the following steps:

[0006] The original image sequence collected by the low-power monitoring device is downsampled and grayed to obtain a preprocessed image and perform background subtraction to obtain a preliminary target mask;

[0007] Extracting human features from the preliminary target mask to obtain a candidate human figure region and calculating a shape descriptor to obtain a pseudo label of the human figure region;

[0008] Based on the pseudo-label training of the human shape region, an initial human shape recognition model is obtained and multi-scale feature extraction is performed to obtain a local spatial depth feature map and a target feature vector;

[0009] Classifying and weighted voting the target feature vectors to obtain preliminary human recognition results, and optimizing the recognition results of consecutive frames to obtain a human target tracking sequence;

[0010] Calculating a scene activity index according to the humanoid target tracking sequence, generating a multi-mode adaptive monitoring strategy, and dynamically adjusting a low-power working scheme of the low-power monitoring device based on the multi-mode adaptive monitoring strategy;

[0011] Incremental learning and model updating are performed according to the humanoid target tracking sequence to obtain a target humanoid recognition model, and a multi-level wake-up control strategy is generated based on the target humanoid recognition model and the low-power working scheme.

[0012] The present invention also provides a low-power monitoring and awakening device based on human figure recognition, comprising:

[0013] The preprocessing module is used to downsample and grayscale the original image sequence collected by the low-power monitoring device to obtain a preprocessed image and perform background difference to obtain a preliminary target mask;

[0014] A calculation module is used to extract human features from the preliminary target mask, obtain a candidate human figure region, and calculate a shape descriptor to obtain a pseudo label of the human figure region;

[0015] A training module, used to obtain an initial human figure recognition model based on the human figure region pseudo-label training and perform multi-scale feature extraction to obtain a local spatial depth feature map and a target feature vector;

[0016] A classification module is used to classify and weighted vote the target feature vectors to obtain preliminary human recognition results, and optimize the recognition results of consecutive frames to obtain a human target tracking sequence;

[0017] An adjustment module, configured to calculate a scene activity index according to the humanoid target tracking sequence, generate a multi-mode adaptive monitoring strategy, and dynamically adjust a low-power working scheme of the low-power monitoring device based on the multi-mode adaptive monitoring strategy;

[0018] A generation module is used to perform incremental learning and model update according to the humanoid target tracking sequence to obtain a target humanoid recognition model, and generate a multi-level wake-up control strategy based on the target humanoid recognition model and the low-power working scheme.

[0019] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0020] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0021] In summary, the technical solution provided by the present invention can effectively reduce the amount of data processing and the computational complexity by downsampling and grayscale preprocessing the original image sequence, combined with background difference technology, and provide high-quality preliminary target masks for subsequent human recognition. The multi-scale feature extraction and spatial pyramid pooling modules, combined with discrete wavelet transform, can capture the multi-level feature information of human targets and improve the robustness and generalization ability of the recognition model. The weighted voting mechanism and spatiotemporal consistency analysis are introduced to effectively reduce the error of single-frame recognition and improve the accuracy and stability of human target tracking. Based on multi-dimensional information such as scene activity indicators, lighting conditions and device power, an adaptive monitoring strategy is generated to achieve dynamic optimization of the working mode of the monitoring equipment, while ensuring the monitoring effect and maximizing the energy saving potential. Through incremental learning and model update mechanisms, the human recognition model can continuously adapt to new scenes and target features, and improve the long-term operation performance of the system. The design of the multi-level wake-up control strategy realizes a progressive wake-up process from low energy consumption to high performance, while ensuring fast response and avoiding unnecessary full-power operation, further optimizing energy utilization efficiency. The present invention can realize efficient and accurate human figure recognition and low-power operation in complex and changeable practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the steps of a low-power monitoring wake-up method based on human figure recognition in one embodiment of the present invention;

[0023] Figure 2 It is a structural block diagram of a low-power monitoring and awakening device based on human figure recognition in one embodiment of the present invention;

[0024] Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0025] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0027] Reference Figure 1 This embodiment provides a low-power monitoring wake-up method based on human figure recognition, comprising the following steps:

[0028] S1, downsample and grayscale the original image sequence collected by the low-power monitoring device to obtain a preprocessed image and perform background difference to obtain a preliminary target mask;

[0029] Specifically, the resolution of the original image sequence collected by the low-power monitoring device is reduced, the downsampled image is obtained by downsampling, and the downsampled image is gray-scale converted to generate a grayscale image. The grayscale image is smoothed to eliminate high-frequency noise and fine textures in the image, enhance the smoothness of the image, and obtain a preprocessed image. The preprocessed image is edge-enhanced to highlight the edge information in the image, improve the outline clarity of the object in the image, and generate a preprocessed image sequence containing more obvious edge features. Multiple frames of images are selected from the generated preprocessed image sequence, and the pixels of these images are averaged to generate a background reference image. The background reference image represents the static background part of the scene and is used for comparison with the newly input preprocessed image. The newly input preprocessed image is differentially operated with the background reference image to obtain a differential image, revealing the possible moving objects or changing areas in the image. The differential image is thresholded, a first threshold is set, and pixels in the differential image greater than the threshold are marked as foreground areas, and a binary differential image is generated, which contains possible foreground objects. The binary difference image is subjected to noise removal processing to eliminate isolated points and small areas in the image, thereby optimizing the moving area image. The optimized moving area image is subjected to edge detection, and the target contour image is obtained by identifying the contour of the object in the image, and the target contour line segment in the image is extracted. The target contour line segment is compared with the preset human contour template, and the similarity score between the contour line segment and the template is calculated to generate a human feature description. The human feature description is compared with the second threshold. Only when the similarity score exceeds the second threshold, the corresponding area is identified as a target area that may contain a human figure, and finally a preliminary target mask is obtained by screening.

[0030] S2, extract human features from the preliminary target mask, obtain candidate human regions and calculate shape descriptors to obtain pseudo labels for human regions;

[0031] Specifically, the preliminary target mask is bounded by a rectangle to define a candidate rectangular region that may contain a human figure. According to the preset human body proportion range, the candidate rectangular regions are screened to exclude those regions that do not conform to the human body proportions to obtain a preliminary human figure region. Feature extraction is performed on the preliminary human figure region, and its gradient features are extracted to generate a gradient direction histogram. The gradient direction histogram reflects the distribution of pixel gradient directions in the region. By analyzing the distribution features, the feature similarity is calculated to obtain a probability score that measures whether the region is a human figure. The score is used to preliminarily judge the probability of a human figure appearing in the candidate region. Based on the human figure possibility score, the preliminary human figure region is screened again to obtain a more refined and more likely human figure region. The overlapping regions of the refined human figure regions are merged to obtain a unified candidate human figure region. Shape description data is extracted from the candidate human figure region, and the obtained shape description data is used to represent the geometric shape features of the region. Feature compression is performed on the shape description data to generate a compact shape descriptor. The compact shape descriptor is matched. The descriptor is matched with a preset standard shape template to obtain a shape matching result. According to the matching result, the candidate regions are sorted by similarity to identify the best matching shape. The best matching shape usually refers to the shape descriptor that is closest to the standard human shape template, representing the part of the candidate area that is most likely to be a human shape. Combining the best matching shape with the previously calculated human shape possibility score, a confidence score is calculated to generate a confidence map for the entire human shape area. The confidence map reflects the possibility that each area in the image is considered to be a human shape. The human shape area confidence map is expanded to cover and capture the complete human shape area through the expansion operation, ensuring that all parts that may contain human shapes are taken into consideration. A feature description vector is extracted from the complete human shape area. The vector contains all important features related to the human shape in the area. By screening and analyzing the feature description vector, it is determined which features meet the preset standards, and finally the corresponding human shape area pseudo-label is generated.

[0032] S3, based on the pseudo-label training of the human shape area, obtains the initial human shape recognition model and performs multi-scale feature extraction to obtain the local spatial depth feature map and target feature vector;

[0033] Specifically, data enhancement processing is performed on the pseudo labels of the humanoid region to expand the diversity of the training data and obtain a more representative expanded training set. The expanded training set is standardized to ensure the consistency and stability of the input data and obtain standardized training data. In terms of model construction, a lightweight convolutional neural network backbone containing 5 convolutional blocks is designed. Each convolutional block consists of a convolutional layer, a batch normalization layer, and a ReLU activation function layer. This structure can effectively extract local features of the image while keeping the model lightweight, which is suitable for real-time processing requirements of low-power devices. Through the convolutional neural network backbone, feature extraction is performed on the input preprocessed image to obtain a preliminary feature map. A discrete wavelet transform layer is inserted between the third and fourth convolutional blocks of the convolutional neural network backbone. By performing discrete wavelet transform on the feature map after the third convolutional block, the feature map is decomposed into low-frequency approximation coefficients and high-frequency detail coefficients. The low-frequency approximation coefficient mainly contains the overall information of the image, while the high-frequency detail coefficient retains the edge and texture information of the image. The multi-scale feature map is obtained by adaptively weighting and fusing these two coefficients. The multi-scale feature map is input into the fourth convolutional block to obtain an enhanced feature representation. A spatial pyramid pooling module is added after the last convolutional block of the convolutional neural network backbone. This module generates feature vectors of different scales by multi-scale pooling the output feature map of the last convolutional block. A complete multi-scale feature representation is generated by splicing feature vectors of different scales, and the multi-scale feature representation is reduced in dimension by a 1x1 convolution operation to obtain a compressed feature vector. A fully connected layer network consisting of two fully connected layers and one Dropout layer is constructed, and the compressed feature vector is input into the fully connected layer network to generate the final prediction result. The error between the prediction result and the true label is calculated by using the cross entropy loss function to obtain a loss value. The network parameters are optimized by the back propagation algorithm, and the network parameters are updated until the loss value converges or the preset number of iterations is reached to obtain the initial human recognition model. After the training of the initial human recognition model is completed, the new pre-processed image is input into the model, and the local spatial depth feature map is generated after being processed by all convolutional blocks, discrete wavelet transform layers and spatial pyramid pooling modules. Perform global average pooling operation on the local spatial depth feature map to extract the target feature vector.

[0034] S4, classify and weighted vote the target feature vector to obtain the preliminary human recognition result, and optimize the recognition results of consecutive frames to obtain the human target tracking sequence;

[0035] Specifically, the target feature vector is normalized, and the value range of the feature vector is adjusted to a standard range to eliminate the magnitude difference between different features. The normalized standardized feature vector is input into multiple pre-configured kernelized extreme learning machines, and each kernelized extreme learning machine generates a set of classification results. The classification results of different kernelized extreme learning machines are weighted averaged to obtain a comprehensive classification score. According to the preset classification threshold, the comprehensive classification score is binarized to obtain a preliminary human recognition result, which indicates the monitoring system's preliminary judgment of the human target in the current frame image. The preliminary human recognition results of multiple consecutive frames are analyzed in time series to evaluate the temporal consistency of the human target in multiple consecutive frames and calculate a temporal consistency score. The single-frame recognition result is corrected by the temporal consistency score to obtain the recognition result after temporal smoothing, reducing the error caused by instantaneous changes or noise. The spatial connectivity analysis is performed on the recognition result after temporal smoothing to evaluate the spatial consistency of the recognition result and calculate the spatial consistency score. By combining the spatial consistency score and the recognition result after temporal smoothing, a spatiotemporally consistent humanoid region is obtained, which represents the region in the surveillance video that is stably recognized as a humanoid by the system. The bounding box of the spatiotemporally consistent humanoid region is extracted to determine the specific position and size information of the humanoid target. This information is used to construct the target state vector, which contains the position, size and other relevant features of the humanoid target in the current frame, thereby obtaining an initial tracking state. In order to predict the position of the humanoid target in the next frame, the Kalman filter is used to predict the initial tracking state. The Kalman filter is a classic state estimation tool that estimates the current state and infers the predicted position of the next frame. The predicted position is associated with the actual recognition result of the next frame to obtain the updated tracking state, thereby achieving continuous tracking of the humanoid target in the video sequence. The updated tracking state is constrained by motion consistency, and abnormal tracking results that do not conform to the conventional motion law are eliminated according to reasonable motion patterns. In this way, abnormal motion trajectories caused by noise or recognition errors are eliminated to obtain an optimized tracking sequence. The optimized tracking sequence is smoothed to generate a humanoid target tracking sequence.

[0036] S5, calculating the scene activity index according to the humanoid target tracking sequence, generating a multi-mode adaptive monitoring strategy, and dynamically adjusting the low-power working scheme of the low-power monitoring device based on the multi-mode adaptive monitoring strategy;

[0037] Specifically, the number of targets in the humanoid target tracking sequence is counted to obtain the frequency of target appearance per unit time. The crowd density index of the scene is calculated based on the target appearance frequency, and the crowd density score is obtained based on the index to reflect the density of crowd activities in the monitoring scene. At the same time, the target movement speed in the humanoid target tracking sequence is analyzed to obtain the average movement speed and the speed change rate. These data are used to calculate the dynamic index of the scene to obtain the scene dynamic score, which reflects the movement activity of the target in the monitoring scene. The crowd density score is combined with the scene dynamic score, and a comprehensive scene activity index is calculated by the weighted summation method. The comprehensive index can fully reflect the overall activity state of the current monitoring scene. The light intensity data of the current environment is collected to obtain the light intensity value. According to the light intensity value, the appropriate image enhancement parameters are selected, and the light adaptive processing strategy is formulated. The strategy can optimize the image quality under the condition of light changes and ensure the stability of the monitoring effect. At the same time, the current power information of the device is read to obtain the power percentage. According to the power percentage, the corresponding power consumption control level is set, and the power adaptive control strategy is generated to ensure that the device can reasonably adjust the power consumption and extend the running time under low power conditions. Based on the scene activity index, light adaptive processing strategy and power adaptive control strategy, a multi-mode adaptive monitoring strategy is generated. This strategy intelligently adjusts the working mode of the monitoring device according to different scene requirements and device status. The image acquisition frequency is set according to the multi-mode adaptive monitoring strategy to form a dynamic sampling scheme, and the working parameters of the image sensor are adjusted according to the scheme to obtain the optimized image acquisition strategy. This strategy can reduce the power consumption during the image acquisition process while ensuring the monitoring effect. According to the multi-mode adaptive monitoring strategy, the appropriate image processing complexity is selected to generate an algorithm complexity control scheme. Combining the algorithm complexity control scheme and the optimized image acquisition strategy, a low-power working scheme for low-power monitoring equipment is formulated. This working scheme flexibly controls the device power consumption in different scenarios by dynamically adjusting the device's image acquisition frequency and processing complexity, achieving a balance between efficient monitoring and energy consumption management.

[0038] S6, performing incremental learning and model updating according to the humanoid target tracking sequence to obtain a target humanoid recognition model, and generating a multi-level wake-up control strategy based on the target humanoid recognition model and a low-power working scheme.

[0039] Specifically, the target images in the human target tracking sequence are cropped and aligned to obtain standardized target samples to ensure consistency in subsequent processing. Feature extraction is performed on the standardized target samples to generate a new sample feature set. The new sample feature set is compared with the historical sample feature library, and the novelty of the sample is evaluated through the comparison process. Based on the sample novelty evaluation results, a sample set suitable for incremental learning is screened out. The incremental learning sample set represents the new or changed target features in the current monitoring scene, which are used to optimize the existing recognition model. Based on the screened incremental learning sample set, the last layer of the initial human recognition model is fine-tuned to obtain the updated model parameters. To ensure the smoothness of the model update, the updated model parameters are weightedly fused with the original model parameters to generate a temporary update model. The performance of the temporary update model is evaluated, and the recognition accuracy and computational complexity are calculated to obtain the model update evaluation indicators. These evaluation indicators are used to determine whether the model update is effective. Whether to accept the update is determined based on the evaluation results. If the update results meet the expected standards, the final target human recognition model is generated. Based on the target humanoid recognition model, multiple confidence thresholds are set to generate multi-level humanoid recognition judgment criteria. These judgment criteria allow the system to quickly screen real-time images according to different confidence levels and generate preliminary wake-up trigger signals. Combined with the scene activity index in the low-power working scheme, the preliminary wake-up trigger signal is weighted to obtain a scene-adaptive wake-up score. Multi-level wake-up thresholds are set based on the scene-adaptive wake-up score to achieve flexible response to different scenes. According to the multi-level wake-up threshold, the wake-up process of the device is divided into multiple energy consumption levels to form a step-by-step wake-up strategy. This strategy minimizes energy consumption while ensuring the effectiveness of monitoring through hierarchical wake-up. The step-by-step wake-up strategy is energy-optimized to ensure balanced energy distribution at different energy consumption levels, and an energy-balanced multi-level wake-up scheme is obtained. The energy-balanced multi-level wake-up scheme is combined with the low-power working scheme to generate a comprehensive multi-level wake-up control strategy. This strategy can intelligently adjust the wake-up and working mode of the device according to the real-time changes of the monitoring scene and the power consumption status of the device to achieve efficient monitoring under low power consumption.

[0040] In one example, the original image sequence collected by the low-power monitoring device is downsampled and grayed to obtain a preprocessed image and perform background subtraction to obtain a preliminary target mask, including:

[0041] The resolution of the original image sequence collected by the low-power monitoring device is reduced to obtain a downsampled image, and the downsampled image is gray-converted to obtain a grayscale image;

[0042] Performing image smoothing on the grayscale image to obtain a preprocessed image, and performing edge enhancement processing on the preprocessed image to obtain a preprocessed image sequence;

[0043] Select multiple frames of images from the preprocessed image sequence and perform pixel-level averaging to obtain a background reference image, and perform a difference operation between the newly input preprocessed image and the background reference image to obtain a difference image;

[0044] Performing threshold processing on the difference image, marking pixels greater than a first threshold as foreground, obtaining a binary difference image, and performing noise removal processing on the binary difference image to obtain an optimized motion area image;

[0045] Perform edge detection on the optimized motion region image to obtain a target contour image, and extract target contour line segments of the target contour image;

[0046] The target contour segment is compared with the preset human contour template, the similarity score is calculated, the human feature description is obtained, and the human feature description is compared with the second threshold to screen and obtain a preliminary target mask.

[0047] In this example, the original image sequence collected by the low-power monitoring device is subjected to a resolution reduction operation to obtain a downsampled image, thereby reducing the number of pixels in the image, thereby reducing the computational complexity and reducing the consumption of device resources. Resolution reduction can be achieved by downsampling the original image, that is, by averaging the color values ​​of adjacent pixels or directly selecting one of the pixel values ​​to represent a larger area, thereby reducing the overall resolution of the image. For example, assuming the resolution of the original image is , by the downsampling factor , the resulting downsampled image resolution will become . Perform grayscale conversion on the downsampled image to generate a grayscale image. Simplify the color image into a single-channel grayscale image to reduce the amount of calculation. The usual grayscale conversion method is to use the weighted average method. For example, the grayscale value can be calculated using the following formula:

[0048]

[0049] in, is the gray value, are the pixel values ​​of the red, green and blue channels in the image respectively. Through this formula, each color pixel is converted into a corresponding grayscale value to generate a single-channel grayscale image. The grayscale image is smoothed. The commonly used smoothing method is Gaussian blur. Gaussian blur uses convolution operation to make the value of each pixel become the weighted average of the surrounding pixel values, thereby achieving a smoothing effect. The convolution kernel of Gaussian blur can be expressed as:

[0050]

[0051] in, is the weight of the convolution kernel, Represents the standard deviation of the Gaussian function, which determines the degree of blur. After Gaussian blur processing, the preprocessed image is obtained. The preprocessed image is edge enhanced. Edge enhancement is usually achieved through gradient operators, such as Sobel operator, Laplace operator, etc. Through edge enhancement, the edge information of objects in the image is highlighted to obtain clearer object contours. The processed multi-frame image set is used as a preprocessed image sequence, and these images are used for subsequent background difference analysis. Multiple frames of images are selected from the preprocessed image sequence for pixel-level averaging to obtain a background reference image. The background reference image represents the static background part of the scene and is used for comparison with the newly input preprocessed image. Background reference image It can be calculated by the following formula:

[0052]

[0053] in, Indicates Frame image at position The pixel value of Indicates the number of frames selected. The background reference image obtained Perform a difference operation with the newly input preprocessed image to generate a difference image. The purpose of the difference operation is to compare the current image with the background and identify the moving objects in the scene. The difference image can be obtained by the following formula:

[0054]

[0055] in, is the preprocessed image of the current frame, is the background reference image,

[0056] This is the difference image, which represents the changed part of the scene. Threshold processing is performed on the difference image, and pixels greater than the preset threshold are marked as foreground to obtain a binary difference image. This processing step distinguishes the moving objects in the image from the background. Binarization is performed in the following way:

[0057]

[0058] in, is the image after binarization, is the threshold. When the pixel value in the difference image is greater than the threshold, the pixel is marked as foreground (value is 1), otherwise it is marked as background (value is 0). The binary difference image is subjected to noise removal. Through morphological operations such as opening or closing operations, isolated small noise points are removed to obtain a more coherent motion region image. The opening operation eliminates small noise points by first corroding and then dilating, while the closing operation fills small holes by first dilating and then corroding. After noise removal, an optimized motion region image is obtained. Edge detection is performed on the optimized motion region image to extract the contour of the object in the image. Common edge detection methods include the Canny edge detection algorithm. Through this algorithm, the target contour image is obtained, and the target contour line segment in the image is extracted. The target contour line segment is compared with the preset human contour template. The similarity score of the contour is calculated. The higher the similarity score, the closer the target contour is to the human contour template. The similarity can be calculated by shape matching metrics, such as Hausdorff distance. The similarity score is used to generate the final human feature description. The calculated human feature description is compared with the second threshold. Only when the similarity score exceeds the threshold, the corresponding area is identified as a target area that may contain a human figure, and finally a preliminary target mask is obtained by screening. The target mask is used to mark the area in the image that is identified as a human figure.

[0059] In one example, human shape features are extracted from the preliminary target mask to obtain a candidate human shape region and shape descriptor is calculated to obtain a pseudo label of the human shape region, including:

[0060] The preliminary target mask is bounded by a rectangle to obtain a candidate rectangular area, and the candidate rectangular area is screened according to a preset human body proportion range to obtain a preliminary human shape area;

[0061] Extract gradient features from the preliminary humanoid region to obtain a gradient direction histogram, and calculate feature similarity based on the gradient direction histogram to obtain a humanoid possibility score;

[0062] Based on the humanoid possibility score, the preliminary humanoid region is screened twice to obtain a refined humanoid region, and the overlapping regions of the refined humanoid region are merged to obtain a candidate humanoid region;

[0063] Extracting shape description data from the candidate human figure region, and performing feature compression on the shape description data to obtain a compact shape descriptor;

[0064] The compact shape descriptors are matched to obtain shape matching results, and the shape matching results are sorted by similarity to obtain the best matching shape;

[0065] Combine the best matching shape and the human shape possibility score, calculate the comprehensive confidence, obtain the human shape area confidence map, and expand the human shape area confidence map to obtain the complete human shape area;

[0066] The complete humanoid region is extracted to obtain a feature description vector, and the feature description vector is filtered to obtain a pseudo label of the humanoid region.

[0067] In this example, the preliminary target mask is rectangularly framed to obtain candidate rectangular regions. Rectangular frame can be formed by finding continuous foreground pixel regions on the preliminary target mask and enclosing them with the minimum circumscribed rectangle. Each rectangular region may contain a complete human figure or a partial human figure. According to the preset human body proportion range, the candidate rectangular regions are screened to eliminate regions that do not conform to the human body proportions. Human body proportions usually include the ratio of height to shoulder width, the ratio of height to leg length, etc. By limiting the proportions, the noise regions that do not conform to the human body structure can be effectively filtered out. The gradient features of each preliminary human-shaped region are extracted. The gradient feature reflects the direction and intensity of the change in pixel brightness in the image, and can capture the edge information in the image. The gradient direction histogram is a commonly used feature representation method. It forms a histogram by calculating the gradient direction of each pixel in the image and counting the gradient frequencies in different directions. For each preliminary human-shaped region, its gradient direction histogram is calculated. , where each entry Indicates The histogram can effectively capture the directional distribution characteristics of the edges in the region. In order to evaluate whether these regions may contain humanoid targets, the feature similarity is calculated based on the gradient direction histogram. The feature similarity can be measured by calculating the distance between the gradient direction histogram of the current region and the preset humanoid template histogram. Assume that the gradient direction histogram of the humanoid template is , then the feature similarity can be calculated by the following formula:

[0068]

[0069] in, represents the similarity score, is the number of entries in the histogram. Similarity score The value of is between 0 and 1. The closer the value is to 1, the more similar the current region is to the humanoid template. In this way, a humanoid possibility score is calculated for each preliminary humanoid region. Based on the calculated humanoid possibility score, the preliminary humanoid region is screened twice. Only those regions with higher possibility scores will be retained to form more refined humanoid regions. In order to avoid repeated identification of the same target by multiple adjacent refined humanoid regions, these regions are merged for overlapping regions. For two regions with high overlap, they are merged into a larger region to avoid redundant detection and obtain candidate humanoid regions. Shape description data is extracted from the candidate humanoid regions. Shape description data usually includes geometric features of the contour, such as perimeter, area, rectangularity, etc. These description data can effectively represent the shape features of objects in the image. In order to reduce the computational complexity, the extracted shape description data is feature compressed to generate a compact shape descriptor. Feature compression can be achieved through dimensionality reduction techniques such as principal component analysis, which compresses high-dimensional shape description data to a lower dimension while retaining the most representative feature information. The generated compact shape descriptor is matched with the preset standard shape template, and the shape matching result is obtained by calculating the similarity. The higher the similarity, the closer the shape of the candidate region is to the humanoid template. In order to determine the best matching shape, all shape matching results are sorted by similarity, and the result with the highest similarity is selected as the best matching shape. The confidence of each candidate region is calculated by combining the best matching shape and the previously calculated humanoid possibility score. Confidence is a comprehensive indicator used to evaluate the possibility of each region as a humanoid target. The confidence information is used to generate a humanoid region confidence map. Each pixel value in the confidence map represents the probability that the pixel belongs to a humanoid target. On this basis, the humanoid region confidence map is expanded to fill in the local gaps caused by noise or other factors to obtain a complete humanoid region. A feature description vector is extracted for the complete humanoid region, which contains the key feature information of the target region. By screening the feature description vector, those regions that do not meet the characteristics of the humanoid target are removed, and the humanoid region pseudo-label is screened.

[0070] In one example, an initial human recognition model is obtained based on human region pseudo-label training and multi-scale feature extraction is performed to obtain a local spatial depth feature map and a target feature vector, including:

[0071] Performing data enhancement processing on the pseudo labels of the human-shaped area to obtain an expanded training set, and performing standardization processing on the expanded training set to obtain standardized training data;

[0072] Construct a lightweight convolutional neural network backbone consisting of 5 convolutional blocks. Each convolutional block contains a convolutional layer, a batch normalization layer, and a ReLU activation function layer. Perform feature extraction on the input preprocessed image through the convolutional neural network backbone to obtain a preliminary feature map.

[0073] A discrete wavelet transform layer is inserted between the third and fourth convolution blocks of the convolutional neural network backbone, and a discrete wavelet transform is performed on the feature map passing through the third convolution block to obtain low-frequency approximation coefficients and high-frequency detail coefficients;

[0074] Adaptively weighted fusion is performed on low-frequency approximation coefficients and high-frequency detail coefficients to obtain a multi-scale feature map, which is then input into the fourth convolution block to obtain an enhanced feature representation.

[0075] Add a spatial pyramid pooling module after the last convolutional block of the convolutional neural network backbone; perform multi-scale pooling on the output feature map of the last convolutional block to obtain feature vectors of different scales;

[0076] The feature vectors of different scales are concatenated to obtain a multi-scale feature representation, and the multi-scale feature representation is reduced in dimension through 1x1 convolution to obtain a compressed feature vector;

[0077] Construct a fully connected layer network, including two fully connected layers and one Dropout layer, input the compressed feature vector into the fully connected layer network, and output the prediction result;

[0078] The error between the predicted result and the true label is calculated using the cross entropy loss function to obtain the loss value; the network parameters are optimized through the back propagation algorithm to obtain the updated network parameters until the loss value converges or reaches the preset number of iterations to obtain the initial human recognition model;

[0079] The new preprocessed image is input into the initial human figure recognition model, and after being processed by all convolution blocks, discrete wavelet transform layers and spatial pyramid pooling modules, a local spatial depth feature map is obtained, and a global average pooling operation is performed on the local spatial depth feature map to obtain the target feature vector.

[0080] In this example, data augmentation is performed on the pseudo labels of the humanoid region. Data augmentation is a method to increase the diversity of training data by rotating, scaling, translating, flipping, adding noise, and other operations on existing samples to generate new training samples. Suppose a humanoid image in the initial dataset is , multiple transformed images can be generated through data enhancement, such as . The enhanced images are used to expand the training set to obtain more diverse training data. The expanded training set is standardized to scale the pixel values ​​of the image data to a uniform range, usually by subtracting the mean of the data set from each pixel value and dividing it by the standard deviation, thereby accelerating the convergence process of the model and obtaining standardized training data. A lightweight convolutional neural network backbone consisting of 5 convolutional blocks is constructed. Each convolutional block consists of a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The convolutional layer is used to extract local features of the image, the batch normalization layer is used to accelerate network training and improve the stability of the model, and the ReLU activation function introduces nonlinearity so that the model can better fit complex data. The convolutional neural network backbone is designed to be lightweight to adapt to the computing power of low-power monitoring devices, while being able to effectively extract feature information from the input image. After the input preprocessed image is processed by the convolutional neural network backbone, a preliminary feature map is generated. A discrete wavelet transform layer is inserted between the third and fourth convolutional blocks of the convolutional neural network backbone. By decomposing the image signal, the discrete wavelet transform can extract the low-frequency approximate coefficients and high-frequency detail coefficients of the image respectively. The low-frequency approximate coefficients mainly represent the overall structural information of the image, while the high-frequency detail coefficients capture the edge and texture details of the image. For example, assuming that the output feature map of the third convolution block is , through discrete wavelet transform, we get the low-frequency approximate coefficients and high frequency detail factor These two coefficients retain different scale information of the image. Adaptive weighted fusion is performed on the low-frequency approximation coefficient and the high-frequency detail coefficient. Combining the feature information of different frequencies, a richer multi-scale feature map is generated. The fused multi-scale feature map It is expressed by the formula:

[0081]

[0082] in, and For adaptive weight parameters, the values ​​of these two parameters are dynamically adjusted according to specific task requirements to balance the contribution of low-frequency and high-frequency information. The fused multi-scale feature map is input into the fourth convolution block for feature extraction to obtain an enhanced feature representation. A spatial pyramid pooling module is added after the last convolution block of the convolutional neural network backbone. The spatial pyramid pooling module generates multiple feature vectors of different scales by pooling the feature maps at different scales. These feature vectors retain information of different spatial scales and can adapt to input images of different sizes. Feature vectors of all scales are concatenated to form a multi-scale feature representation. In order to reduce the computational complexity of the model, the concatenated multi-scale feature representation is reduced in dimension through 1x1 convolution to generate a compressed feature vector. The role of 1x1 convolution is to map high-dimensional features to low-dimensional space through linear combination while retaining the most useful information. Assume that the multi-scale feature representation is , the compressed feature vector is , then the 1x1 convolution can be expressed as:

[0083]

[0084] in, is the weight matrix, is the bias term, is the low-dimensional feature representation generated in the end. A fully connected layer network is constructed. The network contains two fully connected layers and one Dropout layer, which are used to process the feature vector and generate the final prediction result. The fully connected layer maps the feature vector to the target output space through linear transformation and nonlinear activation, while the Dropout layer is used to prevent overfitting and enhance the generalization ability of the model by randomly discarding some neurons. During the training process, the cross entropy loss function is used to calculate the error between the prediction result and the true label. Assume that the prediction result is , the true label is , the cross entropy loss function can be expressed as:

[0085]

[0086] in, is the number of categories, and Respectively The true label and predicted probability of the class. The model parameters are optimized by minimizing the cross entropy loss. The back propagation algorithm is used to adjust the weights and biases in the network to gradually reduce the loss value. The training process continues until the loss value converges or the preset number of iterations is reached to obtain the initial human recognition model. After the model training is completed, the new preprocessed image is input into the initial human recognition model, and the local spatial depth feature map is generated after being processed by all convolution blocks, discrete wavelet transform layers, and spatial pyramid pooling modules. In order to summarize the feature information, the local spatial depth feature map is subjected to global average pooling to generate the target feature vector. Global average pooling generates a fixed-length vector representing the features of the entire image by taking the average of each channel in the feature map.

[0087] In one example, the target feature vectors are classified and weighted voted to obtain a preliminary human recognition result, and the recognition results of consecutive frames are optimized to obtain a human target tracking sequence, including:

[0088] Normalizing the target feature vector to obtain a standardized feature vector, and inputting the standardized feature vector into a plurality of pre-configured kernelized extreme learning machines to obtain a plurality of groups of classification results;

[0089] Perform weighted averaging on multiple groups of classification results to obtain a comprehensive classification score, and perform binarization on the comprehensive classification score according to a preset classification threshold to obtain a preliminary human recognition result;

[0090] Perform time series analysis on the preliminary human recognition results of multiple consecutive frames to obtain the time consistency score of the human target, and correct the single-frame recognition results based on the time consistency score to obtain the recognition results after time smoothing;

[0091] Perform spatial connectivity analysis on the recognition results after time smoothing to obtain a spatial consistency score, and combine the spatial consistency score with the recognition results after time smoothing to obtain a humanoid region that is consistent in time and space.

[0092] Extract the bounding box of the humanoid area that is consistent in time and space to obtain the position and size information of the humanoid target, and construct the target state vector based on the position and size information to obtain the initial tracking state;

[0093] The Kalman filter is used to predict the initial tracking state to obtain the predicted position of the next frame, and the predicted position is associated with the recognition result of the next frame to obtain the updated tracking state;

[0094] The updated tracking state is subjected to motion consistency constraints, and abnormal tracking results that do not conform to the motion pattern are eliminated to obtain an optimized tracking sequence, which is then smoothed to obtain a humanoid target tracking sequence.

[0095] In this example, the target feature vector is normalized to adjust the value range of the feature vector to a uniform range, eliminate the scale difference between the features, and obtain a standardized feature vector. The standardized feature vector is input into multiple pre-configured kernelized extreme learning machines to obtain multiple groups of classification results. The kernelized extreme learning machine is a fast learning algorithm based on kernel functions. It processes data through randomly generated hidden layers and kernel function mappings, and classifies by minimizing the weight of the output layer. The multiple groups of classification results are weighted averaged. The output results of different classifiers are combined, and different weights are assigned to different classifiers to generate a more accurate comprehensive classification score. According to the preset classification threshold, the comprehensive classification score is binarized to generate a preliminary human recognition result. If the comprehensive classification score exceeds the threshold, the area is marked as a human target, otherwise it is marked as a non-human. The preliminary human recognition results of multiple consecutive frames are analyzed in time series. By analyzing the consistency of the target in multiple consecutive frames, the temporal consistency score of the human target is calculated. The temporal consistency score reflects the stability of the target in time. The higher the score, the greater the possibility that the target is continuously recognized as a human in multiple consecutive frames. Based on the temporal consistency score, the recognition result of a single frame is corrected to obtain the recognition result after temporal smoothing. The spatial connectivity analysis is performed on the recognition result after temporal smoothing to check the consistency of the recognition result in space. The spatial consistency score is calculated by evaluating the coherence of the recognition area. A high spatial consistency score means that the recognition result has high connectivity in space and low noise interference. Combining the temporal consistency score and the spatial consistency score, a temporally and spatially consistent humanoid region is obtained, which has high credibility in both time and space. For the recognized humanoid region, bounding box extraction is performed to obtain the position and size information of each humanoid target. The bounding box is used to accurately locate the range of the humanoid target, and the position and size information is used to construct the target state vector. The target state vector usually contains dynamic information such as the position and speed of the target, and describes the state of the target in the current frame. In order to predict the position of the humanoid target in the next frame, the Kalman filter is used to predict the initial tracking state. The Kalman filter is a classic recursive algorithm used to estimate the future state through the current state. Assume that the initial state vector of the target is , then the predicted position of the next frame can be calculated by the following formula:

[0096]

[0097] in, is the state transfer matrix, which describes the state change of the target from the current frame to the next frame. is the control input, is the process noise. Through the prediction of the Kalman filter, the predicted position of the target in the next frame is obtained, and the predicted position is associated with the actual recognition result of the next frame to obtain the updated tracking state. In the updated tracking state, there may be erroneous results caused by noise or abnormal motion. In order to improve the accuracy of tracking, motion consistency constraints are applied to the updated tracking state to eliminate abnormal tracking results that do not conform to the normal motion pattern. For example, if the speed or direction of the target suddenly changes abnormally, the result may be wrong and needs to be eliminated through motion consistency constraints to obtain an optimized tracking sequence. The optimized tracking sequence is smoothed to generate a stable humanoid target tracking sequence. Smoothing removes short-term fluctuations to generate a smoother and continuous tracking path to obtain a humanoid target tracking sequence.

[0098] In one example, a scene activity index is calculated according to a humanoid target tracking sequence, a multi-mode adaptive monitoring strategy is generated, and a low-power working scheme of a low-power monitoring device is dynamically adjusted based on the multi-mode adaptive monitoring strategy, including:

[0099] The number of targets in the humanoid target tracking sequence is counted to obtain the target appearance frequency per unit time, and the crowd density index of the scene is calculated based on the target appearance frequency to obtain the crowd density score;

[0100] The target movement speed in the humanoid target tracking sequence is analyzed to obtain the average movement speed and speed change rate, and the scene dynamic index is calculated based on the average movement speed and speed change rate to obtain the scene dynamic score;

[0101] Combine the crowd density score and scene dynamic score to calculate the comprehensive scene activity index through weighted summation;

[0102] Collect the current ambient light intensity data to obtain the light intensity value, and select appropriate image enhancement parameters according to the light intensity value to obtain the light adaptive processing strategy;

[0103] Read the current power information of the device, get the power percentage, and set the power consumption control level according to the power percentage to obtain the power adaptive control strategy;

[0104] Generate a multi-mode adaptive monitoring strategy based on scene activity indicators, adaptive light processing strategy, and adaptive power control strategy;

[0105] The image acquisition frequency is set according to the multi-mode adaptive monitoring strategy to obtain a dynamic sampling scheme, and the working parameters of the image sensor are adjusted based on the dynamic sampling scheme to obtain an optimized image acquisition strategy;

[0106] The image processing complexity is selected according to the multi-mode adaptive monitoring strategy to obtain the algorithm complexity control scheme. Based on the algorithm complexity control scheme and the optimized image acquisition strategy, a low-power working plan for low-power monitoring equipment is formulated.

[0107] In this example, the number of targets in the humanoid target tracking sequence is analyzed. By counting the frequency of target appearance per unit time, the density of human traffic in the scene is calculated. The number of targets detected within is , then the target frequency It can be expressed as:

[0108]

[0109] in, Indicates the number of targets that appear in a unit of time. The frequency of target appearance directly reflects the density of the crowd in the scene. According to this frequency, the crowd density index of the scene is calculated. The crowd density index is a key parameter for measuring crowd activities in a scene. It is usually combined with the scene area to obtain a more accurate assessment. Assume that the scene area is , then the crowd density index It can be expressed as:

[0110]

[0111] Based on the calculated crowd density index, a crowd density score is generated to describe the crowd density in the scene. The higher the score, the more frequent the crowd activities in the scene and the greater the density. Analyze the target movement speed in the humanoid target tracking sequence. Calculate the average movement speed of each target. Assume that a target is in the time period Distance moved within , then the average speed It can be expressed as:

[0112]

[0113] in, It represents the average moving distance of the target in a unit of time. After calculating the average speed, analyze the target's speed change rate, that is, the change of the target's speed in different time periods. The speed change rate can be measured by calculating the standard deviation of the speed. Assuming that the speed value measured in the time period is Then the rate of change of speed It can be expressed as:

[0114]

[0115] in, Indicates the degree of speed fluctuation. Based on the average movement speed and speed change rate, the dynamic index of the scene is calculated. The scene dynamic index is used to describe the intensity of movement activity in the scene and reflects the movement law of the target. Based on this index, a scene dynamic score is generated to measure the activity level of movement in the scene. The crowd density score is combined with the scene dynamic score, and a comprehensive scene activity index is calculated through weighted summation. Assume that the crowd density score is , the scene dynamic score is , then the comprehensive scene activity index It can be expressed as:

[0116]

[0117] in, and is a weight parameter used to adjust the contribution ratio of the two scores to the scene activity. The comprehensive scene activity index is used to reflect the overall activity level in the scene and is an important basis for subsequent monitoring strategy adjustments. Consider the impact of ambient light intensity on monitoring effects. The current ambient light intensity data is collected through sensors to obtain the light intensity value. According to the light intensity value, appropriate image enhancement parameters are selected to ensure the stability of image quality under different lighting conditions. For example, in dim light conditions, it may be necessary to increase the exposure time or increase the gain to enhance the image brightness. Based on these adjustments, a light adaptive processing strategy is generated to adapt to different lighting conditions. At the same time, the current power information of the device is read to obtain the power percentage. The power percentage is a key indicator for measuring the remaining power of the device. Different power consumption control levels are set according to the power percentage. The power consumption control level determines the operating mode of the device. For example, when the power is low, it enters the energy-saving mode, reduces the image acquisition frequency and processing complexity, and prolongs the working time of the device. Generate a power adaptive control strategy based on the power information to ensure that the device can operate efficiently under different power states. A multi-mode adaptive monitoring strategy is generated based on the scene activity index, the light adaptive processing strategy, and the power adaptive control strategy. The multi-mode adaptive monitoring strategy can dynamically adjust the operating mode of the monitoring device according to the real-time scene and device status. For example, in high scene activity, the system may increase the image acquisition frequency and processing complexity to ensure accurate recognition and tracking of all targets in the scene; while in low power conditions, the system may reduce these parameters to reduce energy consumption. According to the multi-mode adaptive monitoring strategy, the image acquisition frequency is set to obtain a dynamic sampling scheme. The dynamic sampling scheme optimizes the image acquisition process by adjusting the working parameters of the image sensor, such as frame rate, resolution, etc., to ensure that important information can be captured efficiently in different scenarios. For example, in high-density and high-dynamic scenes, the system may increase the frame rate to capture fast-moving targets; while in low-activity scenes, the frame rate is reduced to save energy. According to the multi-mode adaptive monitoring strategy, the complexity of image processing is selected to obtain an algorithm complexity control scheme. The algorithm complexity control scheme involves selecting appropriate image processing algorithms and models. For example, in high-activity scenes, the system may use more complex deep learning models to improve recognition accuracy; while in low-power modes, lightweight models are used to reduce computational burden. Combining the algorithm complexity control scheme and the optimized image acquisition strategy, a low-power working plan for low-power monitoring equipment is developed.

[0118] In one example, incremental learning and model updating are performed according to a humanoid target tracking sequence to obtain a target humanoid recognition model, and a multi-level wake-up control strategy is generated based on the target humanoid recognition model and a low-power working scheme, including:

[0119] The target images in the humanoid target tracking sequence are cropped and aligned to obtain standardized target samples, and features are extracted from the standardized target samples to obtain a new sample feature set;

[0120] Compare the newly added sample feature set with the historical sample feature library to obtain the sample novelty evaluation result, and select the incremental learning sample set based on the sample novelty evaluation result;

[0121] Based on the incremental learning sample set, the last layer of the initial human figure recognition model is fine-tuned to obtain updated model parameters, and the updated model parameters are weightedly fused with the original model parameters to obtain a temporary updated model;

[0122] Evaluate the performance of the temporary updated model, calculate the recognition accuracy and computational complexity, obtain the model update evaluation index, and decide whether to accept the update based on the model update evaluation index to obtain the target human recognition model;

[0123] Based on the target human figure recognition model, multiple confidence thresholds are set to obtain a multi-level human figure recognition judgment standard, and the real-time image is quickly screened according to the multi-level human figure recognition judgment standard to obtain a preliminary wake-up trigger signal;

[0124] Combined with the scene activity index in the low-power working scheme, the initial wake-up trigger signal is weighted to obtain a scene-adaptive wake-up score, and a multi-level wake-up threshold is set based on the scene-adaptive wake-up score;

[0125] According to the multi-level wake-up threshold, the device wake-up process is divided into multiple energy consumption levels to obtain a step-by-step wake-up strategy, and the energy consumption of the step-by-step wake-up strategy is optimized to obtain a multi-level wake-up solution with balanced energy consumption;

[0126] The energy-balanced multi-level wake-up scheme is combined with the low-power working scheme to generate a multi-level wake-up control strategy.

[0127] In this example, the target image in the tracking sequence is cropped and aligned, and the area containing the human target is extracted from the video frame. These areas are standardized to ensure the stability and consistency of the subsequent feature extraction process. The cropping operation cuts out the human target part in the image by analyzing the bounding box of the target; the alignment operation rescales and rotates the cropped image according to the position and posture of the target, so that all target samples are spatially consistent. Through these processes, standardized target samples are obtained. Feature extraction is performed on the standardized target samples to generate a new sample feature set. The image data is converted into high-dimensional feature vectors, which can effectively represent important information such as the shape and texture of the target. Common feature extraction methods include using convolutional neural networks to extract deep features, or using gradient directional histograms to extract edge features. Assume that a standardized target sample is , through the feature extraction process, generate feature vector Each of these Represents a feature of the image. These feature vectors constitute a new sample feature set, which represents the newly collected target information. The new sample feature set is compared with the historical sample feature library to evaluate the similarity and novelty of these new samples. The historical sample feature library stores the target features that have been learned in the past. By calculating the similarity between the new samples and these historical features, it is determined whether the new samples contain new target types or new postures. Novelty evaluation is usually achieved by calculating the Euclidean distance or cosine similarity between feature vectors. Suppose a sample feature in the historical sample feature library is , the Euclidean distance between the newly added sample and It can be expressed as:

[0128]

[0129] in, Represents the distance between the two. The larger the distance, the greater the difference between the new samples and the historical samples, and the stronger the novelty. According to the novelty evaluation results, a part of samples with higher novelty are screened out to form an incremental learning sample set. Based on the screened incremental learning sample set, the last layer of the initial human recognition model is fine-tuned. On the basis of retaining the original model features, adapt to the new information brought by the new samples and improve the generalization ability of the model. Freeze the first few layers of the model and only update the parameters of the last layer. Adjust the weights of the last layer by the gradient descent method to generate updated model parameters. In order to balance the influence of new and old features, the updated model parameters are weightedly fused with the original model parameters to obtain a temporary updated model. Assume that the original model parameters are , the updated parameters are , then the temporary model parameters after fusion are It can be expressed as:

[0130]

[0131] in, is the fusion coefficient, which determines the contribution ratio of the new and old parameters. The performance of the temporary update model is evaluated, and the recognition accuracy and computational complexity are calculated to obtain the model update evaluation index. The recognition accuracy measures the performance of the model on new data, while the computational complexity reflects the resources required for the model to run. Based on these evaluation indicators, it is determined whether to accept the update. If the performance of the temporary update model is better than the original model, the update is accepted to generate the final target human recognition model. Based on the target human recognition model, multiple confidence thresholds are set to generate multi-level human recognition judgment criteria. The judgment criteria are used to quickly screen real-time images to determine whether to trigger device wake-up. When the recognition confidence exceeds a certain threshold, a preliminary wake-up trigger signal is generated to start a higher-precision detection module. The preliminary wake-up trigger signal is combined with the scene activity index in the low-power working scheme to adjust its weight. The scene activity index reflects the dynamics of the current monitoring scene, such as crowd density and movement speed. By adjusting the weight of the preliminary wake-up trigger signal, a scene-adaptive wake-up score is generated. The higher the scene-adaptive wake-up score, the more it indicates that an important event has occurred in the current scene, and the stronger the necessity of waking up the device. Based on the scenario-adaptive wake-up score, multiple wake-up thresholds are set to divide the device wake-up process into multiple energy consumption levels to form a stepped wake-up strategy. For example, when the wake-up score is low, the device may only wake up the low-power mode for basic detection; when the wake-up score is high, the device may start the high-precision mode for comprehensive monitoring. The energy consumption of the stepped wake-up strategy is optimized to obtain a multi-level wake-up solution with balanced energy consumption. The goal of energy consumption optimization is to reasonably allocate energy consumption resources according to the power status and task requirements of the device and extend the working time of the device. The energy-balanced multi-level wake-up solution is combined with the low-power working solution to generate a comprehensive multi-level wake-up control strategy.

[0132] Reference Figure 2 This embodiment provides a low-power monitoring and awakening device based on human figure recognition, including:

[0133] The preprocessing module 1 is used to downsample and grayscale the original image sequence collected by the low-power monitoring device to obtain a preprocessed image and perform background difference to obtain a preliminary target mask;

[0134] Calculation module 2, used to extract human features from the preliminary target mask, obtain candidate human regions and calculate shape descriptors to obtain pseudo labels for human regions;

[0135] Training module 3, used to obtain an initial human recognition model based on human region pseudo-label training and perform multi-scale feature extraction to obtain a local spatial depth feature map and a target feature vector;

[0136] Classification module 4 is used to classify and weighted vote the target feature vectors to obtain preliminary human recognition results, and optimize the recognition results of consecutive frames to obtain a human target tracking sequence;

[0137] An adjustment module 5 is used to calculate the scene activity index according to the humanoid target tracking sequence, generate a multi-mode adaptive monitoring strategy, and dynamically adjust the low-power working scheme of the low-power monitoring device based on the multi-mode adaptive monitoring strategy;

[0138] The generation module 6 is used to perform incremental learning and model update according to the human target tracking sequence to obtain a target human recognition model, and generate a multi-level wake-up control strategy based on the target human recognition model and the low-power working scheme.

[0139] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0140] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0141] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0142] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0144] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0145] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A low-power monitoring wake-up method based on human figure recognition, characterized in that: The following steps are involved: The original image sequence collected by the low-power monitoring device is downsampled and grayed to obtain a preprocessed image and perform background subtraction to obtain a preliminary target mask; Extracting human features from the preliminary target mask to obtain a candidate human figure region and calculating a shape descriptor to obtain a pseudo label of the human figure region; Based on the pseudo-label training of the human shape region, an initial human shape recognition model is obtained and multi-scale feature extraction is performed to obtain a local spatial depth feature map and a target feature vector; Classifying and weighted voting the target feature vectors to obtain preliminary human recognition results, and optimizing the recognition results of consecutive frames to obtain a human target tracking sequence; Calculating a scene activity index according to the humanoid target tracking sequence, generating a multi-mode adaptive monitoring strategy, and dynamically adjusting a low-power working scheme of the low-power monitoring device based on the multi-mode adaptive monitoring strategy; Incremental learning and model updating are performed according to the humanoid target tracking sequence to obtain a target humanoid recognition model, and a multi-level wake-up control strategy is generated based on the target humanoid recognition model and the low-power working scheme.

2. The low-power monitoring wake-up method based on human figure recognition according to claim 1 is characterized in that: The method of downsampling and graying the original image sequence collected by the low-power monitoring device to obtain a pre-processed image and performing background difference to obtain a preliminary target mask includes: Reducing the resolution of an original image sequence collected by a low-power monitoring device to obtain a downsampled image, and performing grayscale conversion on the downsampled image to obtain a grayscale image; Performing image smoothing on the grayscale image to obtain a preprocessed image, and performing edge enhancement processing on the preprocessed image to obtain a preprocessed image sequence; Selecting multiple frames of images from the pre-processed image sequence and performing pixel-level averaging to obtain a background reference image, and performing a difference operation between the newly input pre-processed image and the background reference image to obtain a difference image; Performing threshold processing on the difference image, marking pixels greater than a first threshold as foregrounds to obtain a binary difference image, and performing noise removal processing on the binary difference image to obtain an optimized motion region image; Performing edge detection on the optimized motion region image to obtain a target contour image, and extracting target contour line segments of the target contour image; The target contour line segment is compared with a preset human contour template, a similarity score is calculated, a human feature description is obtained, and the human feature description is compared with a second threshold value to screen and obtain a preliminary target mask.

3. The low-power monitoring wake-up method based on human figure recognition according to claim 1 is characterized in that: The step of extracting human features from the preliminary target mask, obtaining a candidate human figure region, and calculating a shape descriptor to obtain a pseudo label of the human figure region includes: The preliminary target mask is bounded by a rectangle to obtain a candidate rectangular area, and the candidate rectangular area is screened according to a preset human body proportion range to obtain a preliminary human shape area; Extracting gradient features from the preliminary human shape region to obtain a gradient direction histogram, and calculating feature similarity based on the gradient direction histogram to obtain a human shape possibility score; Based on the human shape possibility score, the preliminary human shape region is screened for a second time to obtain a refined human shape region, and the overlapping regions of the refined human shape region are merged to obtain a candidate human shape region; Extracting shape description data from the candidate human figure region, and performing feature compression on the shape description data to obtain a compact shape descriptor; Matching the compact shape descriptors to obtain shape matching results, and sorting the shape matching results by similarity to obtain the best matching shape; Combining the best matching shape and the human shape possibility score, calculating a comprehensive confidence to obtain a human shape region confidence map, and performing region expansion on the human shape region confidence map to obtain a complete human shape region; A feature description vector is obtained by extracting the complete human shape region, and the feature description vector is screened to obtain a pseudo label of the human shape region.

4. The low-power monitoring wake-up method based on human figure recognition according to claim 1 is characterized in that: The initial human figure recognition model is obtained based on the human figure region pseudo-label training and multi-scale feature extraction is performed to obtain a local spatial depth feature map and a target feature vector, including: Performing data enhancement processing on the pseudo-labels of the human-shaped region to obtain an expanded training set, and performing standardization processing on the expanded training set to obtain standardized training data; Construct a lightweight convolutional neural network backbone containing 5 convolutional blocks, each of which contains a convolutional layer, a batch normalization layer, and a ReLU activation function layer, and perform feature extraction on the input preprocessed image through the convolutional neural network backbone to obtain a preliminary feature map; Inserting a discrete wavelet transform layer between the third convolution block and the fourth convolution block of the convolutional neural network backbone, and performing discrete wavelet transform on the feature map passing through the third convolution block to obtain low-frequency approximation coefficients and high-frequency detail coefficients; Adaptively weighting and fusing the low-frequency approximation coefficients and the high-frequency detail coefficients to obtain a multi-scale feature map, and inputting the multi-scale feature map into a fourth convolution block to obtain an enhanced feature representation; Adding a spatial pyramid pooling module after the last convolution block of the convolutional neural network backbone; performing multi-scale pooling on the output feature map of the last convolution block to obtain feature vectors of different scales; The feature vectors of different scales are concatenated to obtain a multi-scale feature representation, and the multi-scale feature representation is reduced in dimension by 1x1 convolution to obtain a compressed feature vector; Constructing a fully connected layer network, including two fully connected layers and one Dropout layer, and inputting the compressed feature vector into the fully connected layer network, and outputting a prediction result; The error between the prediction result and the true label is calculated using a cross entropy loss function to obtain a loss value; the network parameters are optimized using a back propagation algorithm to obtain updated network parameters until the loss value converges or reaches a preset number of iterations, thereby obtaining an initial human recognition model; The new preprocessed image is input into the initial human figure recognition model, and is processed by all convolution blocks, discrete wavelet transform layers and spatial pyramid pooling modules to obtain a local spatial depth feature map, and a global average pooling operation is performed on the local spatial depth feature map to obtain a target feature vector.

5. The low-power monitoring wake-up method based on human figure recognition according to claim 1 is characterized in that: The target feature vectors are classified and weighted voted to obtain preliminary human recognition results, and the recognition results of consecutive frames are optimized to obtain a human target tracking sequence, including: Normalizing the target feature vector to obtain a standardized feature vector, and inputting the standardized feature vector into a plurality of pre-configured kernelized extreme learning machines to obtain a plurality of groups of classification results; Performing weighted averaging on the multiple groups of classification results to obtain a comprehensive classification score, and binarizing the comprehensive classification score according to a preset classification threshold to obtain a preliminary human recognition result; Performing time series analysis on preliminary human recognition results of multiple consecutive frames to obtain a time consistency score of the human target, and correcting a single-frame recognition result based on the time consistency score to obtain a recognition result after time smoothing; Performing spatial connectivity analysis on the time-smoothed recognition result to obtain a spatial consistency score, and combining the spatial consistency score with the time-smoothed recognition result to obtain a temporally and spatially consistent human-shaped region; Extracting a bounding box of the temporally and spatially consistent humanoid region to obtain position and size information of the humanoid target, and constructing a target state vector based on the position and size information to obtain an initial tracking state; Using a Kalman filter to predict the initial tracking state, obtain a predicted position of the next frame, and associate the predicted position with the recognition result of the next frame to obtain an updated tracking state; The updated tracking state is subjected to motion consistency constraints, abnormal tracking results that do not conform to the motion pattern are eliminated to obtain an optimized tracking sequence, and the optimized tracking sequence is smoothed to obtain a humanoid target tracking sequence.

6. The low-power monitoring wake-up method based on human figure recognition according to claim 1 is characterized in that: The calculating of the scene activity index according to the humanoid target tracking sequence, generating a multi-mode adaptive monitoring strategy, and dynamically adjusting the low-power working scheme of the low-power monitoring device based on the multi-mode adaptive monitoring strategy, includes: Counting the number of targets in the humanoid target tracking sequence to obtain the target appearance frequency per unit time, and calculating the crowd density index of the scene based on the target appearance frequency to obtain a crowd density score; Analyzing the target movement speed in the humanoid target tracking sequence to obtain an average movement speed and a speed change rate, and calculating a scene dynamic index according to the average movement speed and the speed change rate to obtain a scene dynamic score; Combining the crowd density score and the scene dynamic score, a comprehensive scene activity index is calculated by weighted summation; Collecting current ambient light intensity data to obtain a light intensity value, and selecting appropriate image enhancement parameters according to the light intensity value to obtain a light adaptive processing strategy; Read the current power information of the device to obtain the power percentage, and set the power consumption control level according to the power percentage to obtain a power adaptive control strategy; Generate a multi-mode adaptive monitoring strategy based on the scene activity index, the adaptive illumination processing strategy and the adaptive power control strategy; Setting the image acquisition frequency according to the multi-mode adaptive monitoring strategy to obtain a dynamic sampling scheme, and adjusting the working parameters of the image sensor based on the dynamic sampling scheme to obtain an optimized image acquisition strategy; The image processing complexity is selected according to the multi-mode adaptive monitoring strategy to obtain an algorithm complexity control scheme, and based on the algorithm complexity control scheme and the optimized image acquisition strategy, a low-power working plan for the low-power monitoring device is formulated.

7. The low-power monitoring wake-up method based on human figure recognition according to claim 1 is characterized in that: The method of performing incremental learning and model updating according to the humanoid target tracking sequence to obtain a target humanoid recognition model, and generating a multi-level wake-up control strategy based on the target humanoid recognition model and the low-power working scheme includes: Cropping and aligning the target images in the humanoid target tracking sequence to obtain standardized target samples, and extracting features from the standardized target samples to obtain a newly added sample feature set; Comparing the newly added sample feature set with the historical sample feature library to obtain a sample novelty evaluation result, and screening out an incremental learning sample set according to the sample novelty evaluation result; Fine-tune the last layer of the initial human figure recognition model based on the incremental learning sample set to obtain updated model parameters, and weightedly fuse the updated model parameters with the original model parameters to obtain a temporary updated model; Performing performance evaluation on the temporary updated model, calculating recognition accuracy and computational complexity, obtaining a model update evaluation index, and deciding whether to accept the update based on the model update evaluation index to obtain a target human recognition model; A plurality of confidence thresholds are set based on the target human figure recognition model to obtain a multi-level human figure recognition judgment standard, and the real-time image is quickly screened according to the multi-level human figure recognition judgment standard to obtain a preliminary wake-up trigger signal; In combination with the scene activity index in the low-power working scheme, the preliminary wake-up trigger signal is weighted to obtain a scene-adaptive wake-up score, and a multi-level wake-up threshold is set based on the scene-adaptive wake-up score; According to the multi-level wake-up threshold, the device wake-up process is divided into multiple energy consumption levels to obtain a step-by-step wake-up strategy, and the step-by-step wake-up strategy is energy-optimized to obtain a multi-level wake-up solution with balanced energy consumption; The energy-balanced multi-level wake-up scheme is combined with the low-power operation scheme to generate a multi-level wake-up control strategy.

8. A low-power monitoring and awakening device based on human figure recognition, characterized in that: For performing the steps of the method according to any one of claims 1 to 7, the device comprises: The preprocessing module is used to downsample and grayscale the original image sequence collected by the low-power monitoring device to obtain a preprocessed image and perform background difference to obtain a preliminary target mask; A calculation module is used to extract human features from the preliminary target mask, obtain a candidate human figure region, and calculate a shape descriptor to obtain a pseudo label of the human figure region; A training module, used to obtain an initial human figure recognition model based on the human figure region pseudo-label training and perform multi-scale feature extraction to obtain a local spatial depth feature map and a target feature vector; A classification module is used to classify and weighted vote the target feature vectors to obtain preliminary human recognition results, and optimize the recognition results of consecutive frames to obtain a human target tracking sequence; An adjustment module, configured to calculate a scene activity index according to the humanoid target tracking sequence, generate a multi-mode adaptive monitoring strategy, and dynamically adjust a low-power working scheme of the low-power monitoring device based on the multi-mode adaptive monitoring strategy; A generation module is used to perform incremental learning and model update according to the humanoid target tracking sequence to obtain a target humanoid recognition model, and generate a multi-level wake-up control strategy based on the target humanoid recognition model and the low-power working scheme.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Ultra-low power consumption monitoring device and method

    CN115604434A

  • Low-power-consumption monitoring system based on scene self-adaptive dynamic time-sharing strategy

    CN117692767A

  • Video recording method and equipment of low-power-consumption video recording system

    CN118509702A

  • System of measuring contact wires and elements using data partitioning and method of implementing thereof

    KR1020250061383A

  • Crowd and traffic monitoring apparatus and method

    US20170017846A1

Cited By

  • Deep sleep wake-up control method and system for multimode SoC main control chip

    CN119415165A

  • High-speed traffic flow detection method and system based on deep learning

    CN119672964A

  • Battery replacement cabinet battery anti-theft system based on artificial intelligence

    CN119854343A

  • An AI-based battery anti-theft system for battery swapping cabinets

    CN119854343B

  • Dairy cow feeding area feed intake estimation system based on computer vision

    CN119942458A