A method, device and medium for recognizing human behavior in a low-illumination scene

Abstract

This invention discloses a method, device, and medium for human behavior recognition in low-light scenarios, relating to the field of image processing technology. It integrates image enhancement and pose estimation into a behavior recognition model to address the difficulties in recognition under low-light conditions. By proposing a multi-task joint neural network model, it integrates image enhancement, human pose estimation, and human behavior recognition into one system. The multi-task neural network model is trained based on pose-weighted global pooling consistency and pose-driven background random suppression. Through feature sharing and task collaborative training, it can solve the problem of low accuracy in human behavior recognition in low-light monitoring environments such as oil fields. It is particularly effective in situations where images are dim and blurry, and human poses are variable, improving the utilization of pose information and enhancing the robustness and accuracy of human behavior recognition in complex environments, achieving end-to-end optimization.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method, device, and medium for recognizing human behavior in low-light scenes. Background Technology

[0002] In oilfield production operations and similar industrial monitoring scenarios, problems such as poor lighting conditions and dim, blurry images are often encountered, posing a significant challenge to video-based human behavior recognition. Existing video surveillance behavior analysis technologies often have insufficient recognition accuracy under low lighting conditions, mainly due to the following reasons: (1) Illumination and imaging quality issues: Oilfield sites are usually monitored continuously day and night. At night or underground, the ambient light is insufficient, resulting in low brightness and high noise in the images captured by the cameras, making it difficult to distinguish details and actions of people. Traditional behavior recognition algorithms are prone to false detections or missed detections on such low-quality images. (2) Insufficient utilization of posture information: The specific actions of personnel are often closely related to their body posture. For example, bending over, reaching out, raising, etc., all correspond to specific joint postures. However, existing methods usually process posture estimation and behavior recognition separately, or do not utilize posture information at all. This fragmented processing method cannot make full use of posture priors to improve recognition accuracy and performs poorly in on-site environments with diverse posture changes. (3) Fragmented algorithm modules: Conventional solutions may use image enhancement (such as histogram equalization, brightness enhancement) and action recognition in sequence, or recognize the pose first and then classify the action. This approach of separating the functions has the problems of information loss and error accumulation: image enhancement may introduce artifacts to interfere with subsequent recognition, and the error of individual pose estimation will also be passed to the action recognition module, making overall optimization difficult. (4) Lack of end-to-end optimization in existing technologies: Although deep learning has made progress in the fields of image enhancement, action recognition and pose estimation, most of them use the method of training models independently, failing to jointly optimize related tasks and failing to give full play to the complementary advantages between tasks. Summary of the Invention

[0003] The technical problems to be solved by this invention are poor imaging quality, insufficient utilization of posture information, fragmented algorithms, and lack of end-to-end optimization. Specifically, in low-light and complex posture oilfield monitoring environments, the recognition accuracy is low and the robustness is poor. The purpose is to provide a method, device, and medium for personnel behavior recognition in low-light scenarios, solving the problem of low accuracy in personnel behavior recognition in low-light monitoring environments such as oilfields. In particular, for situations where images are dim and blurry and human postures are variable, a multi-task joint neural network model is proposed, which integrates image enhancement, human posture estimation, and personnel behavior recognition into one. The multi-task neural network model is trained based on posture-weighted global pooling consistency and posture-driven background random suppression. Through feature sharing and task collaborative training, the utilization of posture information is improved, and the robustness and accuracy of personnel behavior recognition in complex environments are enhanced, achieving end-to-end optimization.

[0004] This invention is achieved through the following technical solution:

[0005] The first aspect of this invention provides a method for recognizing human behavior in low-light scenes, comprising the following specific steps:

[0006] A multi-task neural network model is constructed by feeding the input image into a basic feature extraction module that includes cross-channel attention and multi-scale fusion modules in a convolutional neural network.

[0007] The backbone network based on the feature extraction module performs layer-by-layer convolution operations on the input image and outputs an intermediate feature representation that integrates human morphological features and environmental context information. The intermediate feature representation is a multi-dimensional feature tensor.

[0008] The intermediate feature representations are used as shared feature maps and simultaneously input into multiple task sub-networks:

[0009] The image enhancement branch network upsamples and reconstructs the shared feature maps step by step to generate enhanced images that highlight action-related human details;

[0010] The pose estimation branch network generates heatmaps of key points of each human body based on shared feature maps, parses the two-dimensional coordinates based on the heatmaps, explicitly encodes the skeleton structure prior, and outputs the prediction results of human body key points.

[0011] The behavior recognition branch network, based on the shared feature map and the output of the pose estimation branch network, performs action category determination and outputs the behavior category prediction result.

[0012] Based on the behavior category prediction results, a multi-task neural network model is trained using pose-weighted global pooling consistency and pose-driven background stochastic suppression to obtain the final human behavior recognition model, which is used for human behavior recognition in real-world scenarios.

[0013] Furthermore, the image enhancement branch network includes:

[0014] An encoder-decoder structure is used to perform brightness enhancement, contrast enhancement, and noise reduction on important target areas of the input frame.

[0015] The fusion skip connection passes the final feature map of the base network through deconvolution / upsampling layers, gradually enlarging it to the original image size;

[0016] During the upsampling process, the base features of the corresponding level are fused at each level of upsampling;

[0017] The enhanced output image is compared with a brighter version of the original input image. The enhancement loss is calculated and the enhanced image is used to train the image enhancement branch network, resulting in the trained image enhancement branch network.

[0018] Furthermore, the enhancement loss calculation process includes:

[0019] The enhanced image output by the enhanced branch and the corresponding reference image are both normalized and resized.

[0020] Calculate the structural similarity loss between the enhanced image and the reference image;

[0021] Calculate the pixel-by-pixel loss between the enhanced image and the reference image;

[0022] The two losses mentioned above are combined using a weighted summation method to obtain the image enhancement loss.

[0023] Furthermore, the pose estimation branch network specifically includes:

[0024] The shared feature map is processed by the pose estimation head layer by layer through a channel reduction method, and the output is a tensor of size H×W×K, which is the predicted heatmap of K joints. Each channel value represents the probability that the joint appears in the corresponding spatial location.

[0025] Obtain the labeled two-dimensional joint coordinates, generate a two-dimensional Gaussian kernel centered on the coordinates on the H×W plot of each joint k, and obtain the true heatmap after normalization.

[0026] The predicted heatmap is compared pixel by pixel with the true heatmap, and the attitude loss is calculated using mean square error and the weights are updated in reverse.

[0027] Until the weights reach the set threshold, the soft-argmax operation is used to calculate the expected position for each predicted heatmap, thus obtaining the trained pose estimation branch network.

[0028] The pose estimation branch network outputs the joint coordinates based on the trained pose estimation.

[0029] Furthermore, the soft-argmax operation applied to each predicted heatmap specifically includes:

[0030] Perform a soft-argmax operation on each heatmap: Calculate the expected coordinates in the x and y directions, respectively, using the weights:

[0031] ;

[0032] in, For the first Subpixel-level coordinate estimation of joints The pixel value at position (x, y) of the k-th joint heatmap output by the network is normalized to the interval [0, 1]. To exponentially assign values ​​to the heatmap, the confidence level in the range [0,1] is converted into a non-negative weight. The heatmap of the k-th joint output by the network is located at... The pixel value at that location.

[0033] Furthermore, the behavior recognition branch network specifically includes:

[0034] Global average pooling is performed on the shared feature map to obtain the global feature vector;

[0035] The global feature vector is fused with the joint features obtained from pose estimation to form a comprehensive feature vector;

[0036] For each joint position, the corresponding local feature vector is extracted from the base feature map, and the feature vectors of all joints are concatenated to form a pose feature representation.

[0037] The pose features are concatenated with the global features and then input into the subsequent fully connected classification layer.

[0038] After several fully connected layers and non-linear activations, the output is a probability vector of dimension N, where N is the number of defined behavior categories;

[0039] The category with the highest probability is selected as the recognition result.

[0040] Furthermore, the attitude-weighted global pooling consistency specifically includes:

[0041] Map the input video frame sequence to a shared feature map;

[0042] A salient mask for the character is constructed based on K joint heatmaps output from the pose estimation branch;

[0043] Using the aforementioned character salient mask, ordinary global average pooling representation and pose-weighted global pooling representation are computed in parallel on the shared feature map;

[0044] The ordinary global average pooling representation and the pose-weighted global pooling representation are fed into the same linear classifier to obtain the first classification score and the second classification score, respectively.

[0045] A symmetrical temperature-consistent distillation loss is constructed, using the symmetrical temperature-consistent distillation loss between the first classification score and the second classification score as a pooling consistency constraint.

[0046] Furthermore, the pose-driven background random suppression specifically includes:

[0047] The dilation result is obtained by performing small-radius morphological dilation on a prominent mask of a person.

[0048] Based on the dilation results and the total image pixels, determine the background set;

[0049] Random suppression is performed on pixels in the background set with probability p at the feature layer to obtain the background-suppressed features.

[0050] Spatial attention weight map is generated using pose heatmap, and the background-suppressed features are weighted to obtain weighted features for pose-heuristic attention fusion.

[0051] The background-suppressed features are simultaneously fed into the image enhancement branch, pose estimation branch, and action recognition branch to obtain the image enhancement loss, pose estimation loss, action classification loss, and feature alignment loss, and the overall loss function is constructed.

[0052] End-to-end joint optimization is performed using the overall loss function. Iterative training continues until the overall loss function converges. The parameters of the shared feature extraction module and the behavior recognition branch are fixed. The image enhancement decoder and pose estimation head are removed from the model to obtain the final human behavior recognition model.

[0053] A second aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for recognizing human behavior in low-light scenarios.

[0054] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for recognizing human behavior in low-light scenarios.

[0055] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0056] Improve recognition rate in low-light images: The image enhancement branch effectively eliminates the interference of low light and noise on feature extraction, making the outline of people and the details of their movements clearer. Compared with the original image without enhancement, the model's adaptability to low-light environments is greatly improved, and it can accurately identify illegal / dangerous behaviors of people in nighttime or underground scenes.

[0057] Improving discriminative power by leveraging pose priors: The pose estimation branch provides skeletal information about the human body for behavior recognition, enabling the model to understand the specific movements and postures of a person. For example, the difference between a worker bending over to pick up an item and standing normally can be captured by the model through changes in the position of joints, thus enabling accurate classification. This pose-inspired mechanism is particularly helpful in distinguishing action categories that are difficult to discern from appearance alone, improving the model's sensitivity to subtle differences in movements.

[0058] Robust feature representation for task collaboration: Multi-task joint training enables the model to learn more robust feature representations and has a stronger ability to resist various interference factors. Compared with the model trained on a single task, the model of this invention can still make correct judgments by relying on the context and pose cues in the shared features when encountering situations such as complex backgrounds, partially occluded characters, and changes in camera angles. This reduces the model's dependence on a single signal and improves generalization performance.

[0059] Efficient deployment: Since only the behavior recognition backbone can be run during the inference phase, the model computational overhead is small, which can meet the requirements of real-time monitoring. Even when monitoring multiple camera videos at the same time, the method of this invention can still run smoothly on general GPU workstations or edge computing devices, realizing large-scale real-time behavior analysis of the production site. This provides a practical and feasible technical means for oilfield safety management.

[0060] In summary, this invention, through multi-task fusion of image enhancement, pose estimation, and behavior recognition, significantly improves the accuracy and reliability of automatic identification of personnel violations / dangerous behaviors in low-light monitoring in the petroleum industry, addressing the challenge of such monitoring in such conditions. It has significant practical value. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0062] Figure 1 This is a schematic diagram of the overall architecture of the multi-task joint behavior recognition model in an embodiment of the present invention;

[0063] Figure 2This is a flowchart illustrating the training and inference phases in an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0065] As one possible implementation, this embodiment provides a method for human behavior recognition in low-light scenes. The core of this method is the integration of image enhancement and pose estimation into the behavior recognition model. By combining multi-task fusion and pose heuristics, it can improve the behavior recognition performance in low-quality videos, solving the recognition difficulties under low-light conditions. A unified multi-task deep neural network framework is proposed. Taking surveillance video frames as input, it first obtains intermediate representations that combine human form and environmental context through a shared feature extraction base constructed by a convolutional neural network. Based on this, three tasks are launched in parallel:

[0066] One branch is image enhancement, which uses encoder-decoder upsampling and reconstruction to improve brightness and contrast and suppress noise, thereby highlighting human body details related to movement.

[0067] The second branch is the pose estimation branch, which outputs heatmaps of key points of each human body and obtains two-dimensional coordinates, so that shared features explicitly encode the skeleton structure prior.

[0068] The third branch is behavior recognition, which determines the action category based on shared features (such as normal operation, illegal / dangerous action, fall, etc.).

[0069] The behavior recognition branch introduces a pose-inspired fusion mechanism: attention weights are generated from pose heatmaps, and human body regions are given higher weights and background interference is reduced on the feature map, thereby more accurately distinguishing actions that look similar but have different pose patterns. Overall, the three tasks are jointly optimized and mutually promoted under a unified architecture to achieve robust recognition and early warning of human behavior in low-light and complex scenes.

[0070] This embodiment constructs a shared feature extraction network through (1) a multi-task joint architecture, and outputs the results of three tasks: image enhancement, pose estimation, and behavior recognition, to achieve multi-task collaborative optimization. Compared with the existing schemes that separate each task, this architecture enables each task to share information and promote each other, taking into account both image quality improvement and high-level semantic understanding in a single model. This end-to-end multi-task learning design can be widely applied to various video surveillance systems, not limited to oilfield scenarios, and can be used as long as target behavior recognition is involved in situations with poor image quality. (2) Image enhancement and feature sharing: an image enhancement method oriented towards core targets is proposed, which enhances the contextual features of key targets through task sharing. Unlike general global image enhancement algorithms, the enhancement branch is driven by the behavior recognition task, focusing on improving the brightness and clarity of the human area and filtering out background interference. (3) Pose-inspired behavior recognition: human pose estimation is introduced as an auxiliary sub-task, and the accuracy of behavior recognition is improved through pose-inspired fusion. Compared with the pure visual feature classification method, using joint heatmaps or skeleton information to inspire the model to focus on the essential features of human movements is an important improvement to existing behavior recognition technology. (4) Training-inference optimization strategy: An optimization strategy of multi-task training and single-task inference was designed to ensure both model performance and actual deployment efficiency.

[0071] The specific implementation method of this embodiment is as follows:

[0072] A. Shared feature extraction base network;

[0073] Video streams are collected by surveillance cameras deployed at key locations in the oil field and input into a multi-task neural network model. Assuming the input frame resolution is... The base network of the model will perform downsampling convolutional feature extraction on pixels, such as... Figure 1 As shown, this is the basic feature extraction module built using a convolutional neural network (such as the backbone network EfficientNet). This base network extracts intermediate feature representations that integrate human morphology and environmental context, providing a common feature foundation for subsequent tasks. To extract key details under low-light conditions, the base network introduces multi-scale feature extraction in its structure, enhancing its ability to represent human contours and motion details in low-light and noisy environments.

[0074] B. Image Enhancement Branch;

[0075] The shared base features obtained from the shared base network are processed through upsampling, convolutional reconstruction, and other modules to output an enhanced image or feature map. To adapt to surveillance footage with mixed infrared and low-light visible light at night, the base network is designed to include cross-channel attention and multi-scale fusion modules: the former is used to highlight the response of the human target on the feature map (suppressing irrelevant activations caused by strong noise), and the latter fuses features from different receptive fields to simultaneously capture the overall outline and local motion details of the human.

[0076] This embodiment removes the effects of low light and blur by enhancing the image, highlighting the clarity of people and their action-related areas. The image enhancement branch can employ an encoder-decoder structure to brighten, contrast, and denoise the input frame, resulting in a higher-quality image. Unlike traditional standalone image enhancement algorithms, the enhancement branch in this embodiment shares feature representations with behavior recognition. This means the enhancement network focuses on learning the contextual information and key parts of the person, making the enhancement result highlight behavior-related image details. For example, in dimly lit scenes, the enhancement branch automatically enhances the brightness and clarity of clothing and limb contours without excessively amplifying background noise.

[0077] Specifically, the pedestal features obtained through the shared base network are upsampled stepwise and fused with skip connections to restore image resolution, while enhancing important target regions. The steps are as follows: (1) The final feature map of the pedestal network is passed through a series of deconvolution / upsampling layers and enlarged to the original image size step by step. In order to preserve details, the pedestal features (U-shaped structure) of the corresponding level are fused at each upsampling level to ensure that the enhancement result has both global brightness improvement and local texture details. (2) The last layer outputs the enhanced image. During training, Compared with the original input image Compare with the brightness-enhanced version and calculate the enhancement loss. In this embodiment, the structural similarity index (SSIM) and the L1 norm are used as the loss function:

[0078] ;

[0079] Among them, the enhanced branch output image is The reference / target image is Both are normalized to The size is Here, C represents the number of image channels. In this embodiment, modeling is for RGB images, so C is 3. Let the pixel set... , always like . in the formula , This refers to the weighting coefficients (adjustable hyperparameters). In practice, it's common practice to make the two quantities comparable; for example, by first observing two numerical scales and then setting... .

[0080] C. Attitude estimation branch;

[0081] The shared features obtained from the shared base network are simultaneously input into the pose estimation module, which outputs the prediction results of human key points. The introduction of the pose estimation branch allows the network to explicitly focus on human structural information during feature learning: the shared features are trained to represent the layout of the human skeleton. Especially in action recognition, different actions correspond to different skeletal movement patterns. These patterns are captured by the pose branch and used to inspire behavior recognition, thereby improving the discrimination ability.

[0082] Specifically, the pose estimation head consists of several convolutional layers that progressively transform the channel dimensions into a keypoint heatmap output. It defines a set of key human joints (such as eyes, nose, ears, shoulders, elbows, wrists, hips, knees, ankles, etc.). The output size of the last layer of the pose estimation head is... The heatmap tensor contains a predicted probability distribution for each joint, with each channel corresponding to that joint. During model training, the predicted heatmap is compared pixel-by-pixel with the ground truth heatmap, and the pose loss is calculated using mean squared error. To obtain the coordinates of the joint points, this embodiment uses a soft-argmax operation to calculate the desired location for each heatmap (assuming the first...). The heat map of each joint is as follows ): ;

[0083] Received That is, the first Subpixel-level coordinate estimation of joints. The pose estimation branch also provides salient information about the person obtained by aggregating heatmaps of each joint during the training phase, which is used to guide the training of the behavior recognition branch; this branch is pruned during the inference phase and does not participate in the calculation.

[0084] D. Behavior recognition branch;

[0085] The behavior recognition module needs to determine the category of a person's current action based on shared features and human posture features learned heuristically from the pose estimation branch task. In this embodiment, we define several common personnel behavior patterns in oil fields, such as: normal inspection, equipment operation, climbing, falls / slips, and loitering in unauthorized areas, some of which are violations / dangerous behaviors. The behavior recognition branch uses shared feature maps... As input, the data is processed through several convolutional and fully connected layers to output categories of human actions / behaviors (such as normal work, violations / dangerous actions, falls, etc.). The main classification loss is defined by cross-entropy. :

[0086] Feature fusion: The global feature vector of the pedestal network (obtained by global average pooling of the feature map) is fused with the joint coordinates / features obtained from pose estimation to form a comprehensive feature vector. In implementation, we extract the corresponding local feature vector from the pedestal feature map for each joint position (e.g., features at the joint point obtained by bilinear interpolation feature map), and concatenate the feature vectors of all joints to form a pose feature representation. Then, the pose features are concatenated with the global features and input into the subsequent fully connected classification layer.

[0087] Classification and recognition: After several fully connected layers and nonlinear activation, the output dimension is... The probability vector, where The number of defined behavior categories. The category with the highest probability is selected as the recognition result. Cross-entropy loss is used during training. make Approaching realistic action tags .

[0088] (5) To further enhance the attention to the character area and suppress background dependence, this embodiment innovatively introduces two mechanisms in this branch during the training phase to further improve its robustness:

[0089] Pose-weighted global pooling consistency:

[0090] In this embodiment, a significant mask for the subject is constructed based on the pose heatmap during the training phase, and a pooling consistency constraint is established.

[0091] For shared feature maps The character saliency mask is obtained by summing all joint heatmaps along the channel dimension, truncating and normalizing them.

[0092] Among them, H k This indicates that for the k-th joint, S represents the sum of all pixel values ​​in the salient mask of the person, and S(x,y) represents the specific value of S at the image coordinates (x,y).

[0093] exist The above computes two global representations:

[0094] Normal global average pooling: Where H' represents the actual height of feature map F, W' represents the actual width of feature map F, and F(x,y) represents the entire channel vector at coordinates (x,y).

[0095] Posture-weighted pooling: ;in, This represents the sum of all pixel values ​​in the saliency mask of the person at coordinates (x, y).

[0096] Both representations share the same linear classifier The classification scores were obtained respectively. , ; where z gap z represents the length of the global branch. pw Indicates the length of the character's branch path;

[0097] To ensure consistency between the "view the whole image" and "view only the person" criteria, a symmetrical temperature-based consistency distillation loss (temperature) is introduced. ):

[0098] Where KL represents the consistency loss;

[0099] This constraint prompts the classifier to internalize person-priority preferences during training, but only retains them during the inference phase. There is no need to calculate the pose mask or add branches, and the deployment overhead remains unchanged.

[0100] Pose-driven background random suppression;

[0101] To reduce reliance on scene / background textures, this embodiment is based on Lightweight stochastic suppression is implemented at the feature layer. Specifically, for The expansion results were obtained by performing small-radius morphological expansion. , and define the background set accordingly.

[0102] Unlike traditional classifiers trained separately, the behavior recognition branch in this embodiment benefits from feature assistance from two other tasks: on the one hand, the shared features have been optimized by the image enhancement task and contain clearer image information; on the other hand, the human keypoint information provided by the pose estimation branch can be fused into the behavior recognition process to realize a pose-inspired recognition mechanism.

[0103] Pose-inspired learning mechanism: In a preferred embodiment of this example, the behavior recognition branch combines the output of the pose branch with a specific fusion strategy to improve classification performance. Attention fusion is employed: an attention weight matrix is ​​generated based on the pose heatmap, assigning higher weights to regions belonging to human body parts on the shared feature map and lower weights to background regions, thereby inspiring the classifier to focus on the person's own action information. Through this fusion, the behavior recognition branch can more accurately distinguish similar actions using pose priors (for example, waving and reaching out may be visually similar, but the distribution of pose key points is different, making them easier to distinguish after fusion).

[0104] The three tasks described above are carried out in parallel within the same neural network architecture, achieving complementary advantages by sharing underlying features. It is particularly noteworthy that this embodiment employs a multi-task joint training strategy: during training, the network simultaneously optimizes three objective functions—image enhancement, pose estimation, and action recognition—forming a single overall loss function.

[0105] ;

[0106] in, Image enhancement loss, such as enhancing the output image. Compared with the reference truth image Pixel reconstruction errors (such as SSIM loss) between pixels; For attitude estimation loss, the mean square error between the key point heatmap and the ground truth heatmap or the key point coordinate error can be used. For behavior classification loss, cross-entropy loss is typically used to measure the predicted category. and The differences between them; , , To balance the weights of each task, the base network's parameters are simultaneously used for enhancement, pose estimation, and recognition through joint backpropagation of multi-task losses. This results in learning more robust and discriminative intermediate feature representations. This end-to-end joint optimization significantly outperforms traditional methods that train each task separately and then concatenate them, fully leveraging the inter-task relationships: image enhancement forces features to focus on lighting and texture details, while pose estimation forces features to encode human structural information, directly increasing the effective feature quantity for action recognition tasks.

[0107] Furthermore, this embodiment also designs a decoupling strategy for the training and inference phases: during the training phase, the model is trained using multiple task objectives, while during actual inference deployment, only the behavior recognition backbone is retained for output results. For example... Figure 2 As shown, during the inference phase, the network structure can be selectively simplified as needed: since image augmentation and pose branches primarily provide auxiliary supervision during training, their inference outputs are not essential for the final alarm decision, and therefore these branches can be pruned during deployment to reduce computation. Specific implementation methods include:

[0108] Branch pruning: After joint training is completed, the parameters of the shared pedestal and action recognition branches are fixed, and the image enhancement decoder and pose estimation head are pruned and removed from the model. Since these two branches have already guided the pedestal to learn features that are beneficial to recognition during training, pruning will not significantly affect the accuracy of action recognition, but will significantly reduce the number of model parameters and computational cost during inference, thus improving real-time performance.

[0109] As one possible implementation, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a method for recognizing human behavior in low-light scenarios.

[0110] As one possible implementation, this embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for recognizing human behavior in low-light scenarios.

[0111] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for recognizing human behavior in low-light scenes, characterized in that, The specific steps include the following: Construct a multi-task neural network model and feed the input image into the basic feature extraction module built by the convolutional neural network; The backbone network based on the basic feature extraction module performs layer-by-layer convolution operations on the input image and outputs an intermediate feature representation that integrates human morphological features and environmental context information; wherein, the backbone network includes cross-channel attention and multi-scale fusion modules, and the intermediate feature representation is a multi-dimensional feature tensor; The intermediate feature representations are used as shared feature maps and simultaneously input into multiple task sub-networks: The image enhancement branch network upsamples and reconstructs the shared feature maps step by step to generate enhanced images that highlight action-related human details; The pose estimation branch network generates heatmaps of key points of each human body based on shared feature maps, parses the two-dimensional coordinates based on the heatmaps, explicitly encodes the skeleton structure prior, and outputs the prediction results of human body key points. The behavior recognition branch network, based on the shared feature map and the output of the pose estimation branch network, performs action category determination and outputs the behavior category prediction result. Based on the behavior category prediction results, a multi-task neural network model is trained using pose-weighted global pooling consistency and pose-driven background stochastic suppression to obtain the final human behavior recognition model, which is used for human behavior recognition in real-world scenarios. Specifically, the attitude-weighted global pooling consistency includes: Map the input video frame sequence to a shared feature map; A salient mask for the character is constructed based on K joint heatmaps output from the pose estimation branch; Using the aforementioned character salient mask, ordinary global average pooling representation and pose-weighted global pooling representation are computed in parallel on the shared feature map; The ordinary global average pooling representation and the pose-weighted global pooling representation are fed into the same linear classifier to obtain the first classification score and the second classification score, respectively. A symmetrical temperature-consistent distillation loss is introduced, using the symmetrical temperature-consistent distillation loss between the first classification score and the second classification score as a pooling consistency constraint. The formula for calculating the symmetrical temperature-consistent distillation loss is as follows: ； Where KL represents the consistency loss, z gap z represents the score of the first category. pw This represents the second category score, and T represents the temperature parameter. ; The attitude-driven background random suppression specifically includes: The dilation result is obtained by performing small-radius morphological dilation on a prominent mask of a person. Based on the dilation results and the total image pixels, determine the background set; Random suppression is performed on pixels in the background set with probability p at the feature layer to obtain the background-suppressed features. Spatial attention weight map is generated using pose heatmap, and the background-suppressed features are weighted to obtain weighted features for pose-heuristic attention fusion. The background-suppressed features are simultaneously fed into the image enhancement branch, pose estimation branch, and action recognition branch to obtain the image enhancement loss, pose estimation loss, action classification loss, and symmetric temperature-consistent distillation loss, and the overall loss function is constructed. End-to-end joint optimization is performed using the overall loss function, and iterative training is conducted until the overall loss function converges. The parameters of the shared feature extraction module and the behavior recognition branch are fixed, and the image enhancement decoder and pose estimation head are removed from the model to obtain the final human behavior recognition model.

2. The method for recognizing human behavior in low-light scenes according to claim 1, characterized in that, The image enhancement branch network includes: An encoder-decoder structure is used to perform brightness enhancement, contrast enhancement, and noise reduction on important target areas of the input frame. The fusion skip connection passes the final feature map of the backbone network through deconvolution / upsampling layers, gradually enlarging it to the original image size; During the upsampling process, intermediate features from the corresponding level are fused at each level of upsampling. The enhanced output image is compared with a brighter version of the original input image. The enhancement loss is calculated and the enhanced image is used to train the image enhancement branch network, resulting in the trained image enhancement branch network.

3. The method for recognizing human behavior in low-light scenes according to claim 2, characterized in that, The enhancement loss calculation process includes: The enhanced image output by the enhanced branch and the corresponding reference image are both normalized and resized. Calculate the structural similarity loss between the enhanced image and the reference image; Calculate the pixel-by-pixel loss between the enhanced image and the reference image; The two losses mentioned above are combined using a weighted summation method to obtain the image enhancement loss.

4. The method for recognizing human behavior in low-light scenes according to claim 1, characterized in that, The pose estimation branch network specifically includes: The shared feature map is processed by the pose estimation head layer by layer through a channel reduction method, and the output is a tensor of size H×W×K, which is the predicted heatmap of K joints. Each channel value represents the probability that the joint appears in the corresponding spatial location. Obtain the labeled two-dimensional joint coordinates, generate a two-dimensional Gaussian kernel centered on the coordinates on the H×W plot of each joint k, and obtain the true heatmap after normalization. The predicted heatmap is compared pixel by pixel with the true heatmap, and the attitude loss is calculated using mean square error and the weights are updated in reverse. Until the weights reach the set threshold, the soft-argmax operation is used to calculate the expected position for each predicted heatmap, thus obtaining the trained pose estimation branch network. The pose estimation branch network outputs the joint coordinates based on the trained pose estimation.

5. The method for recognizing human behavior in low-light scenes according to claim 4, characterized in that, The soft-argmax operation applied to each predicted heatmap specifically includes: Perform a soft-argmax operation on each heatmap: Calculate the expected coordinates in the x and y directions, respectively, using the weights: ； in, For the first Subpixel-level coordinate estimation of joints The pixel value at position (x, y) of the k-th joint heatmap output by the network is normalized to the interval [0, 1]. To exponentially assign values ​​to the heatmap, the confidence level in the range [0,1] is converted into a non-negative weight. The heatmap of the k-th joint output by the network is located at... The pixel value at that location.

6. The method for recognizing human behavior in low-light scenes according to claim 1, characterized in that, The behavior recognition branch network specifically includes: Global average pooling is performed on the shared feature map to obtain the global feature vector; For each joint position, the corresponding local feature vector is extracted from the intermediate feature map, and the feature vectors of all joints are concatenated to form a pose feature representation. Pose features and global features are concatenated and fused to form a comprehensive feature vector, which is then input into the subsequent fully connected classification layer; After several fully connected layers and non-linear activations, the output is a probability vector of dimension N, where N is the number of defined behavior categories; The category with the highest probability is selected as the recognition result.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for human behavior recognition in low-light scenarios as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for human behavior recognition in low-light scenarios as described in any one of claims 1 to 6.

Images