An animal behavior recognition software system for low-light scenes

Abstract

The application provides an animal behavior recognition software system for low-illumination scenes, relates to the technical field of image processing, and comprises a data acquisition module, an image information enhancement module, a lightweight animal behavior recognition module, a data storage module, a user interface module and a system management module. The application acquires multiple images under low-illumination scenes through the data acquisition module. In the image enhancement module, the illumination of low-illumination images is improved through an illumination estimation module, and then the noise of the low-illumination images is removed through a noise suppression module. At the same time, the edge information of the low-illumination images is enhanced through an edge information enhancement module, so that the illumination and the edge information of the low-illumination images are both enhanced. Then, when the image with enhanced features is recognized through the lightweight animal behavior recognition module, the accuracy of animal recognition can be improved. Meanwhile, the application has the characteristic of being lightweight, and high-efficiency processing can be realized without relying on high-performance computing devices.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, in particular to an animal behavior recognition software system for low-illumination scenes. Background Art

[0002] In the medical field, drug research can be achieved through animal behavior observation experiments. Specifically, drugs are injected into laboratory mice, and their behavior is observed to determine the effectiveness of the drugs.

[0003] Since strong light exposure has an impact on laboratory mice, when conducting animal behavior observation experiments, it is impossible to determine whether the behavior of the laboratory mice is caused by strong light exposure or drugs. Therefore, in order to avoid the impact of strong light exposure, some experiments need to be conducted in low-light scenes. In the images collected in low-light scenes, the color distinction between the laboratory mice and the environment is low. In order to realize the recognition of the behavior of laboratory mice, it is currently possible to capture images through ordinary cameras and increase the image brightness, or to capture images and recognize them through infrared cameras, or to achieve it through a combination of ordinary cameras and infrared cameras.

[0004] However, when the image brightness is increased, the brightness of the laboratory mouse and the environment in the image is also increased, resulting in inaccurate behavioral recognition of the laboratory mouse. When the image is collected by an infrared camera, the edge contours of the laboratory mouse are not clear, for example, the claws and tail of the laboratory mouse are not clear, resulting in low recognition accuracy, and the cost of using infrared cameras is high. When it is implemented by combining ordinary cameras and infrared cameras, due to the different frame rates collected by the two cameras, the time of the collected images may be different, and there is a deviation in the viewing angles of the two cameras, resulting in errors when the image features of the two cameras are fused, resulting in low recognition accuracy, and this method is highly complex and cannot be monitored in real time. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to propose an animal behavior recognition software system for low-light scenes, comprising: a data acquisition module, an image information enhancement module, a lightweight animal behavior recognition module, a data storage module, a user interface module, and a system management module;

[0006] The data acquisition module is used to collect multiple frames of low-light images in a low-light scene in real time, wherein the multiple frames of low-light images in the low-light scene include animals and background environment, and the animals and the background environment cannot be distinguished, convert the images according to a preset format to obtain multiple frames of low-light images after format conversion, arrange the multiple frames of low-light images after format conversion in chronological order to obtain a low-light image sequence, and input the low-light image sequence into the image information enhancement module;

[0007] The image information enhancement module is used to enhance the illumination and edge information of each low-illumination image frame in the low-illumination image sequence. Specifically, for each low-illumination image frame in the low-illumination image sequence, the illumination estimation module in the image information enhancement module is input to obtain an illumination-enhanced feature map, and the illumination-enhanced feature map is input to the noise suppression module in the image information enhancement module to obtain a denoised feature map. At the same time, each low-illumination image frame in the low-illumination image sequence is input to the edge information enhancement module in the image information enhancement module to obtain an edge information-enhanced feature map, and the denoised feature map and the edge information-enhanced feature map are spliced ​​and superimposed in the dimension of the image channel to obtain a multi-channel feature map. The multi-channel feature map is processed by a channel attention mechanism to obtain a mixed feature map, and then a mixed feature map corresponding to each low-illumination image frame is obtained. The mixed feature map corresponding to each low-illumination image frame is input to a lightweight animal behavior recognition module;

[0008] The lightweight animal behavior recognition module is used to extract spatial features from the mixed feature map corresponding to each frame of low-light image based on the spatial feature extractor contained in the module. The spatial feature extractor is a lightweight deep learning network MobileNetV3. Then, based on the Transformer encoder, temporal features are extracted from the feature maps containing spatial features in multiple frames to obtain a feature map containing temporal features and spatial features, thereby identifying the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior;

[0009] The data storage module is used to store the video in the low-light scene collected in real time by the data acquisition module, and store the behavior of the animal in the video obtained by identification, and the probability that the animal in the video belongs to the behavior;

[0010] The user interface module is used to display the video in the low-light scene collected in real time by the data acquisition module, and store the behavior of the animal in the video obtained by identification, and the probability of the animal in the video belonging to the behavior;

[0011] The system management module is used to coordinate and manage the operation of various modules in the animal behavior recognition software system for low-light scenes.

[0012] Optionally, the illumination estimation module in the image information enhancement module is specifically configured to:

[0013] For each frame image in the low-illumination image sequence, input it into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and input the feature map output by the convolution layer into the Relu function to realize the numerical conversion of the eigenvalues ​​in the feature map of the multi-channel; input the numerically converted feature map into the residual structure, specifically, input the numerically converted feature map into a convolution layer with a convolution kernel size of 3*3, a stride of the convolution operation of 1, and a padding of the convolution operation of 1, and then perform batch normalization on the feature map output by the convolution layer, and pass the batch normalized feature map through the Relu function to obtain a first feature map, and superimpose the first feature map and the numerically converted feature map to obtain a multi-scale feature map; input the multi-scale feature map into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and then input the feature map processed by the convolution layer into the Sigmod function to obtain an illumination enhanced feature map.

[0014] Optionally, the noise suppression module in the image information enhancement module is specifically configured to:

[0015] Filling the illumination enhancement feature map corresponding to each frame image in the low illumination image sequence, wherein the illumination enhancement feature map includes multiple feature points,

[0016] Specifically, the filling size is determined by the following formula:

[0017]

[0018] Among them, pad_size is the padding size, R is the preset search radius in the noise reduction process, and P is the size of the feature block where the preset feature point is located;

[0019] The illumination enhanced feature map is padded based on the padding size. Specifically, the height of the illumination enhanced feature map is increased by 2 times the pad_size, the width of the illumination enhanced feature map is increased by 2 times the pad_size, and the feature values ​​of the added feature points are padded to obtain the padded feature map;

[0020] For each feature point in the filled feature map, determine the neighborhood block corresponding to the feature point. The neighborhood block is expressed by the following formula:

[0021]

[0022] Among them, N i,j Represents the neighborhood block of the feature point, that is, the set of neighborhood points corresponding to the feature point, (i, j) represents the coordinates of the feature point in the filled feature map, x i+m,j+ndenotes the feature value of the neighborhood point corresponding to the feature point, (i+m, j+n) denotes the coordinates of the neighborhood point in the filled feature map, m denotes the displacement amount of the horizontal coordinate, and n denotes the displacement amount of the vertical coordinate;

[0023] The search range is determined with the feature point as the center and R as the radius, for each other feature point in the search range except the feature point, the neighborhood block N k,l (k, l) is the coordinates of the other feature point in the filled feature map, and then the similarity of the neighborhood block N i,j and the neighborhood block N k,l is calculated, which is specifically realized by the following formula:

[0024]

[0025] wherein d(N i,j , N k,l ) denotes the similarity of the neighborhood block N i,j and the neighborhood block N k,l , and x k+m,l+n denotes the feature value of the neighborhood point corresponding to the other feature point;

[0026] Based on the similarity of the neighborhood block N i,j and the neighborhood block N k,l , the similarity weight is calculated, which is specifically represented by the following formula:

[0027]

[0028] wherein ω i,j,k,l denotes the similarity weight, and h denotes a parameter for controlling the degree of noise filtering;

[0029] Based on the above calculation, the similarity weight of each feature point is obtained, and then for each feature point, the cumulative weighted feature value is calculated, which is calculated by the following formula:

[0030]

[0031] wherein x k,l denotes the feature value of the other feature point x k,l , and denoised i,j denotes the cumulative weighted feature value;

[0032] The cumulative weighted feature value is divided by the total weight sum to obtain the final denoising result, which is represented by the following formula:

[0033]

[0034] wherein denoised_result i,jRepresents the final denoising result of each feature point, ε is a constant;

[0035] Thus, the final denoising results of all feature points are obtained, that is, the denoised feature map.

[0036] Optionally, the edge information enhancement module in the image information enhancement module is specifically used to:

[0037] Inputting each frame of the low-light image sequence into a first Laplace function to obtain a first scale feature map, and inputting each frame of the low-light image sequence into a second Laplace function to obtain a second scale feature map, where a size of the first scale feature map is larger than a size of the second scale feature map;

[0038] Using the Thor operator, information is extracted from the first-scale feature map to obtain a feature map containing vertical edge information and a feature map containing horizontal edge information. The feature map containing vertical edge information and the feature map containing horizontal edge information are fused to obtain a first fused feature map. The first fused feature map is input into a convolution layer with a convolution kernel size of 3×3 to extract deep features, thereby obtaining a feature map containing deep features of the first-scale feature map.

[0039] Using the Thor operator, information is extracted from the second-scale feature map to obtain a feature map containing vertical edge information and a feature map containing horizontal edge information. The feature map containing vertical edge information and the feature map containing horizontal edge information are fused to obtain a second fused feature map. The second fused feature map is input into a convolution layer with a convolution kernel size of 3×3 to extract deep features, thereby obtaining a feature map containing the deep features of the second-scale feature map.

[0040] The feature map containing the deep features of the second-scale feature map is upsampled, and the upsampled feature map is fused with the feature map containing the deep features of the first-scale feature map to obtain a feature map with enhanced edge information.

[0041] Optionally, the lightweight animal behavior recognition module is specifically used to:

[0042] For each mixed feature map corresponding to the low-illumination image, the mixed feature map is input into a spatial feature extractor to extract spatial features in the mixed feature map, to obtain a feature map containing spatial features, the feature map containing spatial features is linearly encoded to convert the channel dimension of the feature map containing spatial features into the channel dimension of the input of the Transformer encoder, and the feature map containing spatial features is positionally encoded, the linearly encoded feature map and the positionally encoded feature map corresponding to each low-illumination image in the low-illumination image sequence are fused to obtain a fused spatial feature map corresponding to all low-illumination images, the fused spatial feature map corresponding to all low-illumination images is input into the Transformer encoder, and the fused spatial feature map corresponding to all low-illumination images is extracted to obtain a feature map containing temporal features and spatial features, and then the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, and then a classifier is used to calculate the animal behavior to which the global features belong, that is, the animal behavior corresponding to the low-illumination image sequence, and the probability that the global features belong to the animal behavior, that is, the probability that the animal in the low-illumination image sequence belongs to the animal behavior.

[0043] Optionally, the feature map containing spatial features is positionally encoded, including:

[0044] Based on the dimension of the input data of the Transformer encoder, the range of the value of the position encoding dimension index corresponding to each feature point in the feature map containing spatial features is determined;

[0045] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing spatial features is even, the position encoding is calculated by the following formula:

[0046]

[0047] Where, PE pose,2index represents the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing spatial features is even, pose represents the position index of the low-illumination image in the low-illumination image sequence, index is the position encoding dimension index, that is, the feature index of the input data of the Transformer encoder, d model is the dimension of the input data of the Transformer encoder, and A is a scaling factor;

[0048] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing spatial features is odd, the position encoding is calculated by the following formula:

[0049]

[0050] Among them, PE pose,2index+1 Indicates the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing spatial features is an odd number.

[0051] Optionally, the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, which can be expressed by the following formula:

[0052]

[0053] Among them, global_features represents global features, feature z Represents the feature map containing temporal features and spatial features corresponding to the z-th frame low-light image, α z represents the weight calculated by the attention mechanism corresponding to the z-th low-light image frame, and T is the total number of low-light images in the low-light image sequence.

[0054] The beneficial effects of adopting the above technical solution are:

[0055] The present invention provides an animal behavior recognition software system for low-illumination scenes. In the image enhancement module, the illumination of the low-illumination image is improved by the illumination estimation module, and the noise of the low-illumination image is removed by the noise suppression module. At the same time, the edge information of the low-illumination image is enhanced by the edge information enhancement module, so that both the illumination and edge information of the low-illumination image are enhanced. Then, when the feature-enhanced image is recognized by the lightweight animal behavior recognition module, the accuracy of animal recognition can be improved. At the same time, when performing animal behavior recognition, the present invention adopts a lightweight deep learning network MobileNetV3 as a spatial feature extractor, and combines it with a Transformer encoder, so that the present invention can monitor the behavior of animals in real time, and the present invention does not need to rely on high-performance computing equipment to achieve efficient processing, which not only reduces the equipment cost, but also improves the applicability and flexibility of the software, so that it can be applied in more experimental environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of an animal behavior recognition software system for low-light scenes in an embodiment of the present invention;

[0057] Figure 2 This is a diagram of the architecture of a software system for animal behavior recognition in low-light scenarios according to an embodiment of the present invention;

[0058] Figure 3 Schematic diagram of the structure of an image enhancement module in an embodiment of the present invention;

[0059] Figure 4Schematic diagram of the structure of the illumination estimation module in an embodiment of the present invention;

[0060] Figure 5 Schematic diagram of the edge information enhancement module structure in an embodiment of the present invention;

[0061] Figure 6 Schematic diagram of the structure of a lightweight animal behavior recognition module in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0063] Aiming at the problems existing in the existing technology, combined with Figure 1 The present invention provides an animal behavior recognition software system for low-light scenes, including a data acquisition module, an image information enhancement module, a lightweight animal behavior recognition module, a data storage module, a user interface module, and a system management module. The present invention uses a camera to capture animal behavior videos in real time. These video data are quickly transmitted to the image information enhancement module via data, where advanced image enhancement technology is applied to improve the clarity, visibility, and edge information richness of the video under low-light conditions, ensuring the accuracy of subsequent processing. Next, the enhanced image data is Figure 1 The enhanced information in the data is input into the lightweight animal behavior recognition module. This module uses an optimized deep learning algorithm, which can operate efficiently in resource-limited environments and accurately identify the specific behaviors of animals. These recognition results include not only the type of behavior, but also precise timestamp information for subsequent analysis. The identified behavior data is stored in the data storage module together with the processed video data. This module is responsible for securely storing all data, providing a solid foundation for subsequent analysis, report generation and further model training. At the same time, the user interface module extracts the recognition results from the data storage module and displays detailed behavior information and related video clips to the user, allowing the user to intuitively understand the system's recognition effect and the animal's behavior patterns. Through this entire process, the system achieves efficient management from data collection to result display, providing users with powerful tools to study and analyze animal behavior.

[0064] Combine Figure 2 The six modules, data acquisition module, image information enhancement module, lightweight animal behavior recognition module, data storage module, user interface module, and system management module, are specifically used to:

[0065] The data acquisition module is used to collect multiple frames of low-light images in a low-light scene in real time, wherein the multiple frames of low-light images in the low-light scene include animals and background environment, and the animals and the background environment cannot be distinguished, convert the images according to a preset format to obtain multiple frames of low-light images after format conversion, arrange the multiple frames of low-light images after format conversion in chronological order to obtain a low-light image sequence, and input the low-light image sequence into the image information enhancement module;

[0066] Low illumination specifically refers to low light intensity in an environment. It's often used to describe scenes with insufficient lighting, such as at night, on cloudy days, or in dimly lit indoor spaces. In photography, videography, and visual perception, low illumination can affect image quality and visual effects, necessitating the use of specialized techniques or equipment to enhance light capture.

[0067] The data acquisition module of the present invention can acquire videos in low-light environments. Each frame of the video is a multi-frame low-light image in a low-light scene. The present invention can also directly acquire multiple frames of low-light images.

[0068] The data acquisition module is the starting point of the entire system, responsible for acquiring raw data from external devices and performing preliminary processing to ensure data quality. The data acquisition module needs to establish a connection with a camera or other sensor to achieve real-time acquisition of video streams or image data. In specific implementations, the camera captures videos or images containing animals in low-light environments (wherein the animals in this invention refer to laboratory mice), and transmits these videos or images containing animals in low-light environments to the data acquisition module. The data acquisition module supports multiple device interface protocols, such as USB and IP camera protocols, to accommodate different hardware devices.

[0069] Among them, when converting according to a preset format, specifically, the present invention pre-sets a system-unified format, and then converts the resolution, frame rate, etc. of multiple frames of low-light images into a system-unified format, so as to facilitate processing and analysis by the image information enhancement module and the lightweight animal behavior recognition module.

[0070] The image information enhancement module is used to enhance the illumination and edge information of each low illumination image in the low illumination image sequence. Specifically, Figure 3, for each frame of low-illumination image in the low-illumination image sequence, input the illumination estimation module in the image information enhancement module to obtain an illumination-enhanced feature map, input the illumination-enhanced feature map into the noise suppression module in the image information enhancement module to obtain a denoised feature map, and at the same time, input each frame of low-illumination image in the low-illumination image sequence into the edge information enhancement module in the image information enhancement module to obtain an edge information-enhanced feature map, and perform splicing and superposition processing on the denoised feature map and the edge information-enhanced feature map in the dimension of the image channel to obtain a multi-channel feature map, and process the multi-channel feature map through the channel attention mechanism to obtain a mixed feature map, and then obtain a mixed feature map corresponding to each frame of low-illumination image, and input the mixed feature map corresponding to each frame of low-illumination image into the lightweight animal behavior recognition module;

[0071] Among them, combined Figure 4 The illumination estimation module in the image information enhancement module is specifically used to:

[0072] For each frame image in the low-illumination image sequence, input it into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and input the feature map output by the convolution layer into the Relu function to realize the numerical conversion of the eigenvalues ​​in the feature map of the multi-channel; input the numerically converted feature map into the residual structure, specifically, input the numerically converted feature map into a convolution layer with a convolution kernel size of 3*3, a stride of the convolution operation of 1, and a padding of the convolution operation of 1, and then perform batch normalization on the feature map output by the convolution layer, and pass the batch normalized feature map through the Relu function to obtain a first feature map, and superimpose the first feature map and the numerically converted feature map to obtain a multi-scale feature map; input the multi-scale feature map into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and then input the feature map processed by the convolution layer into the Sigmod function to obtain an illumination enhanced feature map.

[0073] The noise suppression module in the image information enhancement module is specifically used to:

[0074] In order to process edge pixels, the image will be filled using the reverse filling method, and the illumination enhancement feature map corresponding to each frame image in the low-illumination image sequence will be filled. The illumination enhancement feature map contains multiple feature points.

[0075] Specifically, the filling size is determined by the following formula:

[0076]

[0077] Among them, pad_size is the padding size, R is the preset search radius in the noise reduction process, and P is the size of the feature block where the preset feature point is located;

[0078] The illumination enhanced feature map is padded based on the padding size. Specifically, the height of the illumination enhanced feature map is increased by 2 times the pad_size, the width of the illumination enhanced feature map is increased by 2 times the pad_size, and the feature values ​​of the added feature points are padded to obtain the padded feature map;

[0079] It should be noted that the size of the illumination enhanced feature map corresponding to each frame image of the present invention can be expressed as [B, C, H, W], where B is the batch size, C is the number of channels, H is the feature map height, and W is the feature map width. The size of the padded feature map is [B, C, H + 2 × pad_size, W + 2 × pad_size].

[0080] For each feature point in the filled feature map, determine the neighborhood block corresponding to the feature point. The neighborhood block is expressed by the following formula:

[0081]

[0082] Among them, N i,j Represents the neighborhood block of the feature point, that is, the set of neighborhood points corresponding to the feature point, (i, j) represents the coordinates of the feature point in the filled feature map, x i+m,j+n Represents the eigenvalue of the neighborhood point corresponding to the feature point, (i+m,j+n) represents the coordinates of the neighborhood point in the filled feature map, m represents the displacement of the horizontal coordinate, and n represents the displacement of the vertical coordinate;

[0083] With the feature point as the center and R as the radius, the search range is determined. For each feature point other than the feature point in the search range, the neighborhood block N of the other feature points is determined. k,l , (k, l) is the coordinate of other feature points in the filled feature map, and then calculate the neighborhood block N i,j and neighborhood block N k,l The similarity is achieved through the following formula:

[0084]

[0085] Among them, d(N i,j ,N k,l ) represents the neighborhood block N i,j and neighborhood block N k,l The similarity of x k+m,l+n Indicates the eigenvalues ​​of the neighborhood points corresponding to other feature points;

[0086] Based on the neighborhood block N i,j and neighborhood block Nk,l The similarity is calculated and the similarity weight is expressed by the following formula:

[0087]

[0088] Among them, ω i,j,k,l represents the similarity weight, h represents the parameter that controls the degree of noise filtering;

[0089] Based on the above calculation, the similarity weight of each feature point is obtained, and then the cumulative weighted feature value is calculated for each feature point. The specific calculation is performed using the following formula:

[0090]

[0091] Among them, x k,l Represents other feature points x k,l The eigenvalues ​​of i,j represents the cumulative weighted eigenvalue;

[0092] The final denoising result is obtained by dividing the accumulated weighted eigenvalues ​​by the accumulated weight sum, which is specifically expressed by the following formula:

[0093]

[0094] Among them, denoised_result i,j Represents the final denoising result of each feature point, ε is a constant, where ε is a small constant used to avoid division by 0;

[0095] This yields the final denoising result for all feature points, known as the denoised feature map. After the above steps, the denoised image is output, with the same shape as the input image (B, C, H, W). The denoising module effectively reduces image noise while preserving image detail and structure.

[0096] Among them, combined Figure 5 The edge information enhancement module in the image information enhancement module is specifically used to:

[0097] Inputting each frame of the low-light image sequence into a first Laplace function to obtain a first scale feature map, and inputting each frame of the low-light image sequence into a second Laplace function to obtain a second scale feature map, where a size of the first scale feature map is larger than a size of the second scale feature map;

[0098] Using the Thor operator, information is extracted from the first-scale feature map to obtain a feature map containing vertical edge information and a feature map containing horizontal edge information. The feature map containing vertical edge information and the feature map containing horizontal edge information are fused to obtain a first fused feature map. The first fused feature map is input into a convolution layer with a convolution kernel size of 3×3 to extract deep features, thereby obtaining a feature map containing deep features of the first-scale feature map.

[0099] Using the Thor operator, information is extracted from the second-scale feature map to obtain a feature map containing vertical edge information and a feature map containing horizontal edge information. The feature map containing vertical edge information and the feature map containing horizontal edge information are fused to obtain a second fused feature map. The second fused feature map is input into a convolution layer with a convolution kernel size of 3×3 to extract deep features, thereby obtaining a feature map containing the deep features of the second-scale feature map.

[0100] The feature map containing the deep features of the second-scale feature map is upsampled, and the upsampled feature map is fused with the feature map containing the deep features of the first-scale feature map to obtain a feature map with enhanced edge information.

[0101] The lightweight animal behavior recognition module is used to extract spatial features from the mixed feature map corresponding to each frame of low-light image based on the spatial feature extractor contained in the module. The spatial feature extractor is a lightweight deep learning network MobileNetV3. Then, based on the Transformer encoder, temporal features are extracted from the feature maps containing spatial features in multiple frames to obtain a feature map containing temporal features and spatial features, thereby identifying the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior;

[0102] The lightweight animal behavior recognition module focuses on capturing the key features of animal behavior in information-enhanced images and accurately classifying the behavior based on these features. The module also outputs the recognition results in an easy-to-understand format, including the recognized behavior category, confidence score, etc. The lightweight animal behavior recognition module is mainly composed of a convolutional neural network model for extracting spatial features, namely the spatial feature extractor, and a Transformer encoder for extracting temporal features. Figure 6 The lightweight animal behavior recognition module is specifically used for:

[0103] For the mixed feature map corresponding to each frame of low-light image, the mixed feature map is input into the spatial feature extractor, the spatial features in the mixed feature map are extracted, and a feature map containing spatial features is obtained. The feature map containing spatial features is linearly encoded to realize the conversion of the channel dimension of the feature map containing spatial features into the channel dimension of the input of the Transformer encoder. In a specific implementation, a linear layer (fully connected layer) can be used to convert the feature vector in the feature sequence corresponding to the feature map containing spatial features from the output dimension of the spatial extractor to the input dimension of the Transformer encoder; at the same time, the feature map containing spatial features is positionally encoded, and the linearly encoded feature map corresponding to the low-light image of each frame in the low-light image sequence is converted into the channel dimension of the input of the Transformer encoder. The feature maps after position encoding are fused to obtain the fused spatial feature maps corresponding to all low-light images, and the fused spatial feature maps corresponding to all low-light images are input into the Transformer encoder. The temporal features are extracted from the fused spatial feature maps corresponding to all low-light images to obtain a feature map containing temporal features and spatial features. The feature map containing temporal features and spatial features is then processed by attention pooling to obtain global features. The classifier is then used to calculate the animal behavior to which the global feature belongs, that is, the animal behavior corresponding to the low-light image sequence, and the probability that the global feature belongs to the animal behavior, that is, the probability that the animal in the low-light image sequence belongs to the animal behavior, which can also be a confidence score. Figure 6 The classification results include the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior.

[0104] Because the Transformer encoder itself lacks temporal information, positional encoding is required to explicitly incorporate frame order information. This method uses sine and cosine functions to generate positional encodings, which can capture different frequency information in different dimensions. The goal of this design is to allow encodings of different dimensions to vary at different frequencies, thereby providing the model with rich positional information. This combination of sine and cosine gives the positional encodings different periodic variations in different dimensions, helping the model better capture relative position information in the sequence.

[0105] Among them, the feature map containing spatial features is positionally encoded, including:

[0106] Based on the dimension of the transformer encoder input data, determine the range of the numerical value of the position encoding dimension index corresponding to each feature point in the feature map containing the spatial feature; for example, when the dimension of the transformer encoder input data is 1024, this index range is 0 to 1023;

[0107] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an even number, the position encoding is calculated using the following formula:

[0108]

[0109] Among them, PE pose,2index Indicates the position coding when the value of the position coding dimension index corresponding to the feature point in the feature map containing spatial features is an even number; pose represents the position index of the low-light image in the low-light image sequence. For a 30-frame video, this index is an integer from 0 to 29; index is the position coding dimension index, that is, the feature index of the transformer encoder input data. Index indicates the feature position of the sin and cos functions when generating the position coding; d model is the dimension of the input data to the transformer encoder, and A is the scaling factor used to adjust the frequency of different dimensions in the positional encoding. This constant is chosen to ensure that the encodings of different dimensions have different periodicities, thereby capturing multi-scale position information in the input sequence;

[0110] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number, the position encoding is calculated using the following formula:

[0111]

[0112] Among them, PE pose,2index+1 Indicates the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number.

[0113] In positional encoding, a 1024-dimensional encoding vector is generated for each frame. Each dimension is generated using the positional encoding formula described above. Even and odd indices use the sin and cos formulas. This positional encoding is then added to the input features to provide position information.

[0114] Taking pose as 5, A as 10000, and the dimension of the transformer encoder input data as 1024 as an example, based on the above formula, its position encoding is:

[0115] For feature maps containing spatial features, the position encoding dimension index value is an even number (index = 0, 2, 4...), and the position encoding can be calculated as:

[0116]

[0117] For feature maps containing spatial features, the position encoding dimension index corresponding to the feature map is an odd number (index = 1, 3, 5...), and the position encoding can be calculated as:

[0118]

[0119] It should be noted that linear encoding is to convert the channel dimension of the feature map containing spatial features into the channel dimension of the input of the Transformer encoder, and position encoding is to determine the position encoding of each feature point in the feature map containing spatial features. The linearly encoded feature map and the positionally encoded feature map are fused, which can be understood as fusing the eigenvalues ​​of the feature points with the position encoding of the feature points.

[0120] Among them, the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, which is specifically expressed by the following formula:

[0121]

[0122] Among them, global_features represents global features, feature z Represents the feature map containing temporal features and spatial features corresponding to the z-th frame low-light image, α z represents the weight calculated by the attention mechanism corresponding to the z-th low-light image frame, and T is the total number of low-light images in the low-light image sequence.

[0123] The data storage module is used to store the video in the low-light scene collected in real time by the data acquisition module, and store the behavior of the animal in the video obtained by identification, and the probability that the animal in the video belongs to the behavior;

[0124] In practice, the data storage module is the part of the system responsible for storing and managing all data. It ensures that all collected data, processed data, and recognition results are securely stored and efficiently retrieved. The data storage module considers data integrity, security, and accessibility.

[0125] The data storage module categorizes and stores collected raw data. Video streams, images, and recognition results are systematically saved for easy access and analysis. To ensure data security, the module uses encryption technology to protect sensitive information and prevent unauthorized access.

[0126] Data retrieval and query functions are also crucial in the data storage module. Users can quickly find the historical data they need through the interface for analysis and comparison. The module supports multiple query methods, such as filtering by time and behavior type, ensuring users can efficiently access the information they need.

[0127] Furthermore, the data storage module is responsible for data backup and recovery. Regular, automated backups prevent data loss and ensure rapid data recovery in the event of system failures or unexpected events. The data storage module supports integration with other systems. Through an open API, users can export data to other analysis tools or share data with other systems.

[0128] The user interface module is used to display the video in the low-light scene collected in real time by the data collection module, and store the identified behavior of the animal in the video and the probability that the animal in the video belongs to the behavior.

[0129] In practice, the user interface module serves as the interactive window between the system and the user. Designed to be simple and easy to use, it allows users to easily view and operate the system. Within this module, users can monitor camera footage in real time and see the animal behaviors identified by the system. The interface displays the animal's behavior and provides a confidence score, giving users a more intuitive understanding of the recognition results. In addition to real-time monitoring, the user interface also provides data analysis capabilities. Users can view historical data to understand changing trends in animal behavior. Various graphs, such as time series graphs and pie charts, help users better understand the data. These visualization tools allow users to customize views, generate customized reports, and further analyze recognition results. The user interface also supports interactive and control features. Users can use the interface to switch cameras, adjust recognition parameters, and more, customizing system settings to meet specific needs. This flexibility allows users to configure the system to their needs, ensuring optimal performance. Whether on a computer or mobile phone, the interface adapts well to different screen sizes, ensuring a consistent user experience. The user interface module allows users to conveniently view and analyze recognition results and adjust system settings as needed.

[0130] The system management module is used to coordinate and manage the operation of various modules in the animal behavior recognition software system for low-light scenes.

[0131] The System Management module is the core of the entire system, responsible for coordinating and managing the operation of various modules to ensure system stability and efficiency. It provides a range of tools and functions to help administrators monitor system status, configure system parameters, and troubleshoot problems.

[0132] First, the system management module provides comprehensive monitoring capabilities. Administrators can use this module to view the system's operating status in real time, including performance indicators and resource usage for each module. This information helps administrators identify potential issues and ensure smooth system operation.

[0133] In terms of configuration management, the system management module allows administrators to set and adjust various system parameters. Whether it's the frequency of data collection, recognition model parameters, or storage strategy selection, administrators can use this module to flexibly configure to suit different application needs.

[0134] The system management module also features log management. When a system anomaly occurs, administrators can quickly locate the root cause by viewing log files and take appropriate measures to remedy the problem. Detailed log records, including error messages and operation records, provide crucial insights for problem resolution.

[0135] Finally, the system management module supports user rights management. Administrators can assign different permissions to different users to ensure system security and data confidentiality. Through fine-grained permission control, the system can prevent unauthorized access and operation.

[0136] The present invention relates to an animal behavior recognition software system for low-light scenarios. Its core is to achieve integrated optimization of image enhancement and behavior recognition through deep learning technology. Specifically, the present invention is achieved through the following key technical points:

[0137] (1) Deep Learning-Driven Image Enhancement Technology: This paper uses a dedicated deep learning network to automatically optimize the features of low-light images. The system integrates illumination estimation, edge information enhancement, and noise suppression modules, significantly improving image quality and providing high-quality input for subsequent behavior recognition.

[0138] (2) Fusion of multi-channel feature information: The attention mechanism is used to comprehensively process multi-channel feature information to ensure recognition accuracy under low-light conditions. This mechanism can effectively focus on key features in the image and improve recognition accuracy and reliability.

[0139] (3) Lightweight feature extraction and recognition: This paper combines MobileNetV3 with the Transformer encoder to achieve lightweight and efficient feature extraction. This design does not require high-performance computing equipment, reducing equipment costs and improving the applicability and flexibility of the system.

[0140] (4) Innovative spatial and temporal feature extraction technology: Through techniques such as position encoding and attention pooling, the model’s ability to capture temporal information is enhanced. This design enables the system to accurately identify animal behavior in complex backgrounds and dynamically changing scenes.

[0141] (5) Ability to adapt to various experimental environments: The design of the present invention is not only suitable for a variety of low-light experimental environments, but also can provide reliable solutions in complex backgrounds and dynamically changing scenes, significantly improving the accuracy and stability of animal behavior recognition.

[0142] By integrating an image information enhancement module and a lightweight animal behavior recognition module, this invention successfully realizes animal behavior recognition in low-light scenes, with significant technical effects and application value. The specific effects are as follows:

[0143] (1) Significant improvement in image quality: Through an integrated image information enhancement network, the present invention can effectively improve the quality of low-light images. The synergistic effect of illumination estimation, edge information enhancement, and noise suppression modules ensures that the input image remains clear and high-contrast even in low-light conditions, providing a high-quality image foundation for subsequent behavior recognition.

[0144] (2) Accuracy and reliability of behavior recognition: The lightweight animal behavior recognition network of the present invention can accurately capture the behavioral characteristics of animals. Through the optimized design of the deep learning model, the system's behavior recognition accuracy in low-light environments is significantly improved, reducing recognition errors caused by insufficient light and enhancing the reliability of the recognition results.

[0145] (3) Reduced equipment costs and improved system applicability: Due to the use of a lightweight deep learning network structure, the present invention can achieve efficient processing without relying on high-performance computing equipment. This not only reduces equipment costs but also improves the applicability and flexibility of the system, enabling its application in more experimental environments.

[0146] (4) Technical support for animal behavior research: The overall approach significantly improves the accuracy of animal behavior observation in dim light environments, providing strong technical support for animal behavior research. The system can provide reliable recognition results in complex backgrounds and dynamically changing scenes, promoting research progress in related fields.

[0147] In summary, the present invention demonstrates excellent performance and broad application potential in animal behavior recognition in low-light scenes, and provides an innovative technical solution for research and application in related fields.

[0148] The above description is merely a preferred embodiment of the present disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. An animal behavior recognition software system for low-light scenes, characterized in that: It includes data acquisition module, image information enhancement module, lightweight animal behavior recognition module, data storage module, user interface module, and system management module; The data acquisition module is used to collect multiple frames of low-light images in a low-light scene in real time, wherein the multiple frames of low-light images in the low-light scene include animals and background environment, and the animals and the background environment cannot be distinguished, convert the images according to a preset format to obtain multiple frames of low-light images after format conversion, arrange the multiple frames of low-light images after format conversion in chronological order to obtain a low-light image sequence, and input the low-light image sequence into the image information enhancement module; The image information enhancement module is used to enhance the illumination and edge information of each low-illumination image frame in the low-illumination image sequence. Specifically, for each low-illumination image frame in the low-illumination image sequence, the illumination estimation module in the image information enhancement module is input to obtain an illumination-enhanced feature map, and the illumination-enhanced feature map is input to the noise suppression module in the image information enhancement module to obtain a denoised feature map. At the same time, each low-illumination image frame in the low-illumination image sequence is input to the edge information enhancement module in the image information enhancement module to obtain an edge information-enhanced feature map, and the denoised feature map and the edge information-enhanced feature map are spliced ​​and superimposed in the dimension of the image channel to obtain a multi-channel feature map. The multi-channel feature map is processed by a channel attention mechanism to obtain a mixed feature map, and then a mixed feature map corresponding to each low-illumination image frame is obtained. The mixed feature map corresponding to each low-illumination image frame is input to a lightweight animal behavior recognition module; The lightweight animal behavior recognition module is used to extract spatial features from the mixed feature map corresponding to each frame of low-light image based on the spatial feature extractor contained in the module. The spatial feature extractor is a lightweight deep learning network MobileNetV3. Then, based on the Transformer encoder, temporal features are extracted from the feature maps containing spatial features in multiple frames to obtain a feature map containing temporal features and spatial features, thereby identifying the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior; The data storage module is used to store the video in the low-light scene collected in real time by the data acquisition module, and store the behavior of the animal in the video obtained by identification, and the probability that the animal in the video belongs to the behavior; The user interface module is used to display the video in the low-light scene collected in real time by the data acquisition module, and store the behavior of the animal in the video obtained by identification, and the probability of the animal in the video belonging to the behavior; The system management module is used to coordinate and manage the operation of various modules in the animal behavior recognition software system for low-light scenes.

2. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The illumination estimation module in the image information enhancement module is specifically used to: For each frame of the low-light image sequence, input it into a convolution layer with a convolution kernel size of 3*3 and a convolution operation stride of 1, and input the feature map output by the convolution layer into the ReLU function to achieve numerical conversion of the feature values ​​in the multi-channel feature map; The feature map after numerical conversion is input into the residual structure. Specifically, the feature map after numerical conversion is input into a convolution layer with a convolution kernel size of 3*3, a convolution operation stride of 1, and a convolution operation padding of 1, and then the feature map output by the convolution layer is batch normalized. The feature map after batch normalization is passed through a Relu function to obtain a first feature map, and the first feature map and the feature map after numerical conversion are superimposed to obtain a multi-scale feature map; the multi-scale feature map is input into a convolution layer with a convolution kernel size of 3*3 and a convolution operation stride of 1, and then the feature map processed by the convolution layer is input into a Sigmod function to obtain an illumination enhanced feature map.

3. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The noise suppression module in the image information enhancement module is specifically used to: Filling the illumination enhancement feature map corresponding to each frame image in the low illumination image sequence, wherein the illumination enhancement feature map includes multiple feature points, Specifically, the filling size is determined by the following formula: Among them, pad_size is the padding size, R is the preset search radius in the noise reduction process, and P is the size of the feature block where the preset feature point is located; The illumination enhanced feature map is padded based on the padding size. Specifically, the height of the illumination enhanced feature map is increased by 2 times the pad_size, the width of the illumination enhanced feature map is increased by 2 times the pad_size, and the feature values ​​of the added feature points are padded to obtain the padded feature map; For each feature point in the filled feature map, determine the neighborhood block corresponding to the feature point. The neighborhood block is expressed by the following formula: Among them, N i,j Represents the neighborhood block of the feature point, that is, the set of neighborhood points corresponding to the feature point, (i, j) represents the coordinates of the feature point in the filled feature map, x i+m,j+n Represents the eigenvalue of the neighborhood point corresponding to the feature point, (i+m,j+n) represents the coordinates of the neighborhood point in the filled feature map, m represents the displacement of the horizontal coordinate, and n represents the displacement of the vertical coordinate; With the feature point as the center and R as the radius, the search range is determined. For each feature point other than the feature point in the search range, the neighborhood block N of the other feature points is determined. k,l , (k, l) is the coordinate of other feature points in the filled feature map, and then calculate the neighborhood block N i,j and neighborhood block N k,l The similarity is achieved through the following formula: Among them, d(N i,j ,N k,l ) represents the neighborhood block N i,j and neighborhood block N k,l The similarity of x k+m,l+n Indicates the eigenvalues ​​of the neighborhood points corresponding to other feature points; Based on the neighborhood block N i,j and neighborhood block N k,l The similarity is calculated and the similarity weight is expressed by the following formula: Among them, ω i,j,k,l represents the similarity weight, h represents the parameter that controls the degree of noise filtering; Based on the above calculation, the similarity weight of each feature point is obtained, and then the cumulative weighted feature value is calculated for each feature point. The specific calculation is performed using the following formula: Among them, x k,l Represents other feature points x k,l The eigenvalues ​​of i,j represents the cumulative weighted eigenvalue; The final denoising result is obtained by dividing the accumulated weighted eigenvalues ​​by the accumulated weight sum, which is specifically expressed by the following formula: Among them, denoised_result i,j Represents the final denoising result of each feature point, ε is a constant; Thus, the final denoising results of all feature points are obtained, that is, the denoised feature map.

4. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The edge information enhancement module in the image information enhancement module is specifically used to: Inputting each frame of the low-light image sequence into a first Laplace function to obtain a first scale feature map, and inputting each frame of the low-light image sequence into a second Laplace function to obtain a second scale feature map, where a size of the first scale feature map is larger than a size of the second scale feature map; Using the Thor operator, information is extracted from the first-scale feature map to obtain a feature map containing vertical edge information and a feature map containing horizontal edge information. The feature map containing vertical edge information and the feature map containing horizontal edge information are fused to obtain a first fused feature map. The first fused feature map is input into a convolution layer with a convolution kernel size of 3×3 to extract deep features, thereby obtaining a feature map containing deep features of the first-scale feature map. Using the Thor operator, information is extracted from the second-scale feature map to obtain a feature map containing vertical edge information and a feature map containing horizontal edge information. The feature map containing vertical edge information and the feature map containing horizontal edge information are fused to obtain a second fused feature map. The second fused feature map is input into a convolution layer with a convolution kernel size of 3×3 to extract deep features, thereby obtaining a feature map containing the deep features of the second-scale feature map. The feature map containing the deep features of the second-scale feature map is upsampled, and the upsampled feature map is fused with the feature map containing the deep features of the first-scale feature map to obtain a feature map with enhanced edge information.

5. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The lightweight animal behavior recognition module is specifically used for: For the mixed feature map corresponding to each frame of the low-light image, the mixed feature map is input into the spatial feature extractor, the spatial features in the mixed feature map are extracted, and a feature map containing spatial features is obtained. The feature map containing spatial features is linearly encoded to convert the channel dimension of the feature map containing spatial features into the channel dimension of the input of the Transformer encoder. At the same time, the feature map containing spatial features is position-encoded, and the linearly encoded feature map and the position-encoded feature map corresponding to the low-light image of each frame in the low-light image sequence are fused to obtain the fused spatial features corresponding to all low-light images. Figure, the fused spatial feature maps corresponding to all low-light images are input into the Transformer encoder, the temporal features are extracted from the fused spatial feature maps corresponding to all low-light images, and a feature map containing temporal features and spatial features is obtained, and then the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, and then the classifier is used to calculate according to the global features to obtain the animal behavior to which the global feature belongs, that is, the animal behavior corresponding to the low-light image sequence, and the probability that the global feature belongs to the animal behavior, that is, the probability that the animal in the low-light image sequence belongs to the animal behavior.

6. The animal behavior recognition software system for low-light scenes according to claim 5, characterized in that: The feature map containing spatial features is positionally encoded, including: Based on the dimension of the transformer encoder input data, determine the range of the value of the position encoding dimension index corresponding to each feature point in the feature map containing the spatial feature; When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an even number, the position encoding is calculated using the following formula: Among them, PE pose,2index It represents the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing spatial features is an even number. Pose represents the position index of the low-light image in the low-light image sequence. Index is the position encoding dimension index, that is, the feature index of the transformer encoder input data. model is the dimension of the input data of the transformer encoder, and A is the scaling factor; When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number, the position encoding is calculated using the following formula: Among them, PE pose,2index+1 Indicates the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing spatial features is an odd number.

7. The animal behavior recognition software system for low-light scenes according to claim 5, characterized in that: The feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, which can be expressed by the following formula: Among them, global_features represents global features, feature z Represents the feature map containing temporal features and spatial features corresponding to the z-th frame low-light image, α z represents the weight calculated by the attention mechanism corresponding to the z-th low-light image frame, and T is the total number of low-light images in the low-light image sequence.

Images