Animal behavior recognition software system for low-illumination scene

Abstract

The invention provides an animal behavior recognition software system for a low-illumination scene, which relates to the technical field of image processing and comprises a data acquisition module, an image information enhancement module, a lightweight animal behavior recognition module, a data storage module, a user interface module and a system management module. According to the invention, multiple frames of images in a low-illumination scene are acquired through the data acquisition module, the illumination of the low-illumination image is improved through the illumination estimation module in the image enhancement module, the noise of the low-illumination image is removed through the noise suppression module, and the edge information of the low-illumination image is enhanced through the edge information enhancement module. According to the method, the illuminance and edge information of the low-illuminance image are enhanced, the accuracy of animal recognition can be improved when the image with the feature enhanced is recognized through the lightweight animal behavior recognition module, and meanwhile the method has the advantage of being lightweight and can achieve efficient processing without depending on high-performance computing equipment.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, in particular to an animal behavior recognition software system for low-illumination scenes. Background Art

[0002] In the medical field, drug research can be achieved through animal behavior observation experiments. Specifically, drugs are injected into laboratory mice, and their behavior is observed to determine the effectiveness of the drugs.

[0003] Since strong light irradiation affects laboratory mice, when conducting animal behavior observation experiments, it is impossible to determine whether the behavior of the laboratory mice is caused by strong light irradiation or drugs. Therefore, in order to avoid the impact of strong light irradiation, some experiments need to be conducted in low-light scenes. In the images captured in low-light scenes, the color distinction between the laboratory mice and the environment is low. In order to realize the recognition of the behavior of laboratory mice, it is currently possible to capture images through ordinary cameras and increase the image brightness, or to capture images and recognize them through infrared cameras, or to achieve it through a combination of ordinary cameras and infrared cameras.

[0004] However, when the image brightness is increased, the brightness of the laboratory mouse and the environment in the image is also increased, resulting in inaccurate behavior recognition of the laboratory mouse. When the image is captured by an infrared camera, the edge contour of the laboratory mouse is not clear, for example, the claws and tail of the laboratory mouse are not clear, resulting in low recognition accuracy, and the cost of using infrared cameras is high. When it is implemented by combining an ordinary camera and an infrared camera, the time of the captured image may be different due to the different frame rates of the two cameras, and there is a deviation in the viewing angles of the two cameras, resulting in errors when the image features of the two cameras are fused, resulting in low recognition accuracy, and the method is highly complex and cannot be monitored in real time. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to propose an animal behavior recognition software system for low-light scenes, including: a data acquisition module, an image information enhancement module, a lightweight animal behavior recognition module, a data storage module, a user interface module, and a system management module;

[0006] The data acquisition module is used to collect multiple frames of low-light images in a low-light scene in real time, wherein the multiple frames of low-light images in the low-light scene include animals and background environment, and the animals and the background environment cannot be distinguished, and are converted according to a preset format to obtain multiple frames of low-light images after format conversion, and the multiple frames of low-light images after format conversion are arranged in chronological order to obtain a low-light image sequence, and the low-light image sequence is input into the image information enhancement module;

[0007] The image information enhancement module is used to enhance the illumination and edge information of each frame of low illumination image in the low illumination image sequence. Specifically, for each frame of low illumination image in the low illumination image sequence, the illumination estimation module in the image information enhancement module is input to obtain an illumination enhanced feature map, and the illumination enhanced feature map is input to the noise suppression module in the image information enhancement module to obtain a denoised feature map. At the same time, each frame of low illumination image in the low illumination image sequence is input to the edge information enhancement module in the image information enhancement module to obtain an edge information enhanced feature map, and the denoised feature map and the edge information enhanced feature map are spliced ​​and superimposed in the dimension of the image channel to obtain a multi-channel feature map, and the multi-channel feature map is processed through a channel attention mechanism to obtain a mixed feature map, and then a mixed feature map corresponding to each frame of low illumination image is obtained, and the mixed feature map corresponding to each frame of low illumination image is input to a lightweight animal behavior recognition module;

[0008] The lightweight animal behavior recognition module is used to extract spatial features in the mixed feature map corresponding to each frame of low-light image based on the spatial feature extractor contained in the module, wherein the spatial feature extractor is a lightweight deep learning network MobileNetV3, and then according to the Transformer encoder, in the feature map containing spatial features in multiple frames, the temporal features are extracted to obtain a feature map containing temporal features and spatial features, thereby identifying the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior;

[0009] The data storage module is used to store the video in the low-light scene collected in real time by the data collection module, and store the behavior of the animal in the video obtained by identification, and the probability that the animal in the video belongs to the behavior;

[0010] The user interface module is used to display the video in the low-light scene collected in real time by the data collection module, and store the behavior of the animal in the video obtained by identification, and the probability of the animal in the video belonging to the behavior;

[0011] The system management module is used to coordinate and manage the operation of various modules in the animal behavior recognition software system for low-light scenes.

[0012] Optionally, the illumination estimation module in the image information enhancement module is specifically used for:

[0013] For each frame image in the low-illuminance image sequence, input it into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and input the feature map output by the convolution layer into the Relu function to realize the numerical conversion of the eigenvalues ​​in the feature map of the multi-channel; input the feature map after the numerical conversion into the residual structure, specifically, input the feature map after the numerical conversion into a convolution layer with a convolution kernel size of 3*3, a stride of the convolution operation of 1, and a padding of the convolution operation of 1, and then perform batch normalization on the feature map output by the convolution layer, pass the batch normalization feature map through the Relu function to obtain a first feature map, superimpose the first feature map and the feature map after the numerical conversion to obtain a multi-scale feature map; input the multi-scale feature map into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and then input the feature map processed by the convolution layer into the Sigmod function to obtain an illumination enhanced feature map.

[0014] Optionally, the noise suppression module in the image information enhancement module is specifically used to:

[0015] Filling is performed for the illumination enhancement feature map corresponding to each frame image in the low illumination image sequence, wherein the illumination enhancement feature map includes a plurality of feature points.

[0016] Specifically, the filling size is determined by the following formula:

[0017]

[0018] Among them, pad_size is the padding size, R is the preset search radius in the denoising process, and P is the size of the feature block where the preset feature point is located;

[0019] The illumination enhanced feature map is filled based on the filling size. Specifically, the height of the illumination enhanced feature map is increased by 2 times the pad_size, the width of the illumination enhanced feature map is increased by 2 times the pad_size, and the feature values ​​of the increased feature points are filled to obtain a filled feature map;

[0020] For each feature point in the filled feature map, determine the neighborhood block corresponding to the feature point. The neighborhood block is expressed by the following formula:

[0021]

[0022] Among them, N i,j represents the neighborhood block of the feature point, that is, the set of neighborhood points corresponding to the feature point, (i, j) represents the coordinates of the feature point in the filled feature map, x i+m,j+nRepresents the eigenvalue of the neighborhood point corresponding to the feature point, (i+m,j+n) represents the coordinates of the neighborhood point in the filled feature map, m represents the displacement of the horizontal coordinate, and n represents the displacement of the vertical coordinate;

[0023] Take the feature point as the center and R as the radius to determine the search range. For each feature point in the search range except the feature point, determine the neighborhood block N of other feature points. k,l , (k, l) is the coordinate of other feature points in the filled feature map, and then calculate the neighborhood block N i,j and neighborhood block N k,l The similarity is realized by the following formula:

[0024]

[0025] Among them, d(N i,j ,N k,l ) represents the neighborhood block N i,j and neighborhood block N k,l The similarity of x k+m,l+n Indicates the eigenvalues ​​of the neighborhood points corresponding to other feature points;

[0026] Based on the neighborhood block N i,j and neighborhood block N k,l The similarity is calculated by the following formula:

[0027]

[0028] Among them, ω i,j,k,l represents the similarity weight, and h represents the parameter that controls the degree of noise filtering;

[0029] Based on the above calculation, the similarity weight of each feature point is obtained, and then for each feature point, the cumulative weighted feature value is calculated, which is specifically calculated by the following formula:

[0030]

[0031] Among them, x k,l Represents other feature points x k,l The eigenvalues ​​of i,j represents the cumulative weighted eigenvalue;

[0032] The final denoising result is obtained by dividing the accumulated weighted eigenvalues ​​by the accumulated weight sum, which is specifically expressed by the following formula:

[0033]

[0034] Among them, denoised_result i,jRepresents the final denoising result of each feature point, ε is a constant;

[0035] Thus, the final denoising result of all feature points is obtained, that is, the denoised feature map.

[0036] Optionally, the edge information enhancement module in the image information enhancement module is specifically used to:

[0037] Inputting each frame of the low-illumination image sequence into a first Laplace function to obtain a first scale feature map, and inputting each frame of the low-illumination image sequence into a second Laplace function to obtain a second scale feature map, wherein the size of the first scale feature map is larger than the size of the second scale feature map;

[0038] Using the Thor operator, extract information from the first scale feature map to obtain a feature map containing edge information in the vertical direction and a feature map containing edge information in the horizontal direction, fuse the feature map containing edge information in the vertical direction and the feature map containing edge information in the horizontal direction to obtain a first fused feature map, input the first fused feature map into a convolution layer with a convolution kernel size of 3×3 to extract deep features, and obtain a feature map containing deep features of the first scale feature map;

[0039] Using the Thor operator, extract information from the second-scale feature map to obtain a feature map containing edge information in the vertical direction and a feature map containing edge information in the horizontal direction, fuse the feature map containing edge information in the vertical direction and the feature map containing edge information in the horizontal direction to obtain a second fused feature map, input the second fused feature map into a convolution layer with a convolution kernel size of 3×3 to extract deep features, and obtain a feature map containing deep features of the second-scale feature map;

[0040] The feature map containing the deep features of the second-scale feature map is upsampled, and the upsampled feature map is fused with the feature map containing the deep features of the first-scale feature map to obtain a feature map with enhanced edge information.

[0041] Optionally, the lightweight animal behavior recognition module is specifically used for:

[0042] For the mixed feature map corresponding to each frame of the low-light image, the mixed feature map is input into the spatial feature extractor, the spatial features in the mixed feature map are extracted, and a feature map containing the spatial features is obtained. The feature map containing the spatial features is linearly encoded to realize the conversion of the channel dimension of the feature map containing the spatial features into the channel dimension of the input of the Transformer encoder. At the same time, the feature map containing the spatial features is position-encoded, and the linearly encoded feature map and the position-encoded feature map corresponding to each frame of the low-light image in the low-light image sequence are fused to obtain the fused spatial features corresponding to all the low-light images. Figure, the fused spatial feature maps corresponding to all low-light images are input into the Transformer encoder, the temporal features are extracted from the fused spatial feature maps corresponding to all low-light images, and a feature map containing temporal features and spatial features is obtained, and then the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, and then a classifier is used to calculate according to the global features to obtain the animal behavior to which the global feature belongs, that is, the animal behavior corresponding to the low-light image sequence, and the probability that the global feature belongs to the animal behavior, that is, the probability that the animal in the low-light image sequence belongs to the animal behavior.

[0043] Optionally, the feature map containing the spatial features is positionally encoded, including:

[0044] Based on the dimension of the transformer encoder input data, determine the range of the value of the position encoding dimension index corresponding to each feature point in the feature map containing the spatial feature;

[0045] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an even number, the position encoding is calculated using the following formula:

[0046]

[0047] Among them, PE pose,2index It represents the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an even number. Pose represents the position index of the low-light image in the low-light image sequence. Index is the position encoding dimension index, that is, the feature index of the transformer encoder input data. model is the dimension of the input data of the transformer encoder, and A is the scaling factor;

[0048] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number, the position encoding is calculated using the following formula:

[0049]

[0050] Among them, PE pose,2index+1 Indicates the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number.

[0051] Optionally, the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, which are specifically expressed by the following formula:

[0052]

[0053] Among them, global_features represents global features, feature z Represents the feature map containing temporal features and spatial features corresponding to the z-th frame low-light image, α z represents the weight calculated by the attention mechanism corresponding to the z-th low-light image, and T is the total number of low-light images in the low-light image sequence.

[0054] The beneficial effects of adopting the above technical solution are:

[0055] The present invention provides an animal behavior recognition software system for low-illumination scenes. In an image enhancement module, the illumination of a low-illumination image is improved by an illumination estimation module, and then the noise of the low-illumination image is removed by a noise suppression module. At the same time, the edge information of the low-illumination image is enhanced by an edge information enhancement module, so that both the illumination and edge information of the low-illumination image are enhanced. Then, when the feature-enhanced image is recognized by a lightweight animal behavior recognition module, the accuracy of animal recognition can be improved. At the same time, when performing animal behavior recognition, the present invention adopts a lightweight deep learning network MobileNetV3 as a spatial feature extractor, and combines it with a Transformer encoder, so that the present invention can monitor the behavior of animals in real time, and the present invention does not need to rely on high-performance computing equipment to achieve efficient processing, which not only reduces the equipment cost, but also improves the applicability and flexibility of the software, so that it can be applied in more experimental environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of an animal behavior recognition software system for low-light scenes in an embodiment of the present invention;

[0057] Figure 2 This is a system architecture diagram of an animal behavior recognition software for low-light scenes in an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of the structure of an image enhancement module in an embodiment of the present invention;

[0059] Figure 4is a schematic diagram of the structure of an illumination estimation module in an embodiment of the present invention;

[0060] Figure 5 Schematic diagram of the structure of an edge information enhancement module in an embodiment of the present invention;

[0061] Figure 6 Schematic diagram of the structure of a lightweight animal behavior recognition module in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0063] Aiming at the problems existing in the existing technology, Figure 1 The present invention provides an animal behavior recognition software system for low-light scenes, including a data acquisition module, an image information enhancement module, a lightweight animal behavior recognition module, a data storage module, a user interface module, and a system management module. The present invention uses a camera to capture animal behavior videos in real time. These video data are quickly transmitted to the image information enhancement module via data, where advanced image enhancement technology is applied to improve the clarity, visibility, and edge information richness of the video under low-light conditions, ensuring the accuracy of subsequent processing. Next, the enhanced image data is Figure 1 The enhanced information in the video is input into the lightweight animal behavior recognition module. This module uses an optimized deep learning algorithm that can operate efficiently in resource-limited environments and accurately identify the specific behaviors of animals. These recognition results not only include the type of behavior, but also come with precise timestamp information for subsequent analysis. The recognized behavior data is stored in the data storage module together with the processed video data. This module is responsible for securely storing all data, providing a solid foundation for subsequent analysis, report generation, and further model training. At the same time, the user interface module extracts the recognition results from the data storage module and displays detailed behavior information and related video clips to the user, allowing the user to intuitively understand the system's recognition effect and the animal's behavior pattern. Through this entire process, the system achieves efficient management from data acquisition to result display, providing users with powerful tools to study and analyze animal behavior.

[0064] Combination Figure 2 , data acquisition module, image information enhancement module, lightweight animal behavior recognition module, data storage module, user interface module, system management module. These six modules are specifically used for:

[0065] The data acquisition module is used to collect multiple frames of low-light images in a low-light scene in real time, wherein the multiple frames of low-light images in the low-light scene include animals and background environment, and the animals and the background environment cannot be distinguished, and are converted according to a preset format to obtain multiple frames of low-light images after format conversion, and the multiple frames of low-light images after format conversion are arranged in chronological order to obtain a low-light image sequence, and the low-light image sequence is input into the image information enhancement module;

[0066] Low illumination specifically refers to the situation where the light intensity in the environment is low. It is usually used to describe scenes with insufficient light, such as at night, on a cloudy day, or in places with low indoor light. In the fields of photography, videography, and visual perception, low illumination environments may affect image quality and visual effects, so special technologies or equipment are needed to improve the ability to capture light.

[0067] Among them, the data acquisition module of the present invention can collect videos in low-light environments. Each frame of the video is a multiple-frame low-light image in a low-light scene. The present invention can also directly collect multiple frames of low-light images.

[0068] Among them, the data acquisition module is the starting point of the whole system, which is responsible for obtaining raw data from external devices and performing preliminary processing to ensure data quality. The data acquisition module needs to establish a connection with a camera or other sensors to achieve real-time acquisition of video streams or image data. In specific implementation, the video or image containing animals in a low-light environment is collected by the camera, wherein the animal in the present invention refers to a laboratory mouse, and the video or image containing animals in a low-light environment is transmitted to the data acquisition module, wherein the data acquisition module supports a variety of device interface protocols, such as USB, IP camera protocols, etc., to adapt to different hardware devices.

[0069] Among them, when converting according to a preset format, specifically, the present invention pre-sets a system-unified format, and then converts the resolution, frame rate, etc. of multiple frames of low-light images into a system-unified format, so as to facilitate processing and analysis by an image information enhancement module and a lightweight animal behavior recognition module.

[0070] The image information enhancement module is used to enhance the illumination and edge information of each frame of the low illumination image in the low illumination image sequence. Specifically, Figure 3, for each frame of low-illuminance image in the low-illuminance image sequence, input the illumination estimation module in the image information enhancement module to obtain an illumination enhanced feature map, input the illumination enhanced feature map into the noise suppression module in the image information enhancement module to obtain a denoised feature map, and at the same time, input each frame of low-illuminance image in the low-illuminance image sequence into the edge information enhancement module in the image information enhancement module to obtain an edge information enhanced feature map, and perform splicing and superposition processing on the denoised feature map and the edge information enhanced feature map in the dimension of the image channel to obtain a multi-channel feature map, and process the multi-channel feature map through a channel attention mechanism to obtain a mixed feature map, and then obtain a mixed feature map corresponding to each frame of low-illuminance image, and input the mixed feature map corresponding to each frame of low-illuminance image into a lightweight animal behavior recognition module;

[0071] Among them, combined Figure 4 The illumination estimation module in the image information enhancement module is specifically used for:

[0072] For each frame image in the low-illuminance image sequence, input it into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and input the feature map output by the convolution layer into the Relu function to realize the numerical conversion of the eigenvalues ​​in the feature map of the multi-channel; input the feature map after the numerical conversion into the residual structure, specifically, input the feature map after the numerical conversion into a convolution layer with a convolution kernel size of 3*3, a stride of the convolution operation of 1, and a padding of the convolution operation of 1, and then perform batch normalization on the feature map output by the convolution layer, pass the batch normalization feature map through the Relu function to obtain a first feature map, superimpose the first feature map and the feature map after the numerical conversion to obtain a multi-scale feature map; input the multi-scale feature map into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and then input the feature map processed by the convolution layer into the Sigmod function to obtain an illumination enhanced feature map.

[0073] Wherein, the noise suppression module in the image information enhancement module is specifically used for:

[0074] In order to process edge pixels, the image will be filled using the reverse filling method, and the illumination enhancement feature map corresponding to each frame image in the low-illumination image sequence will be filled. The illumination enhancement feature map contains multiple feature points.

[0075] Specifically, the filling size is determined by the following formula:

[0076]

[0077] Among them, pad_size is the padding size, R is the preset search radius in the denoising process, and P is the size of the feature block where the preset feature point is located;

[0078] The illumination enhanced feature map is filled based on the filling size. Specifically, the height of the illumination enhanced feature map is increased by 2 times the pad_size, the width of the illumination enhanced feature map is increased by 2 times the pad_size, and the feature values ​​of the increased feature points are filled to obtain a filled feature map;

[0079] It should be noted that the size of the illumination enhanced feature map corresponding to each frame image of the present invention can be expressed as [B, C, H, W], where B is the batch size, C is the number of channels, H is the feature map height, and W is the feature map width. The size of the padded feature map is [B, C, H+2×pad_size, W+2×pad_size].

[0080] For each feature point in the filled feature map, determine the neighborhood block corresponding to the feature point. The neighborhood block is expressed by the following formula:

[0081]

[0082] Among them, N i,j represents the neighborhood block of the feature point, that is, the set of neighborhood points corresponding to the feature point, (i, j) represents the coordinates of the feature point in the filled feature map, x i+m,j+n Represents the eigenvalue of the neighborhood point corresponding to the feature point, (i+m,j+n) represents the coordinates of the neighborhood point in the filled feature map, m represents the displacement of the horizontal coordinate, and n represents the displacement of the vertical coordinate;

[0083] Take the feature point as the center and R as the radius to determine the search range. For each feature point in the search range except the feature point, determine the neighborhood block N of other feature points. k,l , (k, l) is the coordinate of other feature points in the filled feature map, and then calculate the neighborhood block N i,j and neighborhood block N k,l The similarity is realized by the following formula:

[0084]

[0085] Among them, d(N i,j ,N k,l ) represents the neighborhood block N i,j and neighborhood block N k,l The similarity of x k+m,l+n Indicates the eigenvalues ​​of the neighborhood points corresponding to other feature points;

[0086] Based on the neighborhood block N i,j and neighborhood block Nk,l The similarity is calculated by the following formula:

[0087]

[0088] Among them, ω i,j,k,l represents the similarity weight, and h represents the parameter that controls the degree of noise filtering;

[0089] Based on the above calculation, the similarity weight of each feature point is obtained, and then for each feature point, the cumulative weighted feature value is calculated, which is specifically calculated by the following formula:

[0090]

[0091] Among them, x k,l Represents other feature points x k,l The eigenvalues ​​of i,j represents the cumulative weighted eigenvalue;

[0092] The final denoising result is obtained by dividing the accumulated weighted eigenvalues ​​by the accumulated weight sum, which is specifically expressed by the following formula:

[0093]

[0094] Among them, denoised_result i,j Represents the final denoising result of each feature point, ε is a constant, where ε is a small constant used to avoid division by 0;

[0095] Thus, the final denoising result of all feature points is obtained, that is, the denoised feature map. After the above steps, the denoised image is output, and its shape is the same as the input image [B, C, H, W]. The denoising module can effectively reduce the noise in the image while retaining the details and structure of the image.

[0096] Among them, combined Figure 5 The edge information enhancement module in the image information enhancement module is specifically used for:

[0097] Inputting each frame of the low-illumination image sequence into a first Laplace function to obtain a first scale feature map, and inputting each frame of the low-illumination image sequence into a second Laplace function to obtain a second scale feature map, wherein the size of the first scale feature map is larger than the size of the second scale feature map;

[0098] Using the Thor operator, extract information from the first scale feature map to obtain a feature map containing edge information in the vertical direction and a feature map containing edge information in the horizontal direction, fuse the feature map containing edge information in the vertical direction and the feature map containing edge information in the horizontal direction to obtain a first fused feature map, input the first fused feature map into a convolution layer with a convolution kernel size of 3×3 to extract deep features, and obtain a feature map containing deep features of the first scale feature map;

[0099] Using the Thor operator, extract information from the second-scale feature map to obtain a feature map containing edge information in the vertical direction and a feature map containing edge information in the horizontal direction, fuse the feature map containing edge information in the vertical direction and the feature map containing edge information in the horizontal direction to obtain a second fused feature map, input the second fused feature map into a convolution layer with a convolution kernel size of 3×3 to extract deep features, and obtain a feature map containing deep features of the second-scale feature map;

[0100] The feature map containing the deep features of the second-scale feature map is upsampled, and the upsampled feature map is fused with the feature map containing the deep features of the first-scale feature map to obtain a feature map with enhanced edge information.

[0101] The lightweight animal behavior recognition module is used to extract spatial features in the mixed feature map corresponding to each frame of low-light image based on the spatial feature extractor contained in the module, wherein the spatial feature extractor is a lightweight deep learning network MobileNetV3, and then according to the Transformer encoder, in the feature map containing spatial features in multiple frames, the temporal features are extracted to obtain a feature map containing temporal features and spatial features, thereby identifying the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior;

[0102] The lightweight animal behavior recognition module focuses on capturing the key features of animal behavior in information-enhanced images and accurately classifies the behavior based on these features. It also outputs the recognition results in an easy-to-understand form, including the recognized behavior category, confidence score, etc. The lightweight animal behavior recognition module is mainly composed of a convolutional neural network model for extracting spatial features, namely a spatial feature extractor, and a Transformer encoder for extracting temporal features. Figure 6 , the lightweight animal behavior recognition module is specifically used for:

[0103] For the mixed feature map corresponding to each frame of the low-light image, the mixed feature map is input into the spatial feature extractor, the spatial features in the mixed feature map are extracted, and a feature map containing spatial features is obtained. The feature map containing spatial features is linearly encoded to realize the conversion of the channel dimension of the feature map containing spatial features into the channel dimension of the input of the Transformer encoder. In a specific implementation, a linear layer (fully connected layer) can be used to convert the feature vector in the feature sequence corresponding to the feature map containing spatial features from the output dimension of the spatial extractor to the input dimension of the Transformer encoder; at the same time, the feature map containing spatial features is positionally encoded, and the linearly encoded feature map corresponding to the low-light image of each frame in the low-light image sequence is converted into the input dimension of the Transformer encoder. The feature maps after position encoding are fused to obtain a fused spatial feature map corresponding to all low-illuminance images, and the fused spatial feature maps corresponding to all low-illuminance images are input into the Transformer encoder, and the temporal features are extracted from the fused spatial feature maps corresponding to all low-illuminance images to obtain a feature map containing temporal features and spatial features, and then the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, and then the classifier is used to calculate according to the global features to obtain the animal behavior to which the global feature belongs, that is, the animal behavior corresponding to the low-illuminance image sequence, and the probability that the global feature belongs to the animal behavior, that is, the probability that the animal in the low-illuminance image sequence belongs to the animal behavior, which can also be a confidence score. Figure 6 The classification results in include the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior.

[0104] Since the Transformer encoder itself does not have timing information, it is necessary to explicitly introduce the order information of the frames through position encoding. The specific method is to use sine and cosine functions to generate position encoding, which can capture different frequency information in different dimensions. The purpose of this design is to allow encodings of different dimensions to change at different frequencies, thereby providing the model with rich position information. This combination of sine and cosine makes the position encoding have different periodic changes in different dimensions, helping the model to better capture the relative position information in the sequence.

[0105] Among them, the feature map containing spatial features is positionally encoded, including:

[0106] Based on the dimension of the transformer encoder input data, determine the range of the value of the position encoding dimension index corresponding to each feature point in the feature map containing the spatial feature; for example, when the dimension of the transformer encoder input data is 1024, the range of this index is 0 to 1023;

[0107] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an even number, the position encoding is calculated using the following formula:

[0108]

[0109] Among them, PE pose,2index Indicates the position code when the value of the position code dimension index corresponding to the feature point in the feature map containing spatial features is an even number; pose indicates the position index of the low-light image in the low-light image sequence. For a 30-frame video, this index is an integer from 0 to 29; index is the position code dimension index, that is, the feature index of the transformer encoder input data. Index indicates the feature position of the sin and cos functions calculated when generating the position code; d model is the dimension of the input data of the transformer encoder, and A is the scaling factor used to adjust the frequency of different dimensions in the position encoding. This constant is chosen to ensure that the encodings of different dimensions have different periodicities, so as to capture the multi-scale position information in the input sequence;

[0110] When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number, the position encoding is calculated using the following formula:

[0111]

[0112] Among them, PE pose,2index+1 Indicates the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number.

[0113] In position encoding, a 1024-dimensional encoding vector is generated for each frame. Each dimension uses the position encoding formula above to generate its value. Even and odd indexes use the sin and cos formulas. This position encoding will be added to the input features to provide position information.

[0114] Taking pose as 5, A as 10000, and the dimension of the transformer encoder input data as 1024 as an example, based on the above formula, its position encoding is:

[0115] For feature maps containing spatial features, the value of the position encoding dimension index is an even number (index = 0, 2, 4...), and the position encoding can be calculated as:

[0116]

[0117] For feature maps containing spatial features, the value of the position encoding dimension index is an odd number (index = 1, 3, 5...), and the position encoding can be calculated as:

[0118]

[0119] It should be noted that linear encoding is to convert the channel dimension of the feature map containing spatial features into the channel dimension of the input of the Transformer encoder, and position encoding is to determine the position encoding of each feature point in the feature map containing spatial features. The feature map after linear encoding and the feature map after position encoding are fused, which can be understood as fusing the feature value of the feature point with the position encoding of the feature point.

[0120] Among them, the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, which are specifically expressed by the following formula:

[0121]

[0122] Among them, global_features represents global features, feature z Represents the feature map containing temporal features and spatial features corresponding to the z-th frame low-light image, α z represents the weight calculated by the attention mechanism corresponding to the z-th low-light image, and T is the total number of low-light images in the low-light image sequence.

[0123] The data storage module is used to store the video in the low-light scene collected in real time by the data collection module, and store the behavior of the animal in the video obtained by identification, and the probability that the animal in the video belongs to the behavior;

[0124] In specific implementation, the data storage module is the part of the system responsible for saving and managing all data. It ensures that all collected data, processed data, and recognition results can be stored securely and retrieved efficiently. The data storage module takes into account the integrity, security, and accessibility of the data.

[0125] The data storage module will classify and store the collected raw data. Whether it is video streams, images or recognition results, they will be saved systematically for subsequent access and analysis. In order to ensure the security of the data, the module will use encryption technology to protect sensitive information and prevent unauthorized access.

[0126] In the data storage module, data retrieval and query functions are also very important. Users can quickly find the required historical data through the interface for analysis and comparison. The module supports multiple query methods, such as filtering by time, behavior type, etc., to ensure that users can efficiently obtain the required information.

[0127] In addition, the data storage module is responsible for data backup and recovery. Regular automatic backup mechanisms can prevent data loss and ensure that data can be quickly restored in the event of system failure or unexpected situations. The data storage module supports integration with other systems. Through the open API interface, users can export data to other analysis tools or share data with other systems.

[0128] The user interface module is used to display the video in the low-light scene collected in real time by the data collection module, and store the recognized behavior of the animal in the video and the probability that the animal in the video has the behavior.

[0129] In specific implementation, the user interface module is an interactive window between the system and the user. It is designed to be simple and easy to use, allowing users to easily view and operate. In this module, users can monitor the images captured by the camera in real time and see the animal behaviors recognized by the system. The interface will show what the animal is doing and give a confidence score, so that users can have a more intuitive understanding of the recognition results. In addition to real-time monitoring, the user interface also provides data analysis functions. Users can view past data to understand the changing trends of animal behavior. There will be various charts on the interface, such as time series charts and pie charts, to help users better understand the data. These visualization tools allow users to customize views, generate reports they need, and further analyze recognition results. The user interface also supports interactive and control functions. Users can use the interface to switch cameras, adjust recognition parameters, and adjust system settings according to different needs. This flexibility allows users to configure according to actual conditions to ensure the best performance of the system. Whether on a computer or a mobile phone, the interface can adapt well to different screen sizes to ensure that users have a consistent experience. The user interface module allows users to easily view and analyze recognition results and adjust system settings as needed.

[0130] The system management module is used to coordinate and manage the operation of various modules in the animal behavior recognition software system for low-light scenes.

[0131] The system management module is the core of the entire system, responsible for coordinating and managing the operation of each module to ensure the stability and efficiency of the system. It provides a series of tools and functions to help administrators monitor system status, configure system parameters, and perform troubleshooting.

[0132] First, the system management module provides comprehensive monitoring functions. Administrators can use this module to view the system's operating status in real time, including the performance indicators of each module, resource usage, etc. This information helps administrators identify potential problems in a timely manner and ensure the smooth operation of the system.

[0133] In terms of configuration management, the system management module allows administrators to set and adjust various parameters of the system. Whether it is the frequency of data collection, the parameters of the recognition model, or the selection of storage strategy, administrators can use this module to flexibly configure to meet different application requirements.

[0134] The system management module also has a log management function. When the system is abnormal, the administrator can quickly locate the root cause of the problem by viewing the log file and take appropriate measures to repair it. The log records are detailed, including error information, operation records, etc., providing an important basis for problem solving.

[0135] Finally, the system management module supports user rights management. Administrators can assign different rights to different users to ensure system security and data confidentiality. Through fine-grained rights control, the system can prevent unauthorized access and operation.

[0136] The present invention relates to an animal behavior recognition software system for low-light scenes, the core of which is to achieve integrated optimization of image enhancement and behavior recognition through deep learning technology. Specifically, the present invention is achieved through the following key technical points:

[0137] (1) Deep learning-driven image enhancement technology: This paper uses a dedicated deep learning network to automatically optimize the features of low-light images. The system integrates illumination estimation, edge information enhancement, and noise suppression modules, significantly improving image quality and providing high-quality input for subsequent behavior recognition.

[0138] (2) Fusion of multi-channel feature information: The multi-channel feature information is comprehensively processed through the attention mechanism to ensure recognition accuracy under low-light conditions. This mechanism can effectively focus on the key features in the image and improve the accuracy and reliability of recognition.

[0139] (3) Lightweight feature extraction and recognition: This paper adopts the combination of MobileNetV3 and Transformer encoder to achieve lightweight and efficient feature extraction. This design does not rely on high-performance computing equipment, reduces equipment costs, and improves the applicability and flexibility of the system.

[0140] (4) Innovative spatial and temporal feature extraction technology: Through techniques such as position encoding and attention pooling, the model’s ability to capture temporal information is enhanced. This design enables the system to accurately identify animal behavior in complex backgrounds and dynamically changing scenes.

[0141] (5) Ability to adapt to a variety of experimental environments: The design of the present invention is not only suitable for a variety of low-light experimental environments, but can also provide reliable solutions in complex backgrounds and dynamically changing scenes, significantly improving the accuracy and stability of animal behavior recognition.

[0142] The present invention successfully realizes animal behavior recognition in low-light scenes by integrating an image information enhancement module and a lightweight animal behavior recognition module, and has significant technical effects and application value. The specific effects are as follows:

[0143] (1) Significant improvement in image quality: Through the integrated image information enhancement network, the present invention can effectively improve the quality of low-light images. The synergy of illumination estimation, edge information enhancement, and noise suppression modules ensures that the input image remains clear and high-contrast under low-light conditions, providing a high-quality image foundation for subsequent behavior recognition.

[0144] (2) Accuracy and reliability of behavior recognition: The lightweight animal behavior recognition network of the present invention can accurately capture the behavioral characteristics of animals. Through the optimized design of the deep learning model, the system's behavior recognition accuracy in low-light environments is significantly improved, reducing recognition errors caused by insufficient light and enhancing the reliability of recognition results.

[0145] (3) Reduction of equipment costs and improvement of system applicability: Due to the adoption of a lightweight deep learning network structure, the present invention can achieve efficient processing without relying on high-performance computing equipment. This not only reduces equipment costs, but also improves the applicability and flexibility of the system, enabling it to be applied in more experimental environments.

[0146] (4) Technical support for animal behavior research: The overall approach significantly improves the accuracy of animal behavior observation in dim light environments, providing strong technical support for animal behavior research. The system can provide reliable recognition results in complex backgrounds and dynamically changing scenes, promoting research progress in related fields.

[0147] In summary, the present invention demonstrates excellent performance and broad application potential in animal behavior recognition in low-light scenes, and provides an innovative technical solution for research and application in related fields.

[0148] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. An animal behavior recognition software system for low-light scenes, characterized in that: It includes data acquisition module, image information enhancement module, lightweight animal behavior recognition module, data storage module, user interface module and system management module; The data acquisition module is used to collect multiple frames of low-light images in a low-light scene in real time, wherein the multiple frames of low-light images in the low-light scene include animals and background environment, and the animals and the background environment cannot be distinguished, and are converted according to a preset format to obtain multiple frames of low-light images after format conversion, and the multiple frames of low-light images after format conversion are arranged in chronological order to obtain a low-light image sequence, and the low-light image sequence is input into the image information enhancement module; The image information enhancement module is used to enhance the illumination and edge information of each frame of low illumination image in the low illumination image sequence. Specifically, for each frame of low illumination image in the low illumination image sequence, the illumination estimation module in the image information enhancement module is input to obtain an illumination enhanced feature map, and the illumination enhanced feature map is input to the noise suppression module in the image information enhancement module to obtain a denoised feature map. At the same time, each frame of low illumination image in the low illumination image sequence is input to the edge information enhancement module in the image information enhancement module to obtain an edge information enhanced feature map, and the denoised feature map and the edge information enhanced feature map are spliced ​​and superimposed in the dimension of the image channel to obtain a multi-channel feature map, and the multi-channel feature map is processed through a channel attention mechanism to obtain a mixed feature map, and then a mixed feature map corresponding to each frame of low illumination image is obtained, and the mixed feature map corresponding to each frame of low illumination image is input to a lightweight animal behavior recognition module; The lightweight animal behavior recognition module is used to extract spatial features in the mixed feature map corresponding to each frame of low-light image based on the spatial feature extractor contained in the module, wherein the spatial feature extractor is a lightweight deep learning network MobileNetV3, and then according to the Transformer encoder, in the feature map containing spatial features in multiple frames, the temporal features are extracted to obtain a feature map containing temporal features and spatial features, thereby identifying the animal behavior corresponding to the low-light image sequence and the probability that the animal in the low-light image sequence belongs to the animal behavior; The data storage module is used to store the video in the low-light scene collected in real time by the data collection module, and store the behavior of the animal in the video obtained by identification, and the probability that the animal in the video belongs to the behavior; The user interface module is used to display the video in the low-light scene collected in real time by the data collection module, and store the behavior of the animal in the video obtained by identification, and the probability of the animal in the video belonging to the behavior; The system management module is used to coordinate and manage the operation of various modules in the animal behavior recognition software system for low-light scenes.

2. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The illumination estimation module in the image information enhancement module is specifically used for: For each frame of the low-light image sequence, input it into a convolution layer with a convolution kernel size of 3*3 and a stride of 1, and input the feature map output by the convolution layer into a Relu function to achieve numerical conversion of feature values ​​in the multi-channel feature map; The feature map after numerical conversion is input into the residual structure. Specifically, the feature map after numerical conversion is input into a convolution layer with a convolution kernel size of 3*3, a stride of the convolution operation of 1, and a padding of the convolution operation of 1, and then the feature map output by the convolution layer is batch normalized, and the feature map after batch normalization is passed through a Relu function to obtain a first feature map, and the first feature map and the feature map after numerical conversion are superimposed to obtain a multi-scale feature map; the multi-scale feature map is input into a convolution layer with a convolution kernel size of 3*3 and a stride of the convolution operation of 1, and then the feature map processed by the convolution layer is input into a Sigmod function to obtain an illumination enhanced feature map.

3. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The noise suppression module in the image information enhancement module is specifically used for: Filling is performed for the illumination enhancement feature map corresponding to each frame image in the low illumination image sequence, wherein the illumination enhancement feature map includes a plurality of feature points. Specifically, the filling size is determined by the following formula: Among them, pad_size is the padding size, R is the preset search radius in the denoising process, and P is the size of the feature block where the preset feature point is located; The illumination enhanced feature map is filled based on the filling size. Specifically, the height of the illumination enhanced feature map is increased by 2 times the pad_size, the width of the illumination enhanced feature map is increased by 2 times the pad_size, and the feature values ​​of the increased feature points are filled to obtain a filled feature map; For each feature point in the filled feature map, determine the neighborhood block corresponding to the feature point. The neighborhood block is expressed by the following formula: Among them, N i,j represents the neighborhood block of the feature point, that is, the set of neighborhood points corresponding to the feature point, (i, j) represents the coordinates of the feature point in the filled feature map, x i+m,j+n Represents the eigenvalue of the neighborhood point corresponding to the feature point, (i+m,j+n) represents the coordinates of the neighborhood point in the filled feature map, m represents the displacement of the horizontal coordinate, and n represents the displacement of the vertical coordinate; Take the feature point as the center and R as the radius to determine the search range. For each feature point in the search range except the feature point, determine the neighborhood block N of other feature points. k,l , (k, l) is the coordinate of other feature points in the filled feature map, and then calculate the neighborhood block N i,j and neighborhood block N k,l The similarity is realized by the following formula: Among them, d(N i,j ,N k,l ) represents the neighborhood block N i,j and neighborhood block N k,l The similarity of x k+m,l+n Indicates the eigenvalues ​​of the neighborhood points corresponding to other feature points; Based on the neighborhood block N i,j and neighborhood block N k,l The similarity is calculated by the following formula: Among them, ω i,j,k,l represents the similarity weight, and h represents the parameter that controls the degree of noise filtering; Based on the above calculation, the similarity weight of each feature point is obtained, and then for each feature point, the cumulative weighted feature value is calculated, which is specifically calculated by the following formula: Among them, x k,l Represents other feature points x k,l The eigenvalues ​​of i,j represents the cumulative weighted eigenvalue; The final denoising result is obtained by dividing the accumulated weighted eigenvalues ​​by the accumulated weight sum, which is specifically expressed by the following formula: Among them, denoised_result i,j Represents the final denoising result of each feature point, ε is a constant; Thus, the final denoising result of all feature points is obtained, that is, the denoised feature map.

4. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The edge information enhancement module in the image information enhancement module is specifically used for: Inputting each frame of the low-illumination image sequence into a first Laplace function to obtain a first scale feature map, and inputting each frame of the low-illumination image sequence into a second Laplace function to obtain a second scale feature map, wherein the size of the first scale feature map is larger than the size of the second scale feature map; Using the Thor operator, extract information from the first scale feature map to obtain a feature map containing edge information in the vertical direction and a feature map containing edge information in the horizontal direction, fuse the feature map containing edge information in the vertical direction and the feature map containing edge information in the horizontal direction to obtain a first fused feature map, input the first fused feature map into a convolution layer with a convolution kernel size of 3×3 to extract deep features, and obtain a feature map containing deep features of the first scale feature map; Using the Thor operator, extract information from the second-scale feature map to obtain a feature map containing edge information in the vertical direction and a feature map containing edge information in the horizontal direction, fuse the feature map containing edge information in the vertical direction and the feature map containing edge information in the horizontal direction to obtain a second fused feature map, input the second fused feature map into a convolution layer with a convolution kernel size of 3×3 to extract deep features, and obtain a feature map containing deep features of the second-scale feature map; The feature map containing the deep features of the second-scale feature map is upsampled, and the upsampled feature map is fused with the feature map containing the deep features of the first-scale feature map to obtain a feature map with enhanced edge information.

5. The animal behavior recognition software system for low-light scenes according to claim 1, characterized in that: The lightweight animal behavior recognition module is specifically used for: For the mixed feature map corresponding to each frame of the low-light image, the mixed feature map is input into the spatial feature extractor, the spatial features in the mixed feature map are extracted, and a feature map containing the spatial features is obtained. The feature map containing the spatial features is linearly encoded to realize the conversion of the channel dimension of the feature map containing the spatial features into the channel dimension of the input of the Transformer encoder. At the same time, the feature map containing the spatial features is position-encoded, and the linearly encoded feature map and the position-encoded feature map corresponding to each frame of the low-light image in the low-light image sequence are fused to obtain the fused spatial features corresponding to all the low-light images. Figure, the fused spatial feature maps corresponding to all low-light images are input into the Transformer encoder, the temporal features are extracted from the fused spatial feature maps corresponding to all low-light images, and a feature map containing temporal features and spatial features is obtained, and then the feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, and then a classifier is used to calculate according to the global features to obtain the animal behavior to which the global feature belongs, that is, the animal behavior corresponding to the low-light image sequence, and the probability that the global feature belongs to the animal behavior, that is, the probability that the animal in the low-light image sequence belongs to the animal behavior.

6. The animal behavior recognition software system for low-light scenes according to claim 5, characterized in that: The feature map containing spatial features is positionally encoded, including: Based on the dimension of the transformer encoder input data, determine the range of the value of the position encoding dimension index corresponding to each feature point in the feature map containing the spatial feature; When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an even number, the position encoding is calculated using the following formula: Among them, PE pose,2index It represents the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an even number. Pose represents the position index of the low-light image in the low-light image sequence. Index is the position encoding dimension index, that is, the feature index of the transformer encoder input data. model is the dimension of the input data of the transformer encoder, and A is the scaling factor; When the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number, the position encoding is calculated using the following formula: Among them, PE pose,2index+1 Indicates the position encoding when the value of the position encoding dimension index corresponding to the feature point in the feature map containing the spatial feature is an odd number.

7. The animal behavior recognition software system for low-light scenes according to claim 5, characterized in that: The feature map containing temporal features and spatial features is processed by attention pooling to obtain global features, which are specifically expressed by the following formula: Among them, global_features represents global features, feature z Represents the feature map containing temporal features and spatial features corresponding to the z-th frame low-light image, α z represents the weight calculated by the attention mechanism corresponding to the z-th low-light image, and T is the total number of low-light images in the low-light image sequence.

Images