A method for identifying the species of captured mice based on image recognition

Abstract

The application relates to the technical field of image recognition, and discloses a mouse species recognition method based on image recognition. An original mouse image is input into a posture decoupling recognition network, and a heat map is output based on an anatomical prior anchor point; a rotation bias angle is calculated according to the heat map coordinates, an affine transformation is performed on the image, and a distorted body is cut and reorganized into independent local image blocks of a head, a trunk and a tail at a standard visual angle; spatial features are extracted by using parallel convolution channels, local features are mapped to a global feature space by a cross-region cross-attention mechanism, a species discrimination vector is fused and generated, and a classification result is output. The application converts non-standard posture restoration into extraction and reorganization of standard local features, eliminates the interference of body distortion and local occlusion in a limited space on global feature extraction, and overcomes the misclassification defects caused by the sensitivity of a conventional network to non-standard postures.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology and discloses a method for identifying the species of captured mice based on image recognition. Background Technology

[0002] Current methods for identifying the species of captured mice typically involve deploying a camera on the capture device to acquire images, which are then input into a global image classification network for species determination. Conventional global image classification networks use downsampling and convolution operations to extract feature maps from the entire image, directly mapping the species label to a classifier at the network's end. These methods rely on the target exhibiting a standard extended posture and a complete torso outline in the image, achieving classification by extracting global appearance features. When acquiring images in the field, a single lens combined with supplementary lighting is usually used for shooting.

[0003] In confined spaces such as rat traps, rats exhibit non-standard postures such as curling up and climbing due to stress responses, and their bodies are often obscured by the cage wires or bait. Conventional global image classification networks rely on the standardized posture and complete contour information of the target. When the target in the input image exhibits a non-standard posture and is occluded, the image lacks global integrity, and the features extracted by the network contain background noise and distortion information, leading to misclassification by networks based on global appearance features. Summary of the Invention

[0004] The purpose of this invention is to provide a method for identifying the species of captured mice based on image recognition, which can effectively solve the problems in the background art mentioned above.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for identifying captured mouse species based on image recognition includes: constructing an image acquisition and preprocessing process for the capture scene, and extracting the original mouse image and inputting it into a pre-constructed pose decoupling recognition network;

[0007] The attitude decoupling recognition network includes a key point detection branch, a local region segmentation module, and a feature aggregation and classification module.

[0008] The key point detection branch outputs heat maps of the mouse, trunk midline, and limb joints based on the pre-set anatomical prior anchor points of the captured scene.

[0009] The local region segmentation module performs an affine transformation on the original image based on the heatmap coordinates, and segments and reassembles the distorted body into three independent local image blocks of the head, torso, and tail from a standard perspective.

[0010] The feature aggregation and classification module uses parallel convolution channels to extract spatial features from the three independent local image blocks respectively. It then maps the local features to the global feature space through a cross-regional cross-attention mechanism, fuses them to generate a category discrimination vector, and outputs the classification result.

[0011] Preferably, the key point detection branch includes a main feature extraction layer and a multi-layer heatmap prediction head;

[0012] The backbone feature extraction layer downsamples the original mouse image and outputs a multi-scale feature map;

[0013] The multi-layer heatmap prediction head contains three parallel deconvolution layers, which correspond to the mouse, the torso midline, and the limb joints, respectively.

[0014] Each of the deconvolutional layers receives the multi-scale feature map and outputs a channel response map corresponding to the category;

[0015] An anatomical prior mask is superimposed on the channel response map. The anatomical prior mask is generated based on the physical boundary coordinates of the cage within the capture space and masks the background pixel response outside the physical boundary of the cage.

[0016] Preferably, after the local region segmentation module obtains the heatmap coordinates, it calculates the average horizontal and vertical coordinates of the mouse as the head reference point, and calculates the coordinates of the limb joints that are farthest from the head reference point as the tail reference point.

[0017] Using the line connecting the head reference point and the tail reference point as the rotation reference axis, calculate the rotation offset angle between the rotation reference axis and the horizontal coordinate axis;

[0018] A two-dimensional affine transformation matrix is ​​constructed based on the rotation offset angle, and the original image is then subjected to rotation transformation.

[0019] On the rotated image, a rectangular window is cropped with the head reference point, the tail reference point, and the midpoint of the torso's central axis as the region center, according to a set fixed pixel side length, to generate the three independent local image blocks: the head, the torso, and the tail.

[0020] Preferably, the feature aggregation and classification module includes three independent feature extraction sub-networks and a cross-attention fusion layer;

[0021] The input terminals of the three feature extraction sub-networks are respectively connected to the three independent local image blocks of the head, the torso, and the tail, and output the head feature vector, the torso feature vector, and the tail feature vector;

[0022] The cross-attention fusion layer uses the torso feature vector as a query matrix, concatenates the head feature vector and the tail feature vector as a key matrix and a value matrix, performs multi-head attention calculation, and outputs a fused feature vector.

[0023] The fused feature vector and the trunk feature vector are added together by residual addition, and then mapped through a fully connected layer to generate the category discrimination vector.

[0024] Preferably, the process of constructing the scene image acquisition and preprocessing includes: simultaneously acquiring the left-eye view and the right-eye view through a binocular camera deployed on the acquisition device;

[0025] Epipolar correction and stereo matching are performed on the left-eye view and the right-eye view to generate a disparity map;

[0026] Calculate the depth value of each pixel in the capture space based on the disparity map, and construct a depth mask map;

[0027] The depth mask is used to filter background regions in the original mouse image whose depth values ​​are greater than a preset depth threshold, while retaining foreground connected regions whose depth values ​​are within the preset depth threshold range. The foreground connected regions are then cropped with bounding rectangles to generate the original mouse image that meets the input size requirements of the pose decoupling recognition network.

[0028] Preferably, a class prototype vector library is constructed before outputting the classification results;

[0029] The species prototype vector library stores standard feature vectors of multiple known mouse species. The standard feature vectors are obtained by inputting multiple standard pose images of multiple known mouse species into the pose decoupling recognition network and calculating the average value.

[0030] Calculate the cosine similarity between the category discrimination vector and each of the standard feature vectors in the category prototype vector library;

[0031] Arrange the cosine similarities in descending order, extract the two cosine similarities at the top of the list, and calculate the difference between the two cosine similarities at the top of the list.

[0032] When the difference is less than the set difference threshold, a resampling instruction is triggered, and the construction and capture scene image acquisition and preprocessing process is re-executed.

[0033] Preferably, the step of superimposing an anatomical prior mask on the channel response map includes: obtaining the set of coordinates of the intersection points of the cage wires in the original rodent image, and connecting the set of intersection point coordinates to generate a set of grid line segments;

[0034] Calculate the normal vector of each grid segment in the set of grid segments, and expand the grid segments outward at equal intervals along the direction of the normal vector to generate an expanded grid region;

[0035] In the coordinate system corresponding to the channel response map, the pixel values ​​within the outer grid area are set to 0, while the pixel values ​​outside the outer grid area are retained to generate a filtered channel response map.

[0036] The multi-layer heatmap prediction head extracts local maximum coordinates as the final heatmap coordinates based on the filtered channel response map using a non-maximum suppression algorithm.

[0037] Preferably, the step of cropping a rectangular window according to a set fixed pixel side length includes: extracting all coordinate points on the central axis of the torso, calculating the Euclidean distance between adjacent coordinate points and summing them to obtain the torso length value;

[0038] The torso length value is divided into three equal segments, with the endpoint closest to the head reference point as the center of the head window, the endpoint closest to the tail reference point as the center of the tail window, and the midpoint of the middle segment as the center of the torso window.

[0039] Obtain the pre-configured head proportion coefficient, torso proportion coefficient, and tail proportion coefficient, and multiply the head proportion coefficient, torso proportion coefficient, and tail proportion coefficient by the torso length value respectively to calculate the head window side length, torso window side length, and tail window side length.

[0040] The rectangular window capture operation is performed based on the center of the head window and its side length, the center of the torso window and its side length, and the center of the tail window and its side length.

[0041] Preferably, the multi-head attention calculation includes: inputting the query matrix into a first linear mapping layer to generate a first mapping matrix, inputting the key matrix into a second linear mapping layer to generate a second mapping matrix, and inputting the value matrix into a third linear mapping layer to generate a third mapping matrix;

[0042] Perform matrix multiplication on the first mapping matrix and the second mapping matrix, and divide the product by the scaling factor to obtain the attention weight matrix;

[0043] Input the attention weight matrix into the sparse activation function, and set the elements in the attention weight matrix that are less than the activation threshold to 0;

[0044] The attention weight matrix after sparsification is multiplied by the third mapping matrix, and the output is input into the fourth linear mapping layer to generate the fused feature vector.

[0045] Preferably, the step of cropping the foreground connected region with an outer rectangle includes: obtaining the set of abscissas and the set of ordinates of all pixels in the foreground connected region, extracting the maximum and minimum abscissa values ​​in the set of abscissas, and extracting the maximum and minimum ordinate values ​​in the set of ordinates.

[0046] The initial bounding rectangle is generated by using the difference between the maximum and minimum horizontal coordinate values ​​as the initial width and the difference between the maximum and minimum vertical coordinate values ​​as the initial height.

[0047] Calculate a first ratio of the target input width to the initial width of the attitude decoupling recognition network, and calculate a second ratio of the target input height to the initial height of the attitude decoupling recognition network;

[0048] The minimum value between the first ratio and the second ratio is selected as the uniform scaling factor. The initial bounding rectangle is scaled proportionally according to the uniform scaling factor. The scaled initial bounding rectangle is then placed in the center of the blank canvas. Pixels that exceed the boundary of the blank canvas are filled with zero values.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] 1. This invention constructs a pose decoupling recognition network, extracts heatmaps based on preset anatomical prior anchor points, calculates rotation offset angles based on heatmap coordinates, performs affine transformations on the images, and segments and reassembles distorted bodies into independent local image patches with standard perspectives. A cross-regional attention mechanism is then used to fuse local features to generate a category discrimination vector. This method transforms non-standard pose reconstruction in a confined space into the extraction and reassembly of standard local features, shielding background pixel responses outside the physical boundaries of the cage and eliminating the influence of body distortion and local occlusion on the global feature extraction process.

[0051] 2. Views acquired via binocular cameras are subjected to epipolar correction and stereo matching to generate a disparity map. A depth mask is used to filter background areas while preserving foreground connected components, eliminating depth interference factors in the captured scene. In heatmap prediction, an outer grid region mask generated based on the coordinate set of the cage wire intersections is overlaid, setting pixel values ​​within the outer grid region to 0, thus eliminating interference from the cage wires on the feature response. During feature aggregation, a sparse activation function is used to process the attention weight matrix, setting elements below the activation threshold to 0, suppressing the coupling of irrelevant local features during feature mapping. Attached Figure Description

[0052] Figure 1 This is a flowchart of the scene image acquisition and preprocessing process of the present invention;

[0053] Figure 2 This is a flowchart of the binocular depth mask generation and foreground clipping process of the present invention;

[0054] Figure 3 This is a flowchart of the key point detection branch processing of the present invention;

[0055] Figure 4 This is a flowchart illustrating the local region segmentation and affine transformation process of this invention.

[0056] Figure 5 This is a flowchart of the feature aggregation and classification module of the present invention;

[0057] Figure 6 This is a flowchart illustrating the construction and similarity verification process of the prototype vector library for this invention. Detailed Implementation

[0058] Please refer to the attached document. Figure 1 This embodiment provides a rat species identification method based on image recognition. The capture scenario is the internal capture space of a closed rat capture device. An image acquisition unit is provided on the side wall of the capture device, and the field of view of the image acquisition unit completely covers the entire area of ​​the capture space.

[0059] A capture scene image acquisition and preprocessing workflow is constructed. The image acquisition unit acquires a sequence of images within the capture space according to a preset acquisition cycle. Background modeling and differential processing are performed on the sequence of images to extract target images containing moving foregrounds. Pixel value normalization is performed on the target images, mapping the pixel values ​​to a preset numerical range. Simultaneously, the target images are filtered to remove random noise, resulting in original rodent images containing the rodent target. Before being input into the pre-constructed pose decoupling recognition network, the original rodent images undergo size adaptation processing to match their width and height dimensions with the preset input dimensions of the pose decoupling recognition network. The original rodent images with completed size adaptation are then input into the pose decoupling recognition network.

[0060] The pose decoupling recognition network comprises a keypoint detection branch, a local region segmentation module, and a feature aggregation and classification module. In this embodiment, the keypoint detection branch receives the original mouse image and, based on preset anatomical prior anchor points for the captured scene, outputs heatmaps corresponding to the mouse, the trunk midline, and the limb joints. The preset anatomical prior anchor points are set based on the standard anatomical structure of mice and are divided into three categories: the first category is the anchor points corresponding to the mouse, including the tip of the nose, the center point of the left eye socket, and the center point of the right eye socket; the second category is the anchor points corresponding to the trunk midline, including the starting point of the cervical vertebrae, the midpoint of the thoracic vertebrae, the midpoint of the lumbar vertebrae, and the ending point of the sacrum; and the third category is the anchor points corresponding to the limb joints, including the wrist, elbow, and shoulder joints of the left forelimb, the wrist, elbow, and shoulder joints of the right forelimb, the ankle, knee, and hip joints of the left hindlimb, and the ankle, knee, and hip joints of the right hindlimb.

[0061] The keypoint detection branch performs multi-scale convolutional feature extraction on the input raw mouse image, generating feature maps containing semantic information at different levels. Based on the multi-scale feature maps, it predicts the response of the anatomical prior anchors for the three categories, generating a heatmap for each category. The value of each pixel in the heatmap represents the probability of the presence of the corresponding category anchor at that location.

[0062] The local region segmentation module receives the heatmap output from the keypoint detection branch and the original mouse image. Based on the heatmap coordinates, it performs an affine transformation on the original image, segmenting and reconstructing the distorted body into three independent local image blocks from a standard viewpoint: head, torso, and tail. Local extrema are extracted from the heatmap for each category, obtaining the pixel coordinates corresponding to each anatomical prior anchor point, forming a heatmap coordinate set. The mouse anchor point coordinates are extracted from the heatmap coordinate set, and the mean of the horizontal and vertical coordinates of all mouse anchor points is calculated as the head reference point. The limb joint anchor point coordinates are extracted from the heatmap coordinate set, and the Euclidean distance between each limb joint anchor point coordinate and the head reference point is calculated. The limb joint anchor point coordinate with the largest Euclidean distance is selected as the tail reference point.

[0063] Using the line connecting the head and tail reference points as the rotation reference axis, the rotation offset angle between the rotation reference axis and the horizontal coordinate axis of the image plane is calculated. Based on the rotation offset angle, a two-dimensional affine transformation matrix is ​​constructed. This matrix is ​​then used to rotate the original mouse image, aligning the rotation reference axis of the mouse body with the horizontal coordinate axis, resulting in a pose-normalized corrected image. In the corrected image, the head reference point, the midpoint of the torso's central axis, and the tail reference point are used as region centers. Rectangular windows are cropped according to a preset fixed pixel side length, generating three independent local image patches for the head, torso, and tail. The sizes of these three local image patches match the preset input size of the feature aggregation and classification module.

[0064] The feature aggregation and classification module uses parallel convolutional channels to extract spatial features from three independent local image patches. Through a cross-regional attention mechanism, the local features are mapped to the global feature space, fused to generate a category discrimination vector, and the classification result is output. The feature aggregation and classification module sets up three parallel feature extraction convolutional channels. The three convolutional channels have the same structure and independent weights, corresponding to the head, torso, and tail image patches, respectively.

[0065] Each convolutional channel contains multiple cascaded convolutional blocks. Each convolutional block consists of a convolutional layer, a batch normalization layer, and an activation function layer. The convolutional layer performs sliding window convolution operations on the input local image patches to extract local spatial features. The batch normalization layer normalizes the numerical distribution of the output results of the convolution operation. The activation function layer performs nonlinear mapping on the normalized results. Multiple cascaded convolutional blocks progressively downsample the input image to extract semantic features at different levels. A global average pooling layer is set at the end of each convolutional channel to perform global average pooling on the multi-channel feature maps output by the convolutional block to generate a one-dimensional feature vector. The three convolutional channels output head feature vector, torso feature vector, and tail feature vector, respectively.

[0066] The cross-regional attention mechanism receives three local feature vectors, using the trunk feature vector as the baseline feature for global association, and the head and tail feature vectors as supplementary features. Through cross-attention operations, the local features of the head and tail are mapped to a global feature space based on the trunk features, generating a fused feature vector that integrates semantic information from the three local regions. This fused feature vector is then input into a fully connected layer for linear mapping, generating a species discrimination vector with dimensions matching the preset number of mouse species. Based on the numerical distribution of the species discrimination vector, the mouse species corresponding to the dimension with the largest value is selected as the final classification result.

[0067] In this embodiment, in order to clarify the input and output specifications of each core module of the attitude decoupling recognition network, the input and output parameters of each module are defined as shown in Table 1.

[0068] Table 1. Definition of Input / Output Parameters for Core Modules of Attitude Decoupling Recognition Network

[0069] Module Name Input data Output data Data Dimensions Physical meaning Keypoint detection branch Original rodent images Heat maps of mouse, trunk midline, and limb joints Input: H×W×3 Output: H×W×3 H is the image height, W is the image width, and 3 is the number of RGB channels / number of heatmap categories; the input is a three-channel color image, and the output is a response heatmap for each of the three channels corresponding to the three types of anchor points. Local region segmentation module Original rodent images, three types of heat maps Local image patches of head, torso, and tail Input: H×W×3 + H×W×3 Output: 3×h×w×3 h is the height of the local image patch, and w is the width of the local image patch; outputs three sets of three-channel local image patches with the same size. Feature aggregation classification module Three independent local image patches Specimen vector, classification result Input: 3 × h × w × 3 Output: C × 1 + category label C represents the total number of preset mouse species; the output dimension is a species discrimination vector of C and its corresponding category label.

[0070] In this embodiment, a heatmap of anatomical anchor points of rodents is extracted through a key point detection branch. Based on the coordinates of the heatmap, the original image is subjected to affine transformation and local region segmentation. The rodent body with non-standard posture in the confined space is transformed into three independent local image blocks with a standard viewpoint. Local features are then extracted through parallel convolution channels and fused through a cross-regional attention mechanism to generate classification results, thus avoiding interference from non-standard posture and local occlusion on global feature extraction.

[0071] Please refer to the attached document. Figure 2 In one optional embodiment, when constructing the scene image acquisition and preprocessing workflow, a binocular camera deployed on the acquisition device simultaneously acquires left-eye and right-eye views. The optical axes of the two lenses of the binocular camera are parallel to each other, and the imaging planes of the two lenses are in the same plane. Epipolar correction is performed on the simultaneously acquired left-eye and right-eye views to ensure that the corresponding epipolar lines of the left-eye and right-eye views are in the same horizontal row, eliminating the vertical parallax between the binocular views and retaining only the horizontal parallax component.

[0072] Stereo matching is performed on the left and right eye views after epipolar correction. Based on a preset matching window size, a matching pixel in the right eye view is searched along the horizontal epipolar line corresponding to each pixel position in the left eye view. The horizontal disparity value for each pixel is calculated, generating a disparity map consistent with the view size. The depth value of each pixel in the capture space is calculated based on the disparity map, and a depth mask is constructed. Specifically, based on the intrinsic and extrinsic parameters of the binocular camera, a mapping relationship between disparity values ​​and depth values ​​is established. Each disparity value in the disparity map is mapped and calculated to obtain the depth value corresponding to each pixel. The depth value represents the vertical distance between the scene point corresponding to that pixel and the binocular camera's imaging plane. The formula for calculating the depth value is:

[0073]

[0074] in, This represents the depth value corresponding to the pixel. The equivalent focal length of the binocular camera. The baseline length of the binocular camera. This represents the horizontal disparity value corresponding to a pixel.

[0075] A preset depth threshold is set, corresponding to the distance between the front and rear boundaries of the capture space. Pixels with depth values ​​greater than the preset threshold in the depth mask are set to 0, while pixels with depth values ​​within the preset threshold range are set to 1, generating a binarized depth mask. A matrix multiplication operation is performed between the depth mask and the original captured image to filter out background regions with depth values ​​greater than the preset threshold, retaining foreground connected components with depth values ​​within the preset threshold range. The retained foreground connected components are then cropped with bounding rectangles to generate an original mouse image that meets the input size requirements of the pose decoupling recognition network.

[0076] Please refer to the attached document. Figure 3 Furthermore, the keypoint detection branch includes a backbone feature extraction layer and a multi-layer heatmap prediction head. The backbone feature extraction layer downsamples the input raw mouse image and outputs multi-scale feature maps. The backbone feature extraction layer adopts a cascaded convolutional structure, containing multiple downsampling modules and residual connection modules. The input raw mouse image first passes through an initial convolutional layer for feature extraction. The kernel size of the initial convolutional layer is 7×7, the stride is 2, and the number of output channels is 64. After batch normalization and activation function processing, the output of the initial convolutional layer is input into multiple cascaded residual modules. Each residual module contains two 3×3 convolutional layers and a cross-layer residual connection. The stride of some residual modules is set to 2 to achieve feature map downsampling. After processing by multiple residual modules, the backbone feature extraction layer outputs four sets of feature maps at different scales, which are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size, forming multi-scale feature maps.

[0077] The multi-layer heatmap prediction head contains three parallel deconvolutional layers, corresponding to the mouse, torso midline, and limb joints categories, respectively. Each deconvolutional layer receives a multi-scale feature map output from the main feature extraction layer and outputs a channel response map for the corresponding category. Each deconvolutional layer employs a three-stage upsampling structure, with each stage containing one deconvolutional layer and one convolutional layer. The stride of the deconvolutional layer is 2, achieving feature map size upsampling. The convolutional layer adjusts the number of channels in the feature map. The three-stage upsampling structure progressively upsamples the input multi-scale feature map to the same size as the original mouse image. At the end of each deconvolutional layer, a 1×1 convolutional layer is placed to adjust the number of channels in the feature map to 1, generating a single-channel response map. The value of each pixel in the channel response map represents the response probability of the corresponding category anchor point at that location.

[0078] An anatomical prior mask is superimposed on the channel response map. This mask is generated based on the physical boundary coordinates of the cage within the capture space, masking the background pixel responses outside the cage's physical boundaries. Specifically, the set of intersection coordinates of the cage wires in the original rodent image is obtained. This set is acquired by performing edge detection and line fitting on the original rodent image. Edge detection is performed on the original rodent image to extract straight edges, and intersection points are calculated for these straight edges to obtain the grid intersection coordinates of the cage wires, forming the intersection coordinate set. Adjacent coordinates in the intersection coordinate set are connected to generate a set of grid segments, which correspond to the physical locations of the cage wires.

[0079] Calculate the normal vector of each grid segment in the grid segment set. Expand the grid segments outwards at equal intervals along the two opposite directions of the normal vector to generate an expanded grid region. The expansion distance is set according to the imaging width of the cage wire, ensuring that the expanded grid region completely covers the imaging area of ​​the cage wire in the image. In the coordinate system corresponding to the channel response map, establish a binary mask matrix with the same size as the channel response map. Set the mask matrix values ​​corresponding to the pixel positions within the expanded grid region to 1, and set them to 0 in other areas to generate the cage mask matrix. Perform a matrix multiplication operation on the original channel response map and the result of subtracting the cage mask matrix from 1. Set the pixel values ​​within the expanded grid region to 0, and retain the pixel values ​​outside the expanded grid region to generate the filtered channel response map. The corresponding calculation formula is:

[0080]

[0081] in, The filtered channel response graph is plotted on coordinates. Pixel value at that location, The original channel response map in coordinates Pixel value at that location, The cage mask matrix in coordinates The value at that location, These are the horizontal and vertical coordinates of the image pixels.

[0082] The multi-layer heatmap prediction head is based on the filtered channel response map. It extracts the coordinates of local maxima as the final heatmap coordinates through a non-maximum suppression algorithm. It performs a sliding window traversal on the filtered channel response map. Within each sliding window of a preset size, it retains only the pixel with the largest value in the window and suppresses the response of the other pixels. After the traversal is completed, it extracts the coordinates of all retained pixels as the heatmap coordinates of the corresponding category.

[0083] Furthermore, when clipping the foreground connected region with a bounding rectangle, the set of x-coordinates and y-coordinates of all pixels within the foreground connected region are obtained. The maximum and minimum x-coordinate values ​​in the x-coordinate set and the maximum and minimum y-coordinate values ​​in the y-coordinate set are extracted. The difference between the maximum and minimum x-coordinate values ​​is used as the initial width, and the difference between the maximum and minimum y-coordinate values ​​is used as the initial height to generate an initial bounding rectangle that completely encloses all pixels of the foreground connected region.

[0084] Calculate a first ratio of the target input width to the initial width for the attitude decoupling recognition network, and a second ratio of the target input height to the initial height. Select the minimum of the first and second ratios as a uniform scaling factor. Scale the image region corresponding to the initial bounding rectangle proportionally according to this uniform scaling factor, ensuring that the width or height of the scaled image region is equal to the corresponding side of the target input size, and the other side is less than or equal to the corresponding side of the target input size. Center the scaled image region within a blank canvas of the same size as the target input size. Fill pixels that extend beyond the blank canvas boundary with zero values. Similarly, fill pixel positions in the blank canvas not covered by the scaled image region with zero values, generating an original mouse image that meets the input size requirements of the attitude decoupling recognition network.

[0085] In this embodiment, to clarify the key parameter configuration specifications of the binocular image acquisition and preprocessing process, each core parameter is defined as shown in Table 2.

[0086] Table 2 Key Parameter Configuration Table for Binocular Image Acquisition and Preprocessing Process

[0087] Parameter name Parameter value range Physical meaning of parameters Configuration basis Preset depth threshold Capture space front and rear boundary distance range Depth threshold for distinguishing foreground from background Based on the physical size settings of the capture space, ensure that only the target area within the capture space is retained. 3D matching window size Odd-numbered sizes from 3×3 to 11×11 Sliding window size during stereo matching Based on image resolution and scene texture richness settings, a balance is struck between matching accuracy and computation speed. Grid segment extension distance 1 to 5 pixels The outer width of the cage wire area Based on the imaging width setting of the cage wire in the image, ensure complete coverage of the wire imaging area. Non-maximum suppression window size Odd-numbered sizes from 3×3 to 7×7 Sliding window size for local maximum extraction Based on the imaging size setting of the anchor points, avoid mutual interference between the responses of adjacent anchor points. Uniform scaling factor Positive numbers between 0 and 1 Foreground area scaling factor Calculated based on the ratio of the target input size to the initial bounding rectangle size.

[0088] In this embodiment, a depth mask is generated by acquiring a view using a binocular camera, which filters out the background area outside the capture space. A mask for the outward-expanding grid area is generated using the intersection coordinates of the cage wires, which shields the pixel response corresponding to the cage wires, eliminating the interference of background and cage occlusion on key point detection and improving the extraction accuracy of heat map coordinates. At the same time, an image that conforms to the network input size is generated by scaling proportionally and zero-value filling, avoiding deformation and distortion of the rodent target.

[0089] Please refer to the attached document. Figure 4 In one optional embodiment, after the local region segmentation module obtains the heatmap coordinates, it calculates the average of the horizontal and vertical coordinates of all anchor points corresponding to the mouse as the head reference point, and calculates the coordinates of the limb joints farthest from the head reference point as the tail reference point. The set of anchor point coordinates corresponding to the mouse is as follows: ,in The number of mouse anchor points, and the coordinates of the head reference point. The set of anchor point coordinates corresponding to the joints of the limbs is obtained by averaging the x and y coordinates of all coordinates within the set. ,in Given the number of limb joint anchor points, calculate the Euclidean distance between each coordinate in the set and the head reference point. The formula for calculating the Euclidean distance is: The coordinate point with the largest Euclidean distance is selected as the tail reference point. .

[0090] Using the line connecting the head reference point and the tail reference point as the rotation reference axis, calculate the rotation offset angle between the rotation reference axis and the horizontal coordinate axis. The formula for calculating the rotation offset angle is:

[0091]

[0092] in, This is the rotation offset angle, in radians. The pixel coordinates of the head reference point. The pixel coordinates of the tail reference point are shown. The positive direction of the horizontal axis is the direction in which the horizontal coordinates of the image pixel coordinates increase, and the positive direction of the vertical axis is the direction in which the vertical coordinates of the image pixel coordinates increase.

[0093] The two-dimensional affine transformation matrix is ​​constructed based on the rotation bias angle. The expression for the two-dimensional affine transformation matrix is ​​as follows:

[0094]

[0095] in, The matrix is ​​a 2x3 two-dimensional affine transformation matrix. The first two columns are rotation components and the third column is a translation component. The affine transformation rotates the original mouse image with the head reference point as the rotation center, so that the rotation reference axis coincides with the horizontal coordinate axis, resulting in a posture-normalized corrected image. In the corrected image, the central axis of the mouse body is parallel to the horizontal direction, eliminating the rotational distortion of the mouse body in the original image.

[0096] On the corrected image after rotation transformation, rectangular windows are cropped with fixed pixel side lengths, using the head reference point, tail reference point, and the midpoint of the torso's central axis as region centers, to generate three independent local image blocks for the head, torso, and tail. All coordinate points on the torso's central axis are extracted to form a torso central axis coordinate set. ,in The number of anchor points along the central axis of the torso is given, with coordinate points arranged from head to tail. The Euclidean distances between adjacent coordinate points are calculated and summed to obtain the torso length value. The formula for calculating the torso length value is:

[0097]

[0098] in, This is the torso length value, in pixels. The number of coordinate points along the midline of the torso. The first on the central axis of the torso The pixel coordinates of each coordinate point.

[0099] The torso length is divided into three equal segments, resulting in three equal-length intervals. The endpoint of the segment closest to the head reference point is used as the center of the head window, the endpoint of the segment closest to the tail reference point is used as the center of the tail window, and the midpoint of the middle segment is used as the center of the torso window. Pre-configured head, torso, and tail proportion coefficients are obtained. These coefficients are then multiplied by the torso length value to calculate the side lengths of the head, torso, and tail windows. The corresponding calculation formulas are as follows:

[0100]

[0101] in, , , These are the side lengths of the squares representing the head window, torso window, and tail window, respectively, in pixels. , , These are pre-configured head proportion coefficients, trunk proportion coefficients, and tail proportion coefficients, which are pre-set based on the body proportion characteristics of different mouse species.

[0102] Based on the center and side length of the head window, the center and side length of the torso window, and the center and side length of the tail window, a rectangular window is cropped. Taking the center of each window as the center point, the corresponding window side length is expanded by half in the horizontal and vertical directions to form a square cropping window. The pixel area within the cropping window in the corrected image is extracted to generate three independent local image blocks for the head, torso, and tail. Zero-value padding is performed on the area of ​​the cropping window that exceeds the boundary of the corrected image to ensure that the size of the three local image blocks is consistent.

[0103] In this embodiment, in order to clarify the configuration specifications of the local image block cropping ratio coefficients corresponding to different mouse species, the ratio coefficients for each mouse species are defined, as shown in Table 3.

[0104] Table 3. Correspondence between local image patch cropping ratio coefficients and window parameters

[0105] Mouse species Head proportion coefficient Torso proportion coefficient Tail proportion factor Torso length reference range (pixels) Brown rat 0.6 0.8 1.2 80-200 Yellow-breasted rat 0.55 0.75 1.3 70-180 House mouse 0.5 0.7 0.9 40-100 Black-striped field mouse 0.52 0.72 1.1 50-120 Society rat 0.58 0.78 1.25 60-150

[0106] In this embodiment, the size of the cropping window for the local image patch is calculated based on the torso length value of the rodent and a preset scaling factor, so that the cropped local image patch can adapt to the body proportion features of different rodent species and fully cover the anatomical features of the corresponding local area. At the same time, the rotational distortion of the rodent's body is eliminated by affine transformation, and the non-standard posture body is transformed into a local image patch with a standard viewpoint, avoiding the interference of body distortion on local feature extraction.

[0107] Please refer to the attached document. Figure 5 In one optional embodiment, the feature aggregation and classification module includes three independent feature extraction sub-networks and a cross-attention fusion layer. The inputs of the three feature extraction sub-networks are respectively connected to three independent local image patches of the head, torso, and tail, and output head feature vector, torso feature vector, and tail feature vector. Each feature extraction subnetwork contains five cascaded convolutional blocks. Each convolutional block consists of a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The first convolutional block has a 7×7 kernel size, a stride of 2, and 64 output channels. The second to fifth convolutional blocks have a 3×3 kernel size, a stride of 1, and 128, 256, 512, and 1024 output channels, respectively. The output of each convolutional block is processed by the ReLU activation function layer before being input into the next convolutional block. The output of the fifth convolutional block is input into a global average pooling layer, which performs a global average calculation on each channel of the multi-channel feature map, outputting a one-dimensional feature vector with a dimension of 1024. The three feature extraction subnetworks each output a head feature vector with the same dimension. Trunk feature vector With tail feature vector .

[0108] The cross-attention fusion layer uses the torso feature vector as the query matrix and concatenates the head and tail feature vectors as the key and value matrices, respectively. Multi-head attention computation is then performed to output the fused feature vector. First, the head and tail feature vectors are concatenated along the channel dimension to generate the concatenated feature vector. The concatenated feature vector has a dimension of 2048. The torso feature vector is input into the first linear mapping layer to generate the first mapping matrix, the concatenated feature vector is input into the second linear mapping layer to generate the second mapping matrix, and the concatenated feature vector is input into the third linear mapping layer to generate the third mapping matrix. The corresponding calculation formula is as follows:

[0109]

[0110] in, For querying the matrix, the dimension is , The key matrix has dimensions of . , It is a value matrix with dimension . , , , These are the weight matrices for the first, second, and third linear mapping layers, respectively, each with dimension [missing information]. , , , For multi-head attention feature dimensions, For the torso feature vector, This is for concatenating feature vectors.

[0111] Perform matrix multiplication on the first mapping matrix and the second mapping matrix, and divide the product by the scaling factor to obtain the attention weight matrix. The corresponding calculation formula is as follows:

[0112]

[0113] in, This is the attention weight matrix, with dimension 1. , The dimensions of the key matrix are the same as the dimensions of the query matrix. This is a scaling factor used to prevent the activation function from saturating due to excessively large values ​​after matrix multiplication.

[0114] The attention weight matrix is ​​input into the sparse activation function, and elements in the attention weight matrix less than the activation threshold are set to 0. The expression for the sparse activation function is:

[0115]

[0116] in, This is the attention weight matrix after sparsification. Let be the element in the i-th row and j-th column of the attention weight matrix. The preset activation threshold, These are the row and column indices of the matrix elements.

[0117] The sparsified attention weight matrix is ​​multiplied by the third mapping matrix, and the output is input into the fourth linear mapping layer to generate the fused feature vector. The corresponding calculation formula is as follows:

[0118]

[0119] in, To fuse feature vectors, the dimension is 1024. This is the weight matrix of the fourth linear mapping layer, with dimension . , It is a value matrix.

[0120] The fused feature vector is added to the body feature vector by their residuals, and then mapped through a fully connected layer to generate a class discrimination vector. The corresponding calculation formula is as follows:

[0121]

[0122] in, This is the feature vector resulting from the sum of the residuals, with a dimension of 1024. The category discrimination vector has a dimension of . , The total number of preset mouse species. Here is the weight matrix of the fully connected layer, with dimension 1. , The bias term for the fully connected layer has a dimension of .

[0123] Please refer to the attached document. Figure 6 Before outputting the classification results, a species prototype vector library is constructed. This library stores standard feature vectors for multiple known rodent species. These standard feature vectors are obtained by inputting multiple standard pose images of known rodent species into a pose decoupling recognition network and averaging the results. For each known rodent species, a predetermined number of standard pose images are collected. In these images, the rodent's body is extended, unobstructed, and the viewing angle matches the image acquisition angle of the captured scene. All standard pose images are input into the pose decoupling recognition network, and the residual feature vector preceding the species discrimination vector for each image is extracted. The element-wise average of all residual feature vectors for the same rodent species is calculated to obtain the standard feature vector corresponding to that species. The standard feature vectors of all known rodent species and their corresponding species category labels are stored to form the species prototype vector library.

[0124] Calculate the cosine similarity between the residual feature vector corresponding to the category discrimination vector and each standard feature vector in the category prototype vector library. The formula for calculating the cosine similarity is:

[0125]

[0126] in, Let be the cosine similarity between the residual eigenvector and the standard eigenvector of the i-th mouse species. This represents the standard feature vector of the i-th mouse species in the species prototype vector library. Let L2 norm be the residual eigenvector. The L2 norm of the standard feature vectors is given, and the cosine similarity ranges from -1 to 1. A larger value indicates a higher similarity between the two feature vectors.

[0127] Sort all the calculated cosine similarities in descending order, extract the two cosine similarities at the top of the list, and calculate the difference between the two top cosine similarities. The formula for calculating the difference is:

[0128]

[0129] in, The similarity difference. This represents the first cosine similarity after descending order. This is the second cosine similarity after descending order. When the difference is less than the set difference threshold, it indicates that the confidence of the current classification result is insufficient, triggering a resampling instruction to re-execute the scene image acquisition and preprocessing process, obtain new original mouse images, and input them into the pose decoupling recognition network for recognition; when the difference is greater than or equal to the set difference threshold, the mouse species category corresponding to the standard feature vector with the highest cosine similarity is output as the final classification result.

[0130] In this embodiment, to clarify the construction parameters of the category prototype vector library and the threshold configuration specifications for similarity verification, each core parameter is defined, as shown in Table 4.

[0131] Table 4. Parameters and Similarity Threshold Configuration for Rat Species Prototype Vector Library Construction

[0132] Mouse species name Number of standard pose images Standard feature vector dimension Difference threshold Activation threshold Brown rat 2000 1024 0.15 0.1 Yellow-breasted rat 1800 1024 0.15 0.1 House mouse 2000 1024 0.12 0.1 Black-striped field mouse 1500 1024 0.12 0.1 Society rat 1200 1024 0.12 0.1

[0133] In this embodiment, a cross-attention mechanism using torso features as the query matrix is ​​used to associate and fuse local features of the head and tail with torso features. The sparse activation function suppresses the weights of irrelevant features, avoiding interference from invalid features in occluded areas on the fusion process. At the same time, similarity verification using the category prototype vector library triggers resampling for classification results with low confidence, further reducing the probability of misjudgment and improving the stability and accuracy of the classification results.

Claims

1. A method for identifying the species of captured mice based on image recognition, characterized in that, include: A capture scene image acquisition and preprocessing process is constructed, and the original mouse images are extracted and input into a pre-constructed pose decoupling recognition network; The attitude decoupling recognition network includes a key point detection branch, a local region segmentation module, and a feature aggregation and classification module. The key point detection branch outputs heat maps of the mouse, trunk midline, and limb joints based on the pre-set anatomical prior anchor points of the captured scene. The local region segmentation module performs an affine transformation on the original image based on the heatmap coordinates, and segments and reassembles the distorted body into three independent local image blocks of the head, torso, and tail from a standard perspective. The feature aggregation and classification module uses parallel convolution channels to extract spatial features from the three independent local image blocks respectively. It then maps the local features to the global feature space through a cross-regional cross-attention mechanism, fuses them to generate a category discrimination vector, and outputs the classification result.

2. The method for identifying mouse species based on image recognition according to claim 1, characterized in that, The key point detection branch includes a main feature extraction layer and a multi-layer heatmap prediction head; The backbone feature extraction layer downsamples the original mouse image and outputs a multi-scale feature map; The multi-layer heatmap prediction head contains three parallel deconvolution layers, which correspond to the mouse, the torso midline, and the limb joints, respectively. Each of the deconvolutional layers receives the multi-scale feature map and outputs a channel response map corresponding to the category; An anatomical prior mask is superimposed on the channel response map. The anatomical prior mask is generated based on the physical boundary coordinates of the cage within the capture space and masks the background pixel response outside the physical boundary of the cage.

3. The method for identifying mouse species based on image recognition according to claim 1, characterized in that, After obtaining the heatmap coordinates, the local region segmentation module calculates the average horizontal and vertical coordinates of the mouse as the head reference point, and calculates the coordinates of the limb joints that are farthest from the head reference point as the tail reference point. Using the line connecting the head reference point and the tail reference point as the rotation reference axis, calculate the rotation offset angle between the rotation reference axis and the horizontal coordinate axis; A two-dimensional affine transformation matrix is ​​constructed based on the rotation offset angle, and the original image is then subjected to rotation transformation. On the rotated image, a rectangular window is cropped with the head reference point, the tail reference point, and the midpoint of the torso's central axis as the region center, according to a set fixed pixel side length, to generate the three independent local image blocks: the head, the torso, and the tail.

4. The method for identifying mouse species based on image recognition according to claim 1, characterized in that, The feature aggregation and classification module includes three independent feature extraction sub-networks and a cross-attention fusion layer; The input terminals of the three feature extraction sub-networks are respectively connected to the three independent local image blocks of the head, the torso, and the tail, and output the head feature vector, the torso feature vector, and the tail feature vector; The cross-attention fusion layer uses the torso feature vector as a query matrix, concatenates the head feature vector and the tail feature vector as a key matrix and a value matrix, performs multi-head attention calculation, and outputs a fused feature vector. The fused feature vector and the trunk feature vector are added together by residual addition, and then mapped through a fully connected layer to generate the category discrimination vector.

5. The method for identifying the species of captured mice based on image recognition according to claim 1, characterized in that, The process of constructing and capturing scene images and preprocessing includes: simultaneously capturing the left-eye view and the right-eye view using a binocular camera deployed on the capture device; Epipolar correction and stereo matching are performed on the left-eye view and the right-eye view to generate a disparity map; Calculate the depth value of each pixel in the capture space based on the disparity map, and construct a depth mask map; The depth mask is used to filter background regions in the original mouse image whose depth values ​​are greater than a preset depth threshold, while retaining foreground connected regions whose depth values ​​are within the preset depth threshold range. The foreground connected regions are then cropped with bounding rectangles to generate the original mouse image that meets the input size requirements of the pose decoupling recognition network.

6. The method for identifying the species of captured mice based on image recognition according to claim 1, characterized in that, Before outputting the classification results, construct a category prototype vector library; The species prototype vector library stores standard feature vectors of multiple known mouse species. The standard feature vectors are obtained by inputting multiple standard pose images of multiple known mouse species into the pose decoupling recognition network and calculating the average value. Calculate the cosine similarity between the category discrimination vector and each of the standard feature vectors in the category prototype vector library; Arrange the cosine similarities in descending order, extract the two cosine similarities at the top of the list, and calculate the difference between the two cosine similarities at the top of the list. When the difference is less than the set difference threshold, a resampling instruction is triggered, and the construction and capture scene image acquisition and preprocessing process is re-executed.

7. The method for identifying the species of captured mice based on image recognition according to claim 2, characterized in that, The superimposing of the anatomical prior mask on the channel response map includes: obtaining the set of coordinates of the intersection points of the cage wires in the original rodent image, and connecting the set of intersection point coordinates to generate a set of grid line segments; Calculate the normal vector of each grid segment in the set of grid segments, and expand the grid segments outward at equal intervals along the direction of the normal vector to generate an expanded grid region; In the coordinate system corresponding to the channel response map, the pixel values ​​within the outer grid area are set to 0, while the pixel values ​​outside the outer grid area are retained to generate a filtered channel response map. The multi-layer heatmap prediction head extracts local maximum coordinates as the final heatmap coordinates based on the filtered channel response map using a non-maximum suppression algorithm.

8. The method for identifying the species of captured mice based on image recognition according to claim 3, characterized in that, The step of cropping a rectangular window according to a set fixed pixel side length includes: extracting all coordinate points on the central axis of the torso, calculating the Euclidean distance between adjacent coordinate points and summing them to obtain the torso length value; The torso length value is divided into three equal segments, with the endpoint closest to the head reference point as the center of the head window, the endpoint closest to the tail reference point as the center of the tail window, and the midpoint of the middle segment as the center of the torso window. Obtain the pre-configured head proportion coefficient, torso proportion coefficient, and tail proportion coefficient, and multiply the head proportion coefficient, torso proportion coefficient, and tail proportion coefficient by the torso length value respectively to calculate the head window side length, torso window side length, and tail window side length. The rectangular window capture operation is performed based on the center of the head window and its side length, the center of the torso window and its side length, and the center of the tail window and its side length.

9. The method for identifying the species of captured mice based on image recognition according to claim 4, characterized in that, The multi-head attention calculation includes: inputting the query matrix into a first linear mapping layer to generate a first mapping matrix, inputting the key matrix into a second linear mapping layer to generate a second mapping matrix, and inputting the value matrix into a third linear mapping layer to generate a third mapping matrix; Perform matrix multiplication on the first mapping matrix and the second mapping matrix, and divide the product by the scaling factor to obtain the attention weight matrix; Input the attention weight matrix into the sparse activation function, and set the elements in the attention weight matrix that are less than the activation threshold to 0; The attention weight matrix after sparsification is multiplied by the third mapping matrix, and the output is input into the fourth linear mapping layer to generate the fused feature vector.

10. The method for identifying the species of captured mice based on image recognition according to claim 5, characterized in that, The step of cropping the foreground connected region with an outer rectangle includes: obtaining the set of horizontal coordinates and the set of vertical coordinates of all pixels in the foreground connected region, extracting the maximum and minimum horizontal coordinate values ​​in the set of horizontal coordinates, and extracting the maximum and minimum vertical coordinate values ​​in the set of vertical coordinates. The initial bounding rectangle is generated by using the difference between the maximum and minimum horizontal coordinate values ​​as the initial width and the difference between the maximum and minimum vertical coordinate values ​​as the initial height. Calculate a first ratio of the target input width to the initial width of the attitude decoupling recognition network, and calculate a second ratio of the target input height to the initial height of the attitude decoupling recognition network; The minimum value between the first ratio and the second ratio is selected as the uniform scaling factor. The initial bounding rectangle is scaled proportionally according to the uniform scaling factor. The scaled initial bounding rectangle is then placed in the center of the blank canvas. Pixels that exceed the boundary of the blank canvas are filled with zero values.

Images