Mouse scratching behavior characteristic acquisition device and method

Abstract

The invention discloses a mouse scratching behavior characteristic acquisition device and method, belongs to the field of animal experiments, and aims to solve the problem that results are inaccurate due to the fact that mouse scratching behaviors are traditionally detected in a manual observation mode. The collecting device comprises a cover plate, a stand column, a bottom plate and a camera. The stand column is of a barrel structure with an upper opening and a lower opening, the upper end opening of the stand column is in threaded connection with the lower end opening of the cover plate, a transparent circular acrylic plate is arranged at the joint, and the transparent circular acrylic plate and the cover plate form a mouse containing space; a lower port of the stand column is in threaded connection with an upper port of the bottom plate, a camera used for collecting mouse scratching behavior characteristics is installed on the bottom plate, and a space defined by the stand column and the bottom plate serves as a camera observation adjusting space. And a mixed model of an ST-GCN model and a Transform model is adopted to predict the scratching behavior of the mouse.

Description

Technical Field

[0001] The present invention relates to a device and method for collecting mouse scratching behavior characteristics, belonging to the field of animal experiments. Background Art

[0002] As the most commonly used model animal in itching research and antipruritic drug development, the quantification of the scratching behavior of mice is crucial for evaluating the intensity of itching. Traditionally, laboratories have quantified the scratching behavior of mice by manual counting. This method not only consumes a large amount of manpower but also limits the ability of large-scale gene or drug screening.

[0003] Traditional itching research usually has obvious limitations. It requires researchers to watch and record videos for a long time, which is extremely labor-consuming. Different researchers may have different judgments on the scratching behavior in the same video, resulting in a lack of consistency in the results. It is difficult to accurately measure the specific characteristics of a single scratching behavior (such as intensity or duration). Summary of the Invention

[0004] Aiming at the problem of inaccurate results in the traditional manual observation method for detecting mouse scratching behavior, the present invention provides a device and method for collecting mouse scratching behavior characteristics. By introducing automated image recognition technology and deep learning algorithms, the efficiency is improved, the experimental period is shortened, the demand for human resources is reduced, and large-scale gene or drug screening becomes possible.

[0005] On the one hand, the present invention provides a device for collecting mouse scratching behavior characteristics, including a cover plate 1, a column 2, a bottom plate 3, and a camera;

[0006] The column 2 is a cylindrical structure with upper and lower openings. The upper port of the column 2 is threadedly connected to the lower opening of the cover plate 1, and a transparent circular acrylic plate 201 is provided at the connection, and the transparent circular acrylic plate 201 and the cover plate 1 construct a mouse accommodation space;

[0007] The lower port of the column 2 is threadedly connected to the upper port of the bottom plate 3. A camera for collecting mouse scratching behavior characteristics is installed on the bottom plate 3, and the space enclosed by the column 2 and the bottom plate 3 serves as a camera observation and adjustment space.

[0008] Preferably, the cover plate 1 includes a cover plate column body 102 and a top plate. A plurality of small round holes 101 are dispersedly arranged on the top plate to provide air and light for the mouse accommodation space; the lower opening of the cover plate column body 102 is provided with a cover plate and column connection groove 103 for cooperating with the connection groove at the upper port of the column 2 for threaded connection and fixing the transparent circular acrylic plate 201.

[0009] Preferably, the column 2 includes a column body 203, the upper and lower ends of the column body 203 are both provided with connecting grooves, and the upper end of the column body 203 has a notch as a semicircular light-transmitting opening 202 for providing air and light for the mouse accommodation space.

[0010] Preferably, the base plate 3 includes a base plate column body 304 and a bottom end plate, the upper surface of the bottom end plate is provided with a camera mounting groove 302, the camera mounting groove 302 is used to install the camera, and a rectangular wire inlet groove 303 is provided on the base plate column body 304, and the camera USB cable passes through the rectangular wire inlet groove 303 to connect to an external device; the upper end opening of the base plate column body 304 is provided with a base plate and column connecting groove 301, which is used to cooperate with the connecting groove of the lower port of the main column 2 for threaded connection.

[0011] In another aspect, the present invention provides a method for collecting characteristics of mouse scratching behavior. The method for collecting characteristics of mouse scratching behavior adopts a hybrid model of ST-GCN model + Transformer model for prediction. The hybrid model includes:

[0012] In the step of constructing the adjacency matrix A, the camera collects the mouse video, defines the head, back, limbs, and tail of the mouse in the video frame as nodes in the graph, extracts the position coordinates of each node, and then determines the connection relationship between the nodes based on the posture estimation, and constructs the adjacency matrix A;

[0013] The design steps of the spatiotemporal graph convolution layer are to use graph convolution operations to encode the spatial relationship between different nodes at the same time point; input the adjacency matrix A into the ST-GCN layer and output the node feature matrix H at the t+1th time step (t+1) ;

[0014] Time series processing step, node feature matrix H at the t+1 time step (t+1) Input to the Transformer encoder for timing processing;

[0015] The results of the ST-GCN layer and the Transformer encoder are fused and the final prediction result is obtained through the fully connected layer.

[0016] Preferably, the ST-GCN layer outputs the node feature matrix H at the t+1th time step (t+1) for:

[0017]

[0018] Among them, coal() is the ReLU activation function;

[0019] H (t) Represents the node feature matrix at the tth time step;

[0020] A is the adjacency matrix, Aij represents an element of the adjacency matrix A, where i and j represent nodes;

[0021] is the adjacency matrix after adding self-loops, represents the adjacency matrix after adding self-loops of the element, I N represents an N×N identity matrix;

[0022] is the degree matrix, is the degree matrix element;

[0023] W (GCN) is a trainable weight matrix.

[0024] Preferably, the Transformer encoder performs temporal processing using the multi-head attention mechanism, and outputs MultiHead(Q, K, V):

[0025] MultiHead(Q, K, V) = Concat(head 1 ,..., head h )W o

[0026] where the three matrices Q, K, and V are:

[0027] Q = H (t) W Q , K = H (t) W K , V = H (t) W V

[0028] W Q , W K , W V are trainable projection matrices;

[0029] head 1 ,..., head h represent the first to h multiple heads; the m-th head head m is obtained according to the following formula:

[0030]

[0031] is the projection matrix of the m-th head, m = 1, 2,..., h;

[0032] W o is the weight matrix of the output linear transformation, d k is the dimension of the key vector.

[0033] Preferably, a residual connection and layer normalization are added after each layer.

[0034] Preferably, an absolute position encoding is introduced to endow the model with the awareness of sequence positions, and its formula is as follows:

[0035] When it is even, the position is

[0036] When it is odd, the position is

[0037] where pos is the position index, and d model is the feature dimension of the model.

[0038] Preferably, the model is trained by combining self-supervised and weakly-supervised learning. In the pre-training stage, a large amount of unlabeled single-view video data is used for self-supervised learning to enhance the feature representation ability of the model; then, a small amount of labeled data is introduced for weakly-supervised learning, and the pseudo labels are used to further optimize the model performance, and the consistency regularization technology is applied to ensure the consistency of the model output.

[0039] Advantages of the present invention: The present invention provides a feature acquisition device for detecting mouse scratching behavior, aiming to significantly improve the research method of mouse scratching behavior through automated and intelligent means. Specifically, it includes the following advantages:

[0040] 1. Improve data accuracy and research efficiency: By introducing automated image recognition and deep learning algorithms, the subjective errors of manual counting are avoided, ensuring the objectivity and consistency of experimental data. The high-resolution camera enables long-term monitoring without affecting the natural behavior of mice, guaranteeing the authenticity and reliability of the data. The fully automated video recording and data analysis greatly reduce the workload of researchers, support large-scale gene or drug screening, and accelerate the experimental progress.

[0041] 2. Optimize the experimental environment and reduce animal stress: The device design simulates natural conditions, reduces the stress of mice, maintains their natural behavior patterns, and improves the effectiveness of observation results. The compact and reasonable structure provides a spacious and safe activity space, ensuring clear recording of mouse behavior.

[0042] 3. Advanced algorithm combination and improve model performance: Combining ST-GCN and Transformer enhances the understanding of temporal actions, and improves the accuracy and robustness of the model through self-supervised and weakly-supervised learning in the case of limited labeled data. The optimized model structure ensures efficient operation in resource-constrained environments, supports processing large-scale data sets, and has broad application prospects. Description of the Drawings

[0043] Figure 1It is a schematic three-dimensional structure diagram of a mouse scratching behavior characteristic acquisition device according to the present invention;

[0044] Figure 2 It is a front view of a mouse scratching behavior characteristic acquisition device according to the present invention;

[0045] Figure 3 It is a top view of a mouse scratching behavior characteristic acquisition device according to the present invention;

[0046] Figure 4 It is a side view of a mouse scratching behavior characteristic acquisition device according to the present invention;

[0047] Figure 5 It is a schematic three-dimensional structure diagram of the cover plate;

[0048] Figure 6 It is a front view of the cover plate;

[0049] Figure 7 It is a top view of the cover plate;

[0050] Figure 8 It is a schematic three-dimensional structure diagram of the column;

[0051] Figure 9 It is a front view of the column;

[0052] Figure 10 It is a top view of the column;

[0053] Figure 11 It is a side view of the column;

[0054] Figure 12 It is a schematic three-dimensional structure diagram of the bottom plate;

[0055] Figure 13 It is a front view of the bottom plate;

[0056] Figure 14 It is a top view of the bottom plate;

[0057] Figure 15 It is a side view of the bottom plate;

[0058] Figure 16 It is a schematic diagram of the principle of the mouse scratching behavior characteristic acquisition method according to the present invention.

[0059] In the figure: 1. Cover plate; 2. Column; 3. Bottom plate.

[0060] 101. Small round hole at the top of the cover plate; 102. Column body of the cover plate; 103. Connection groove between the cover plate and the column;

[0061] 201. Transparent circular acrylic plate; 202. Semi-circular light-transmitting opening; 203. Column body of the column;

[0062] 301. Bottom plate and column connection groove; 302. Camera placement groove; 303. Rectangular wire inlet groove; 304. Bottom plate column body. Detailed implementation mode

[0063] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0064] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0065] Next, the present invention will be further described in conjunction with the accompanying drawings and specific embodiments, but it is not limited to the present invention.

[0066] Detailed implementation mode one: Next, in conjunction with Figures 1 to 15 Describe this implementation mode. The mouse scratching behavior feature acquisition device described in this implementation mode includes a cover plate 1, a column 2, a bottom plate 3 and a camera; see Figures 1 to 4 .

[0067] The column 2 is a cylindrical structure with upper and lower openings. The upper port of the column 2 is threadedly connected to the lower opening of the cover plate 1, and a transparent circular acrylic plate 201 is provided at the connection, and the transparent circular acrylic plate 201 and the cover plate 1 construct a mouse accommodation space;

[0068] The lower port of the column 2 is threadedly connected to the upper port of the bottom plate 3. A camera for collecting mouse scratching behavior features is installed on the bottom plate 3. The space enclosed by the column 2 and the bottom plate 3 serves as a camera observation and adjustment space.

[0069] The cover plate 1 is firmly connected to the upper part of the column 2 through the connection groove below, and at the same time, the device bottom plate 3 is tightly connected to the lower part of the column 2 through the connection groove above, forming a stable overall structure. The mouse moves freely on the transparent circular acrylic plate 201, and the camera located on the bottom plate 3 can clearly capture the scratching behavior of the mouse. The device provides a natural and stress-free experimental environment, which is convenient for accurately detecting and analyzing its scratching behavior and obtaining accurate and reliable research data.

[0070] See Figures 5 to 7 , the cover plate 1 includes a cover plate column body 102 and a top plate. A plurality of small circular holes 101 are scattered on the top plate for providing air and light for the mouse accommodation space; the lower opening of the cover plate column body 102 is provided with a cover plate and column connection groove 103 for cooperating with the connection groove of the upper port of the column 2 for threaded connection and fixing the transparent circular acrylic plate 201.

[0071] There are five small round holes on the top of the cover plate, the main function of which is ventilation and lighting. These holes ensure air circulation in the living environment of the mice, maintain a suitable oxygen level, and allow an appropriate amount of natural light to enter, simulating a light environment close to actual living conditions, thereby further reducing the stress of the experimental animals. The connection groove below the cover plate 3 not only facilitates a firm connection with the top of the column 2, but also further fixes the transparent acrylic plate 201 to ensure the stability of the entire device.

[0072] See also Figures 8 to 11 The column 2 includes a column body 203, and the upper and lower ends of the column body 203 are both provided with connecting grooves. The upper end of the column body 203 has a notch as a semicircular light-transmitting opening 202 for providing air and light for the mouse accommodation space.

[0073] The column 2 is the supporting structure of the device. It is hollow inside and firmly connected to the bottom plate 3 at the bottom, which plays a role in stabilizing the entire device. A semicircular light-transmitting port 202 is provided at the top of the column 2, which provides necessary light for the inside of the device, simulates natural lighting conditions, and avoids the interference and pressure that direct light sources may cause to mice. This indirect lighting method helps to maintain the natural behavior patterns of mice, making the observation results more real and reliable.

[0074] Covering the top of the column 2 is a transparent circular acrylic plate 201, which not only provides a spacious activity space for the mice, but also ensures that the camera can clearly record the behavior of the mice. The choice of acrylic material takes into account both transparency and durability, and does not hinder the camera's line of sight or affect the mouse's visual perception, thereby ensuring the accuracy and authenticity of the behavioral data.

[0075] See also Figures 12 to 15 The bottom plate 3 includes a bottom plate column body 304 and a bottom end plate. A camera mounting groove 302 is provided on the upper surface of the bottom end plate. The camera mounting groove 302 is used to install the camera. A rectangular wire inlet groove 303 is provided on the bottom plate column body 304. The camera USB cable passes through the rectangular wire inlet groove 303 to connect to an external device; the upper end opening of the bottom plate column body 304 is provided with a bottom plate and column connecting groove 301, which is used to cooperate with the connecting groove of the lower port of the main column 2 for threaded connection.

[0076] The bottom plate 3 is provided with a rectangular wire inlet groove 303 to ensure that the connection of the camera is stable and the wires do not affect the experimental environment. The camera is connected to an external computer or data processing device via a USB cable. The bottom plate 3 is closely connected to the bottom of the column 2 through the upper connection groove to form a stable overall structure. The stability of the device is ensured, and the internal wiring is neat and orderly.

[0077] The camera is the core component of the device. It is installed on the bottom plate 3 with the lens facing upwards and is used to record the behavior of the mice. The camera can capture the subtle movements of the mice, especially scratching behavior, at high resolution, providing high-quality video materials for subsequent automated analysis.

[0078] Specific Embodiment 2: The following combines Figure 16 to illustrate this embodiment. The method for collecting mouse scratching behavior characteristics described in this embodiment is implemented using the device described in Embodiment 1. The method for collecting mouse scratching behavior characteristics uses a hybrid model of the ST-GCN model + Transformer model for prediction. Refer to Figure 16 The hybrid model includes:

[0079] Steps for constructing the adjacency matrix A: The camera captures the video of the mice. The head, back, limbs, and tail of the mice in the video frames are defined as nodes in the graph, and the position coordinates of each node are extracted. Then, the connection relationship between the nodes is determined according to pose estimation, and the adjacency matrix A is constructed;

[0080] Steps for designing the spatio-temporal graph convolutional layer: At the same time point, graph convolutional operations are used to encode the spatial relationships between different nodes; the adjacency matrix A is input into the ST-GCN layer, and the node feature matrix H at the t+1 time step is output (t+1) ;

[0081] Steps for temporal processing: The node feature matrix H at the t+1 time step (t+1) is input into the Transformer encoder for temporal processing;

[0082] The results of the ST-GCN layer and the Transformer encoder are fused, and the final prediction result is obtained through a fully connected layer.

[0083] Combining the spatial modeling ability of ST-GCN and the time series processing ability of Transformer enhances the understanding of temporal actions. Through this combination, the model can not only capture the spatial relationships between different nodes at the same time point but also effectively handle the dependencies and global context information over a long time span. The implementation steps are as follows:

[0084] 1. Construct the graph structure:

[0085] (1) Define the head, back, limbs, and tail of the mice as nodes in the graph, and use a high-resolution pose estimation model (HRNet) to extract the position coordinates of these key points from the video frames.

[0086] (2) Define the connection relationships between nodes according to anatomical knowledge and pose estimation results. The head is connected to the neck, the limbs are connected to the trunk, and the tail is connected to the tail root. These connection relationships reflect the spatial adjacency and functional relevance of various parts of the mouse body.

[0087] Construct an adjacency matrix A to represent the connection relationships between nodes. The adjacency matrix is a symmetric matrix, where the element A ij indicates whether there is an edge between node i and node j.

[0088] 2. Design of spatio-temporal graph convolutional layer:

[0089] At the same time point, use graph convolutional operations to encode the spatial relationships between different nodes. Graph convolution updates the feature representation of each node by aggregating the information of neighboring nodes. The ST-GCN layer can be expressed as:

[0090] where σ() is the ReLU activation function;

[0091]

[0092] H

[0093] represents the node feature matrix at the t-th time step; (t) A is the adjacency matrix, and A

[0094] represents the element of the adjacency matrix A, and i, j represent nodes; ij is the adjacency matrix after adding self-loops,

[0095] is the element of the adjacency matrix after adding self-loops, and I represents an N×N identity matrix; N is the degree matrix,

[0096] is the element of the degree matrix;

[0097]

[0098] W (GCN) is the trainable weight matrix.

[0098] 3. Transformer encoder:

[0099] For each time step t, first obtain the node feature H (t) through the ST-GCN layer, and then input it into the Transformer encoder.

[0100] The Transformer encoder performs temporal processing using the multi-head attention mechanism and outputs MultiHead(Q, K, V):

[0101] MultiHead(Q, K, V) = Concat(head 1 ,..., head h )W o

[0102] Generation of Q (query), K (key), and V (value):

[0103] Q = H (t) W Q , K = H (t) W K , V = H (t) W V

[0104] W Q , W K , W V are trainable projection matrices; H (t) is the input sequence.

[0105] head 1 ,..., head h represents the first to the h-th heads; the m-th head head m is obtained as follows:

[0106]

[0107] is the projection matrix of the m-th head, m = 1, 2,..., h;

[0108] W o is the weight matrix of the output linear transformation, d k is the dimension of the key vector.

[0109] 4. Residual connection and normalization:

[0110] To prevent the vanishing or exploding of gradients and to facilitate the training of deeper networks, a residual connection and layer normalization are added after each layer to ensure the stability and consistency of the information flow. The formula is as follows:

[0111] X out = LayerNorm(X in + Sublayer(X in ))

[0112] where X in and X out are the input and output features respectively, and Sublater represents a sublayer.

[0113] 5. Positional encoding:

[0114] Since the Transformer itself does not have the ability to understand sequential information, absolute position encoding is introduced to endow the model with the awareness of sequence positions. The formula is as follows:

[0115] When it is even, the position is

[0116] When it is odd, the position is

[0117] where pos is the position index and d model is the feature dimension of the model.

[0118] 6. Fusion and Optimization

[0119] Fuse the results of the ST-GCN layer and the Transformer encoder, and obtain the final prediction result through a fully connected layer. At the same time, use an appropriate loss function (cross-entropy loss) and optimization algorithm (Adam) for model training.

[0120] This embodiment uses a combination of self-supervised and weakly supervised methods to train the model. In the pre-training stage, a large amount of unlabeled single-view video data is used for self-supervised learning to enhance the feature representation ability of the model; then, a small amount of labeled data is introduced for weakly supervised learning, and pseudo-labels are used to further optimize the model performance, and consistency regularization technology is applied to ensure the consistency of the model output. This can significantly improve the accuracy and robustness of the mouse scratching behavior detection model under the condition of limited labeled data, providing strong support for itching research and antipruritic drug development. The implementation steps are as follows:

[0121] 1. Self-supervised Pre-training:

[0122] (1) Design contrastive learning tasks for inter-frame prediction, occluded region prediction, and tasks based on image transformation (rotation, scaling, cropping). For each frame, generate multiple enhanced versions, and use different enhanced versions of the same frame as positive sample pairs, and enhanced versions between different frames as negative sample pairs. Use the contrastive loss function (InfoNCE Loss) to maximize the similarity between positive sample pairs and minimize the similarity between negative sample pairs.

[0123] (2) Design frame prediction tasks to predict the content of the next frame or several future frames, and train the model to capture temporal continuity. Randomly occlude a part of the image, and train the model to predict the content of the occluded region to enhance the understanding of local and global information. Perform simple transformations on the image such as rotation, scaling, and cropping, and train the model to recognize the relationship between the transformed image and the original image.

[0124] 2. Weakly Supervised Fine-tuning:

[0125] (1) Collect and label a small amount of high-quality video data of mouse scratching behavior, ensuring that the behavior patterns under various conditions are covered.

[0126] (2) Use a pre-trained model to infer unlabeled data and generate pseudo-labels with high confidence. Set a threshold, and only accept it as a pseudo-label when the model prediction probability exceeds this threshold.

[0127] (3) Apply the consistency regularization technique (Mean Teacher) to make the model output consistent results under different data augmentation conditions. Use the mixed sample technique to linearly combine the labeled data and pseudo-labeled data to produce new training samples.

[0128] The main steps of the Mean Teacher model are as follows:

[0129] ① Use two models: a student model and a teacher model. The weights of the teacher model are the exponential moving average of the weights of the student model and do not participate in backpropagation.

[0130] ② In each iteration, the student model receives the augmented input, while the teacher model receives the non-augmented input. Calculate the output difference between the two and add it as the consistency loss to the total loss.

[0131] Combine the cross-entropy loss (for labeled data) and the consistency loss (for pseudo-labeled data) to form a comprehensive loss function to guide the update of model parameters.

[0132] Although the present invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed, as long as they do not depart from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the different dependent claims and the features described herein can be combined in a manner different from that described in the original claims. It should also be understood that the features described in connection with a single embodiment can be used in other described embodiments.

Claims

1. A device for collecting characteristics of mouse scratching behavior, characterized in that: It comprises a cover plate (1), a column (2), a base plate (3) and a camera; The column (2) is a cylindrical structure with upper and lower openings. The upper end of the column (2) is threadedly connected to the lower end opening of the cover plate (1), and a transparent circular acrylic plate (201) is provided at the connection. The transparent circular acrylic plate (201) and the cover plate (1) form a mouse accommodation space. The lower end of the column (2) is threadedly connected to the upper end of the bottom plate (3); a camera for collecting scratching behavior characteristics of mice is installed on the bottom plate (3); and the space enclosed by the column (2) and the bottom plate (3) serves as a camera observation and adjustment space.

2. A mouse scratching behavior feature acquisition device according to claim 1, characterized in that: The cover plate (1) comprises a cover plate column body (102) and a top plate, wherein a plurality of small circular holes (101) are dispersedly arranged on the top plate, and are used to provide air and light for the mouse accommodation space; a cover plate and column connection groove (103) is arranged at the lower end opening of the cover plate column body (102), and is used to cooperate with the connection groove of the upper end of the column (2) to perform threaded connection and fix the transparent circular acrylic plate (201).

3. A mouse scratching behavior feature acquisition device according to claim 1, characterized in that: The column (2) comprises a column body (203), the upper and lower ends of the column body (203) are both provided with connection grooves, and the upper end of the column body (203) has a notch as a semicircular light-transmitting opening (202) for providing air and light for the mouse accommodation space.

4. A mouse scratching behavior feature acquisition device according to claim 1, characterized in that: The bottom plate (3) comprises a bottom plate column body (304) and a bottom end plate. The upper surface of the bottom end plate is provided with a camera placement groove (302), and the camera placement groove (302) is used to install a camera. The bottom plate column body (304) is provided with a rectangular wire entry groove (303), and the camera USB cable passes through the rectangular wire entry groove (303) to connect to an external device. The upper end opening of the bottom plate column body (304) is provided with a bottom plate and column connection groove (301), which is used to cooperate with the connection groove of the lower port of the main column (2) for threaded connection.

5. A method for collecting characteristics of mouse scratching behavior, the method is implemented based on the device for collecting characteristics of mouse scratching behavior according to any one of claims 1 to 4, characterized in that: The method for collecting mouse scratching behavior characteristics uses a hybrid model of the ST-GCN model + Transformer model for prediction. The hybrid model includes: In the step of constructing the adjacency matrix A, the camera collects the mouse video, defines the head, back, limbs, and tail of the mouse in the video frame as nodes in the graph, extracts the position coordinates of each node, and then determines the connection relationship between the nodes based on the posture estimation, and constructs the adjacency matrix A; The design steps of the spatiotemporal graph convolution layer are to use graph convolution operations to encode the spatial relationship between different nodes at the same time point; input the adjacency matrix A into the ST-GCN layer and output the node feature matrix H at the t+1th time step (t+1) ; Time series processing step, node feature matrix H at the t+1 time step (t+1) Input to the Transformer encoder for timing processing; The results of the ST-GCN layer and the Transformer encoder are fused and the final prediction result is obtained through the fully connected layer.

6. A method for collecting mouse scratching behavior characteristics according to claim 5, characterized in that: The ST-GCN layer outputs the node feature matrix H at the t+1th time step (t+1) for: Among them, μ() is the ReLU activation function; H (t) Represents the node feature matrix at the tth time step; A is the adjacency matrix, A ij represents the elements of the adjacency matrix A, i, j represent nodes; is the adjacency matrix after adding the self-loop, Represents the adjacency matrix after adding the self-loop Elements, I N Represents an N×N identity matrix; is the degree matrix, is the degree matrix element; W (GCN) is a trainable weight matrix.

7. A method for collecting mouse scratching behavior characteristics according to claim 5, characterized in that: The Transformer encoder uses a multi-head attention mechanism for time series processing and outputs MultiHead(Q, K, V): MultiHead(Q,K,V)=Concat(head1,...,head h )W o Among them, the three matrices Q, K, and V are: Q=H (t) W Q ,K=H (t) W K ,V=H (t) W V W Q , W K , W V is a trainable projection matrix; head1,...,head h Indicates the 1st to hth heads; the mth head m Get it by pressing: is the projection matrix of the mth head, m = 1, 2, …, h; W o is the weight matrix of the output linear transformation, d k is the dimension of the key vector.

8. A method for collecting mouse scratching behavior characteristics according to claim 5, characterized in that: Add residual connections and layer normalization after each layer.

9. A method for collecting mouse scratching behavior characteristics according to claim 6, characterized in that: Absolute position encoding is introduced to give the model awareness of sequence positions. The formula is as follows: When even, the position is When odd, the position is Where pos is the position index, d model is the characteristic dimension of the model.

10. A method for collecting mouse scratching behavior characteristics according to claim 5, characterized in that: The model is trained by combining self-supervision and weak supervision. In the pre-training stage, a large amount of unlabeled single-view video data is used for self-supervised learning to enhance the model's feature representation ability. Then, a small amount of labeled data is introduced for weakly supervised learning, pseudo labels are used to further optimize the model performance, and consistency regularization techniques are applied to ensure the consistency of the model output.

Images