Mouse scratching behavior detection method based on machine vision

Abstract

The invention provides a mouse scratching behavior detection method based on machine vision, which belongs to the technical field of image processing and biological behavior recognition, and comprises the following steps: establishing a required shooting environment, and recording mouse behaviors under different conditions such as illumination, darker light, coexistence of two mice and the like at the angle of high-angle shooting of a camera. The system firstly carries out gray processing operation on an image, then obtains a segmentation threshold value through an Otsu algorithm, respectively carries out binary segmentation twice, carries out small object removal, morphological dilation, corrosion and other operations after extracting mouse claw and ear contours through image subtraction operation, so as to reduce other influences except for a required target to the greatest extent. And calculating the area size of the foreground corresponding to the mouse, the coordinates of the central point and the distance between the claw and the ear, and comparing a set corresponding distance threshold value with the calculated distance to judge whether the mouse has a scratching action or not. The method can effectively improve the accuracy of mouse scratching behavior detection and save a large amount of labor cost.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and biological behavior recognition, relates to a behavior detection method, and specifically to a mouse scratching behavior detection method based on machine vision. Background Art

[0002] Experimental animal behavior analysis is an important technical means for studying the functions of higher neural centers. It is not only widely used in traditional fields such as neuropsychology and neuropharmacology, but also extends to emerging cross-disciplinary research fields such as artificial intelligence, bionics, and brain-computer interfaces.

[0003] Animal behavior is composed of body posture and changes in body posture. Kinematic parameters are only a rough description of animal behavior. Animal posture contains more psychological information and is more important in behavioral analysis.

[0004] Behavioral research is of great value in fields such as neuroscience and pharmacology, and mice are one of the most commonly used experimental animals. Understanding the scratching behavior of mice can help researchers assess the functional state of their nervous system and the effects of different conditions, drugs, or treatments. However, the traditional method of evaluating mouse scratching behavior requires manual observation of mice spontaneously scratching. Different observers may have different interpretations of the same behavior, thus affecting the reliability and repeatability of the results. In addition, the traditional manual observation and recording of mouse scratching behavior requires a lot of time and human resources, which not only increases the cost of research, but also limits the scale and sustainability of experiments. Summary of the invention

[0005] In order to solve the problems that the current traditional manual observation of mouse scratching action is subjective, complicated to operate, and consumes a lot of time and human resources, the present invention provides a mouse scratching behavior detection method based on machine vision, which can detect mouse scratching behavior through automation technology.

[0006] The technical solution of a method for detecting mouse scratching behavior based on machine vision of the present invention is:

[0007] A method for detecting mouse scratching behavior based on machine vision comprises the following steps:

[0008] S1, video image acquisition of mice;

[0009] S2, grayscale processing is performed on the video frame image acquired by S1;

[0010] S3, using the Otsu algorithm to obtain a segmentation threshold with obvious distinction between mouse ears and paws;

[0011] S4, performing two binary segmentations, and using image subtraction operation to subtract the two results to obtain a result image highlighting the contours of the mouse ears and paws;

[0012] S5, performing operations such as small object removal, morphological dilation and corrosion on the subtraction result image and obtaining the mouse ears and paws required for detection;

[0013] S6, calculating the area size of the foreground corresponding to the mouse and the coordinates of the center point;

[0014] S7, using a bubble sort method, setting different ascending and descending methods according to the ear segmentation conditions of different videos; when the mouse ear is obviously large in the segmented image, a descending method is used, on the contrary, when the mouse ear is obviously small, an ascending method is used, and finally the mouse ear is used as the first index to calculate the Euclidean distance between the paw and the ear;

[0015] S8. Set a distance threshold and use it as a condition to determine whether the mouse has a scratching action; if the calculated distance is less than the threshold, the mouse has a scratching action, and the mouse's paw is marked with a box; if the calculated distance is greater than the threshold, the mouse has no scratching action, and the mouse is not marked;

[0016] S9. The above method was applied to the detection of scratching behavior of mice under different conditions: a single mouse under artificial lighting, a single mouse under dim lighting, and two mice under sunlight, to verify the feasibility of the method.

[0017] A further improvement of the technical solution of the present invention is that: the step S1 includes the following steps:

[0018] A camera was used to capture video images. The camera was installed just above the open field test box. Its viewing angle was required to cover the entire open field. A lighting lamp with a light intensity of 200 lx was installed above the test box to provide lighting for a single mouse under artificial lighting conditions. The frame rate of the captured video images was 30 frames per second.

[0019] A further improvement of the technical solution of the present invention is that: the step S3 includes the following steps:

[0020] S3.1, calculate the grayscale histogram of the image;

[0021] S3.2, iterate each possible threshold value to classify the image into two categories: background and foreground;

[0022] S3.3. For each threshold, calculate the inter-class variance of the background and foreground categories, that is, the degree of difference between the two categories, and select the threshold that maximizes the inter-class variance as the optimal threshold;

[0023] S3.4, applying the optimal threshold to the image for binarization processing, setting pixels less than or equal to the threshold as the background, and pixels greater than the threshold as the foreground, thereby determining a binarization threshold suitable for the image, so that the target object of the image is separated from the background more clearly;

[0024] A further improvement of the technical solution of the present invention is that: the step S4 includes the following steps:

[0025] For the image image, it is regarded as an M×N matrix, that is, the pixels in the image, each value is the pixel value, where the pixel value is between 0 and 255, the segmentation threshold of the foreground and background is recorded as threshold, the proportion of the number of pixels belonging to the foreground in the whole image is recorded as ω0, and the average grayscale of the foreground is recorded as μ0; the proportion of the number of background pixels in the whole image is recorded as ω1, and the average grayscale of the background is recorded as μ1. The total average grayscale of the image is recorded as μ, and the inter-class variance is recorded as σ; assuming that the background of the image is darker and the size of the image is M×N, the number of pixels in the image whose grayscale value is less than the threshold threshold is recorded as N0, and the number of pixels whose grayscale is greater than or equal to the threshold threshold is recorded as N1, then:

[0026]

[0027] N0+N1=M×N (3)

[0028] 1=ω0+ω1 (4)

[0029] μ=ω0×μ0+ω1×μ1 (5)

[0030] σ=ω0×(μ0-μ) 2 +ω1×(μ1-μ) 2 (6).

[0031] A further improvement of the technical solution of the present invention is that the principle of the image subtraction operation in step S4 is to subtract corresponding pixels. Assuming that there are two images A and B, and their corresponding pixels are A(i,j) and B(i,j) respectively, then when performing image subtraction operation, a new image C(i,j)=A(i,j)-B(i,j) can be obtained, and each pixel value in image C is the difference between the pixel values ​​at the corresponding positions of images A and B.

[0032] A further improvement of the technical solution of the present invention is that: the principle of morphological dilation and erosion in step S5 is to convolve the image or a part of the image with the structural element; wherein the image or a part of the image is represented by image A, and the structural element is represented by a convolution kernel B, and the convolution kernel can be of any shape and size, and has a separately defined reference point, which is called an anchor point;

[0033] Dilation is an operation to find the local maximum value. The convolution kernel B is convolved with the image, that is, the maximum value of the pixel points in the area covered by the convolution kernel B is calculated, and this maximum value is assigned to the pixel specified by the reference point, so that the highlight area in the image gradually grows. The formula can be expressed as:

[0034]

[0035] Erosion is to remove the burrs and details of the image. Erosion can be used to eliminate noise and segment independent image elements. It is essentially a spatial filter. A mask is set, and the center of the mask slides over each pixel point one by one. The value of the current pixel point is set to the minimum value of the pixel in the mask coverage area. The formula can be expressed as:

[0036]

[0037] The operation of dilating and then corroding an image is called a closing operation, which is used to fill small holes in the foreground object or erase small black spots on the foreground object. First, the white part is enlarged, the small black part is squeezed out, and then some large black parts are restored. The overall effect is to erase the small black spots on the foreground object. The formula can be expressed as:

[0038]

[0039] The operation of first corroding and then dilating an image is called opening operation, which is used to eliminate small objects, separate objects at thin points, and smooth the boundaries of larger objects without significantly changing their areas. The formula can be expressed as:

[0040]

[0041] A further improvement of the technical solution of the present invention is that: the small object removal in step S5 is to remove the connected areas smaller than the specified area in the binary image;

[0042] First, the connected regions of the input binary image are labeled, and each connected region in the image is assigned a unique label;

[0043] Then, the number of pixels contained in each connected region is calculated, that is, the area of ​​the region;

[0044] Traverse all connected areas and set the pixel values ​​of areas smaller than the specified threshold (small connected areas) to 0, thereby removing these small connected areas;

[0045] Finally, the filtered pixels are reassembled into a new binary image, where small connected regions are removed.

[0046] A further improvement of the technical solution of the present invention is that the principle of the bubble sort in step S7 is that, assuming that a known unordered sequence is to be arranged from small to large, the first element is compared with the following elements in sequence. If the following element is smaller, it is exchanged with a certain element behind it, and then this element is used to compare in sequence. If it is smaller than this element, it is exchanged with this element, until the last element, a bubble sequence is completed, and the last element must be the largest element in this sequence.

[0047] A further improvement of the technical solution of the present invention is that the principle of calculating the Euclidean distance in step S7 is that the Euclidean distance is the real distance between two points in n-dimensional space, and the Euclidean distance in two-dimensional and three-dimensional space is the distance between two points. The calculation of the Euclidean distance is to first calculate the difference in each dimension, then square these interpolation values, then add all the square differences, and take the square root of the result. The mathematical model formula is as follows:

[0048]

[0049] Where d(A,B) represents the distance between point A and point B, (a1-b1) 2 +(a2-b2) 2 +...+(a n -b n ) 2 Represents the sum of the squared distances between two points.

[0050] A further improvement of the technical solution of the present invention is that: the step S9 applies the above method to different situations of artificial lighting of a single mouse, dim light of a single mouse, and sunlight exposure of two mice.

[0051] For a single mouse, artificial lighting was provided by lighting from above the open field test box, and the light intensity of the entire lighting area was selected to be 200 lx. Black mice were selected. For a single mouse, the lighting above the test box was turned off when the light was dim, and the sunlight was blocked to a certain extent. Black mice were selected. For two mice, the two mice were placed in the open field test box at the same time, and the lighting above was turned off, leaving only normal sunlight exposure. Both mice were selected to be white mice.

[0052] Due to the adoption of the above technical solution, the technical advances achieved by the present invention include:

[0053] The present invention adopts the environment construction method of the open field experiment, and the camera is installed on the top to continuously shoot the mouse from a bird's-eye view angle, which can avoid the influence of surrounding objects, people and noise on the mouse, and can also avoid the influence of the mouse's activities around the camera on the shooting effect; in the scratching behavior recognition method, the distance between the claws and ears that are obviously different in color from the mouse's body is used as the judgment condition, which avoids the inaccurate recognition of the mouse's scratching behavior due to the influence of the segmentation effect, and improves the accuracy of scratching behavior recognition.

[0054] At the same time, the present invention applies automated equipment to the study of mouse behavior, and can monitor the behavior of mice through technologies such as cameras, thereby reducing human intervention and subjective errors to improve evaluation efficiency and accuracy. In addition, the application of image processing and data analysis technology makes the quantitative analysis of mouse behavior more feasible. By performing image processing and data analysis on the collected mouse videos, various behavioral parameters can be extracted, thereby more accurately evaluating the behavioral characteristics of mice. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of a method for detecting mouse scratching behavior based on machine vision of the present invention;

[0056] Figure 2 It is a schematic diagram of the environment construction and camera installation of a method for detecting mouse scratching behavior based on machine vision of the present invention;

[0057] Figure 3 It is an image of a single mouse when the mouse does not have scratching behavior under artificial lighting conditions in a method for detecting mouse scratching behavior based on machine vision of the present invention;

[0058] Figure 4 The present invention is a method for detecting mouse scratching behavior based on machine vision, which is an image of a single mouse under artificial lighting, and the scratching behavior of the mouse is annotated after image processing;

[0059] Figure 5 It is an image of a single mouse in a method for detecting scratching behavior of mice based on machine vision in the present invention when the mouse does not have scratching behavior under dim light conditions;

[0060] Figure 6 The present invention is a method for detecting mouse scratching behavior based on machine vision, in which an image of a single mouse in dim light is processed and the scratching behavior of the mouse is annotated;

[0061] Figure 7 The present invention is a method for detecting scratching behavior of mice based on machine vision, which is an image of two mice when the mice do not have scratching behavior under sunlight exposure;

[0062] Figure 8 The present invention discloses a method for detecting mouse scratching behavior based on machine vision, in which images of two mice under sunlight irradiation are processed and the scratching behavior of the mice is annotated. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in combination with specific implementation methods and with reference to the accompanying drawings. In the following description, the description of the public structure and technology is omitted to avoid unnecessary confusion of the concept of the present invention.

[0064] like Figure 1 As shown, the present invention provides a method for detecting mouse scratching behavior based on machine vision, which mainly determines whether there is scratching behavior by detecting and analyzing the distance between the mouse's paw and ear. In the environment set up by this method, since the mouse is free to move without other interference, in order to prevent the mouse's activities from affecting the shooting effect, the camera should be installed directly above for overhead shooting, and the specific situation is as follows:

[0065] S1. Collect video images of mice.

[0066] Specifically, a camera is used to collect video images. The camera is installed above the open field experimental environment and shoots from a bird's-eye view, ensuring that the mice and the overall environment are within the shooting range. The installation position is shown in Figure 2 .

[0067] S2. grayscale the video frame image acquired in S1.

[0068] Specifically, image grayscale processing is the process of converting a color image into a grayscale image. Through this process, the complexity of the image can be reduced, which is convenient for image analysis and processing. In image grayscale processing, the RGB color image will be converted into a grayscale image, that is, each pixel only retains one grayscale value, which represents the brightness of the pixel. Image grayscale processing is based on the sensitivity of the human eye to color and brightness. The human eye perceives brightness much higher than color, so in many cases, only using the brightness information of the image is sufficient to convey most of the information of the image. In grayscale processing, commonly used algorithms include: weighted average method, maximum value method, average value method, etc.

[0069] The weighted average method adds the pixel values ​​of the three channels R, G, and B in the RGB color image according to certain weights to obtain the grayscale value. The weights usually used are determined based on the human eye's perception of different colors. Generally, the weight of red is 0.299, the weight of green is 0.587, and the weight of blue is 0.114. After weighted averaging, each pixel can be represented by a grayscale value.

[0070] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j) (12)

[0071] The maximum value method is based on the brightness represented by the maximum value in the color component. In a color image, each pixel is composed of three components: red (R), green (G), and blue (B), and the maximum value method selects the maximum value of these three components to represent the brightness of the pixel. Because in natural scenes, brightness is usually determined by the brightest color component, the maximum value method can reflect the brightness characteristics of the image to a certain extent. When using the maximum value method for grayscale processing, for each pixel, the largest value is selected from its RGB components as the grayscale value to obtain a grayscale image. This can quickly convert color images into grayscale images, reducing the amount of calculation and complexity.

[0072] R=G=B=max(R,G,B) (13)

[0073] The average value method is based on the average brightness of the color components. When using the average value method for grayscale processing, for each pixel, the pixel values ​​of its R, G, and B components are added together, and then divided by 3 to get the average value as the grayscale value of the pixel, and finally a grayscale image is obtained.

[0074] R=G=B=(R+G+B) / 3 (14).

[0075] S3. Obtain a segmentation threshold with obvious distinction between mouse ears and paws through the Otsu algorithm.

[0076] Specifically, the optimal threshold is selected based on maximizing the inter-class variance. First, the grayscale histogram of the image is calculated, and then each possible threshold is iterated to divide the image into two categories: background and foreground. For each threshold, the inter-class variance of the two categories is calculated, that is, the degree of difference between the two categories. The threshold that maximizes the inter-class variance is selected as the optimal threshold. Finally, the optimal threshold is applied to the image for binarization, and pixels less than or equal to the threshold are set as the background, and pixels greater than the threshold are set as the foreground. In this way, the binarization threshold suitable for the image can be automatically determined, so that the target object of the image is separated from the background more clearly.

[0077] S4. Perform two binary segmentations, and use image subtraction to subtract the two results to obtain a contour result image that highlights the mouse ears and paws.

[0078] Specifically, taking the grayscale image as an example, for the image image, we can regard it as an M×N matrix, that is, the pixels in the image, each value is the pixel value, where the pixel value is between 0 and 255. The segmentation threshold of the foreground and background is denoted as threshold, the proportion of the number of pixels belonging to the foreground in the entire image is denoted as ω0, and the average grayscale of the foreground is denoted as μ0; the proportion of the number of background pixels in the entire image is denoted as ω1, and the average grayscale of the background is denoted as μ1. The total average grayscale of the image is denoted as μ, and the inter-class variance is denoted as σ. Assuming that the background of the image is darker and the size of the image is M×N, the number of pixels in the image whose grayscale value is less than the threshold threshold is denoted as N0, and the number of pixels whose grayscale is greater than or equal to the threshold threshold is denoted as N1, then:

[0079]

[0080] N0+N1=M×N (3)

[0081] 1=ω0+ω1 (4)

[0082] μ=ω0×μ0+ω1×μ1 (5)

[0083] σ=ω0×(μ0-μ) 2 +ω1×(μ1-μ) 2 (6)

[0084] Traverse all the values ​​of the threshold threshold, and calculate the inter-class variance σ for each threshold. The threshold that maximizes the inter-class variance is the final selected binarization threshold. Then, traverse each pixel of the image and compare its grayscale value with the threshold. If the grayscale value of the pixel is greater than the threshold, it is set to white (255), otherwise it is set to black (0). In this way, after the binarization process, each pixel in the image can only take one of the two values, thereby realizing the binarization of the image.

[0085] At the same time, the principle of the above image subtraction operation is to subtract corresponding pixels. Suppose there are two images A and B, and their corresponding pixels are A(i,j) and B(i,j) respectively. Then when performing image subtraction operation, a new image C(i,j)=A(i,j)-B(i,j) can be obtained. Each pixel value in image C is the difference between the pixel values ​​at the corresponding positions of images A and B.

[0086] S5. Perform operations such as small object removal, morphological dilation and corrosion on the subtraction result image to obtain the mouse ears and paws required for detection.

[0087] The above-mentioned small object removal is to remove the connected areas smaller than the specified area in the binary image. First, the connected areas of the input binary image are marked, and each connected area in the image is assigned a unique label. Then, the number of pixels contained in each connected area, that is, the area of ​​the area, is calculated. All connected areas are traversed, and the pixel values ​​of areas with an area smaller than the specified threshold (small connected areas) are set to 0, thereby removing these small connected areas. Finally, the filtered pixels are recombined into a new binary image, in which the small connected areas are removed.

[0088] The above dilation or erosion operation is to convolve the image (or a part of the image, we call it image A) with the structural element (we call it the convolution kernel B). The convolution kernel can be of any shape and size, with a single defined reference point, which we call the anchor point. In most cases, the convolution kernel is a small square or disk with a reference point and a solid center. The convolution kernel can be regarded as a template or mask.

[0089] Dilation is the operation of finding the local maximum value. The convolution kernel B is convolved with the image, that is, the maximum value of the pixels in the area covered by the convolution kernel B is calculated, and this maximum value is assigned to the pixel specified by the reference point. This will gradually increase the highlight area in the image. The formula can be expressed as:

[0090]

[0091] Erosion is to remove some burrs and details of the image. Erosion can generally be used to eliminate noise, segment independent image elements, etc. It is essentially a spatial filter. A mask is set, and the center of the mask slides over each pixel point one by one. The value of the current pixel point (that is, the position corresponding to the center of the mask) is set to the minimum value of the pixels in the mask coverage area. The formula can be expressed as:

[0092]

[0093] The operation of dilating and then corroding an image is called a closing operation. It is usually used to fill small holes in foreground objects or erase small black spots on foreground objects. You can imagine that the white part is enlarged first, the small black part is squeezed out, and then some large black parts are restored. The overall effect is to erase the small black spots on the foreground object. The formula can be expressed as:

[0094]

[0095] The operation of first corroding and then dilating an image is called opening. It is used to eliminate small objects, separate objects at thin points, and smooth the boundaries of larger objects without significantly changing their area. The formula can be expressed as:

[0096]

[0097] S6. Calculate the area size of the foreground corresponding to the mouse and the coordinates of the center point.

[0098] Specifically, to calculate the foreground area, first, label the connected regions of the input binary image and assign a unique label to each connected region in the image. Count the pixels of each connected region and count the number of all pixels in the region. These pixels all belong to the same object, so they constitute the area of ​​the object. Then, convert the number of pixels into area units. For binary images, each pixel represents a unit area, so the number of pixels is used as the area of ​​the object.

[0099] The coordinates of the foreground center point are similar to the calculation of the area. For each connected area, all pixels are traversed and the row and column coordinates of each pixel are calculated. The centroid is the weighted average of all pixel positions of the object, and its formula is:

[0100]

[0101] S7. Use the bubble sort method to set different ascending and descending methods according to the ear segmentation conditions of different videos; when the mouse ears are obviously large in the segmented image, use the descending method; on the contrary, when the mouse ears are obviously small, use the ascending method. Finally, use the mouse ears as the first index to calculate the Euclidean distance between the paw and the ear.

[0102] The principle of the above bubble sort is that, assuming that a known unordered sequence is to be arranged from small to large, the first element is compared with the following elements in sequence. If the following element is smaller, it is swapped with a certain element behind it, and then this element is used to compare in sequence. If it is smaller than this element, it is swapped with this element until the last element. A trip of the bubble sequence is completed, and the last element must be the largest element in this sequence.

[0103] At the same time, the principle of the above Euler distance calculation is that the Euclidean distance is the real distance between two points in n-dimensional space, and the Euclidean distance in two-dimensional and three-dimensional space is the distance between two points. The calculation of the Euclidean distance is to first calculate the difference in each dimension, then square these interpolations, then add all the square differences, and take the square root of the result. The mathematical model formula is as follows:

[0104]

[0105] Where d(A,B) represents the distance between point A and point B, (a1-b1) 2 +(a2-b2) 2 +...+(a n -b n )2 Represents the sum of the squared distances between two points.

[0106] S8. Set a distance threshold and use it as a condition to determine whether the mouse has a scratching action; if the calculated distance is less than the threshold, the mouse has a scratching action, and the mouse's paw is marked with a box; if the calculated distance is greater than the threshold, the mouse has no scratching action, and the mouse is not marked;

[0107] S9. The above method was applied to the detection of scratching behavior of mice under different conditions: a single mouse under artificial lighting, a single mouse under dim lighting, and two mice under sunlight, to verify the feasibility of the method.

[0108] The present invention is respectively applied to different situations such as artificial lighting of a single mouse, dim light of a single mouse, and sunlight exposure of two mice: in the case of artificial lighting of a single mouse, the lighting is from above the open field test box, the light intensity of the entire lighting area is selected to be 200lx, and black mice are selected. In the case of dim light of a single mouse, the lighting above the test box is turned off, and the sunlight is shielded to a certain extent, and black mice are selected. In the case of sunlight exposure of two mice, two mice are placed in the open field test box at the same time, the lighting above is turned off, and only sunlight exposure under normal conditions is retained, and white mice are selected for both mice.

[0109] The original image and image processing annotation of mouse scratching detection under artificial lighting of a single mouse are as follows Figure 3 , Figure 4 The original image and image processing annotation of a single mouse scratching detection in dim light are shown in Figure 1. Figure 5 , Figure 6 The original image and annotation of the mouse scratching test of two mice under sunlight exposure are shown in Figure 7 , Figure 8 As shown above. Figure 3 , 4 As shown in , 5, 6, 7, and 8, the present invention can automatically mark the mouse when it exhibits scratching behavior.

[0110] In the above embodiment, the present invention provides a method for detecting mouse scratching behavior based on machine vision. The environment construction method of the open field experiment is adopted, and the camera is installed on the top to continuously shoot the mouse from a bird's-eye view. This can not only avoid the influence of surrounding objects, people and noise on the mouse, but also avoid the activities of the mouse around the camera affecting the shooting effect; in the scratching behavior identification method, the distance between the claws and ears that have obvious color differences from the mouse body is used as the judgment condition, which avoids the inaccurate identification of the mouse scratching behavior due to the influence of the segmentation effect, thereby improving the accuracy of scratching behavior identification.

[0111] The above-described embodiments are merely descriptions of preferred implementations of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention, and the technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. A method for detecting mouse scratching behavior based on machine vision, characterized in that: The following steps are included: S1, video image acquisition of mice; S2, grayscale processing is performed on the video frame image acquired by S1; S3, using the Otsu algorithm to obtain a segmentation threshold with obvious distinction between mouse ears and paws; S4, performing two binary segmentations, and using image subtraction operation to subtract the two results to obtain a result image highlighting the contours of the mouse ears and paws; S5, performing small object removal, morphological dilation or corrosion operations on the subtraction result image and obtaining the mouse ears and paws required for detection; S6, calculating the area size of the foreground corresponding to the mouse and the coordinates of the center point; S7, using a bubble sort method, setting different ascending and descending methods according to the ear segmentation conditions of different videos; when the mouse ear is obviously large in the segmented image, a descending method is used, on the contrary, when the mouse ear is obviously small, an ascending method is used, and finally the mouse ear is used as the first index to calculate the Euclidean distance between the paw and the ear; S8. Set a distance threshold and use it as a condition to determine whether the mouse has a scratching action; if the calculated distance is less than the threshold, the mouse has a scratching action, and the mouse's paw is marked with a box; if the calculated distance is greater than the threshold, the mouse has no scratching action, and the mouse is not marked; S9. The above method was applied to detect the scratching behavior of mice under different conditions: a single mouse under artificial lighting, a single mouse under dim lighting, and two mice under sunlight, to verify the feasibility of the method.

2. A method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The step S1 comprises the following steps: A camera was used to capture video images. The camera was installed just above the open field test box. Its viewing angle was required to cover the entire open field. A lighting lamp with a light intensity of 200 lx was installed above the test box to provide lighting for a single mouse under artificial lighting conditions. The frame rate of the captured video images was 30 frames per second.

3. The method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: Step S3 The following steps are included: S3.1, calculate the grayscale histogram of the image; S3.2, iterate each possible threshold value to classify the image into two categories: background and foreground; S3.

3. For each threshold, calculate the inter-class variance of the background and foreground categories, that is, the degree of difference between the two categories, and select the threshold that maximizes the inter-class variance as the optimal threshold; S3.

4. Apply the optimal threshold to the image for binarization processing, set the pixels less than or equal to the threshold as the background, and the pixels greater than the threshold as the foreground, so as to determine the binarization threshold suitable for the image, so that the target object of the image is separated from the background more clearly.

4. The method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The step S4 comprises the following steps: For the image image, it is regarded as an M×N matrix, that is, the pixels in the image, each value is the pixel value, where the pixel value is between 0 and 255, the segmentation threshold of the foreground and background is recorded as threshold, the proportion of the number of pixels belonging to the foreground to the entire image is recorded as ω0, and the average grayscale of the foreground is recorded as μ0; The ratio of background pixels to the entire image is ω1, and the average background grayscale is μ1. The total average grayscale of the image is μ, and the inter-class variance is σ; assuming that the background of the image is dark, and the image size is M×N, the number of pixels in the image whose grayscale value is less than the threshold threshold is N0, and the number of pixels whose grayscale is greater than or equal to the threshold threshold is N1, then: N0+N1=M×N (3) 1=ω0+ω1 (4) μ=ω0×μ0+ω1×μ1 (5) σ=ω0×(μ0-μ) 2 +ω1×(μ1-μ) 2 (6)。 5. The method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The principle of the image subtraction operation in step S4 is to subtract corresponding pixels. Assuming there are two images A and B, and their corresponding pixels are A(i,j) and B(i,j) respectively, then when performing image subtraction operation, a new image C(i,j)=A(i,j)-B(i,j) can be obtained, and each pixel value in image C is the difference between the pixel values ​​at the corresponding position of images A and B.

6. A method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The principle of morphological dilation or erosion in step S5 is to convolve the image or a part of the image with the structural element; wherein the image or a part of the image is represented by image A, and the structural element is represented by convolution kernel B, and the convolution kernel can be of any shape and size, and has a separately defined reference point, which is called an anchor point; Dilation is an operation to find the local maximum value. The convolution kernel B is convolved with the image, that is, the maximum value of the pixel points in the area covered by the convolution kernel B is calculated, and this maximum value is assigned to the pixel specified by the reference point, so that the highlight area in the image gradually grows. The formula can be expressed as: Erosion is to remove the burrs and details of the image. Erosion can be used to eliminate noise and segment independent image elements. It is essentially a spatial filter. A mask is set, and the center of the mask slides over each pixel point one by one. The value of the current pixel point is set to the minimum value of the pixel in the mask coverage area. The formula can be expressed as: The operation of dilating and then corroding an image is called a closing operation, which is used to fill small holes in the foreground object or erase small black spots on the foreground object. First, the white part is enlarged, the small black part is squeezed out, and then some large black parts are restored. The overall effect is to erase the small black spots on the foreground object. The formula can be expressed as: The operation of first corroding and then dilating an image is called opening operation, which is used to eliminate small objects, separate objects at thin points, and smooth the boundaries of larger objects without significantly changing their areas. The formula can be expressed as:

7. The method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The small object removal in step S5 is to remove the connected areas smaller than the specified area in the binary image; First, the connected regions of the input binary image are labeled, and each connected region in the image is assigned a unique label; Then, the number of pixels contained in each connected region is calculated, that is, the area of ​​the region; Traverse all connected areas and set the pixel values ​​of areas smaller than the specified threshold (small connected areas) to 0, thereby removing these small connected areas; Finally, the filtered pixels are reassembled into a new binary image, where small connected regions are removed.

8. The method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The principle of the bubble sort in step S7 is that, assuming that a known unordered sequence is to be arranged from small to large, the first element is compared with the following elements in sequence. If the following element is smaller, it is exchanged with a certain element behind it, and then this element is used to compare in sequence. If it is smaller than this element, it is exchanged with this element until the last element. A bubble sequence is completed, and the last element must be the largest element in this sequence.

9. The method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The principle of calculating the Euclidean distance in step S7 is that the Euclidean distance is the real distance between two points in n-dimensional space, and the Euclidean distance in two-dimensional and three-dimensional space is the distance between two points. The calculation of the Euclidean distance is to first calculate the difference in each dimension, then square these interpolations, then add all the square differences, and take the square root of the result. The mathematical model formula is as follows: Where d(A,B) represents the distance between point A and point B, (a1-b1) 2 +(a2-b2) 2 +...+(a n -b n ) 2 Represents the sum of the squared distances between two points.

10. The method for detecting mouse scratching behavior based on machine vision according to claim 1, characterized in that: The step S9 applies the above method to different situations: a single mouse is illuminated by artificial light, a single mouse is illuminated by dim light, and two mice are illuminated by sunlight. For a single mouse, artificial lighting was provided by lighting from above the open field test box, and the light intensity of the entire lighting area was selected to be 200 lx. Black mice were selected. For a single mouse, the lighting above the test box was turned off when the light was dim, and the sunlight was blocked to a certain extent. Black mice were selected. For two mice, the two mice were placed in the open field test box at the same time, and the lighting above was turned off, leaving only normal sunlight exposure. Both mice were selected to be white mice.

Images