Mouse gait analysis method and system based on color regionalization

Abstract

The invention belongs to the technical field of behavior analysis, and discloses a mouse gait analysis method and system based on color regionalization; the method comprises: demarcating a mouse interest area in a preset closed walking platform, collecting background pictures, allowing the mouse to move freely in the closed walking platform, turning on a camera to continuously shoot, and collecting a natural gait video; performing a first detection on the natural gait video to obtain a foreground image set containing only the mouse, establishing a color regionalization coordinate system, combining a spatial logic algorithm to distinguish footprints, and performing detection through a short-term interference rejection algorithm and a four-phase cross-validation algorithm; performing a second detection, performing regional coloring on the image through the first detection result, and calculating the gait analysis data of a specified paw through a specified color; the invention can obtain accurate gait analysis data, and provide strong support for mouse behavior research and research in the field of neuroscience.

Description

Technical Field

[0001] The present invention relates to the technical field of behavior analysis, and more specifically, to a mouse gait analysis method and system based on color regionalization. Background Art

[0002] Mouse gait analysis is an important research method and experimental technique, which is widely used in many fields such as animal behavior, neuroscience, pharmacology, and kinematics. It provides scientists and researchers with a very useful means to deeply study the gait characteristics, movement patterns, and the relationship between the nervous system and motor behavior of mice. In traditional mouse gait analysis, researchers usually use a variety of techniques and tools to collect and analyze gait data. Some of the important steps and techniques include: Footprint sensor is a commonly used device installed on a walking platform to detect the footprints of mice in real time. Footprint sensor can record the location and time information of footprints for subsequent gait analysis and data processing. Infrared photoelectric sensor can be used to detect the gait of mice. When the foot of the mouse passes through the sensor, it triggers an infrared signal, thereby recording the location and time information of the footprint. With the continuous development of computer vision, machine learning and other technologies, automated mouse gait analysis technology has also developed rapidly.

[0003] The Chinese patent with publication number CN105373780A discloses a method, device and system for identifying footsteps in mouse gait analysis, including: selecting a preset number of frames of image data in the mouse gait video data, and eliminating the mouse body coverage area in the several frames of image data; constructing a background image by superimposing and averaging the several frames of image data excluding the mouse body coverage area; removing interference noise in the image data according to the pre-constructed background image, the interference noise includes the light spot area generated by mouse excrement; identifying the mouse body coverage area in the image data, and extracting the sole light spot area in the mouse body coverage area through a binarization algorithm; determining the attributes of the sole light spot area, the attributes including the left forefoot, right forefoot, left hindfoot or right hindfoot. The invention can accurately complete the footstep identification in mouse gait analysis.

[0004] Although the above method can meet most scenarios, research and practical application of the above method and existing technology have found that the above method and existing technology have at least the following defects:

[0005] It relies on the characteristics of the light spot area on the sole of the foot to identify footsteps. When distinguishing different claws, complex algorithms and feature analysis are required. When the characteristics of the light spot area are not obvious or there is interference, misjudgment is easy to occur, resulting in inaccurate claw distinction. The ability to capture and analyze details such as posture changes and speed changes of the claws during movement is relatively weak.

[0006] In view of this, the present invention proposes a mouse gait analysis method and system based on color regionalization to solve the above problems. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a mouse gait analysis method based on color regionalization, comprising the following steps:

[0008] A closed walking platform is preset, and a side-light footprint enhancement device is built into the bottom of the closed walking platform. The area of ​​interest of the mouse is demarcated, and a background picture of the closed walking platform excluding the mouse is collected by using a high-definition camera. The shooting angle and shooting height of the background picture are the same as the shooting angle and shooting height of the subsequent natural gait video collection. No coercive measures are taken on the mouse, and the mouse is allowed to move freely in the closed walking platform. The high-definition camera is turned on to continuously shoot the mouse, and a green LED light is used as the light source of the side-light footprint enhancement device to enter the glass side of the side-light footprint enhancement device, and a natural gait video containing green fluorescent footprints of the mouse is obtained based on the footprint light refraction technology;

[0009] The natural gait video is first detected. Each frame of the natural gait video is subtracted from the background image pixel by pixel through the background subtraction technology to obtain a foreground image set containing only mice. A color regionalization coordinate system is established based on the foreground image set. The mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs is distinguished by the spatial logic algorithm. The short-term interference rejection algorithm and the four-phase cross-validation algorithm are used for detection to obtain the final classification result and the coordinates of the footprint circumscribed rectangular box.

[0010] The natural gait video is tested a second time, and each frame is regionally colored according to the result of the first test, and the gait analysis data of the specified paw is calculated by specifying the color.

[0011] Furthermore, the method of establishing a color regionalization coordinate system based on the foreground image set includes:

[0012] Step 1: Binarize the foreground image set so that the foreground area where the mouse is located is white and the background area is black. Step 2: Remove the mouse's tail area through corrosion and expansion operations, and only keep the mouse's body area, which is marked as A. body ; Step 3: Calculate the maximum contour of the body area and mark it as counter max , if the maximum contour satisfies counter max >1000, then calculate the centroid corresponding to the maximum contour (M x ,M y ); Step 4, compare the horizontal coordinate M of the center of mass xThe size of the horizontal coordinate of the center point of the image, if M x If the horizontal coordinate is greater than the horizontal coordinate of the center point of the image, it is judged that the mouse is walking from right to left, otherwise it is judged that the mouse is walking from left to right; Step 5, calculate the maximum contour counter max The horizontal coordinate of the left convex hull point is left x and the horizontal coordinate of the right convex hull point right x , and establish a dynamic coordinate system with the centroid, left convex hull points and right convex hull points. When the centroid is on the right side of the image center, M x and left x The line connecting the two is the x-axis, and the line passing through M y The perpendicular line is the y-axis. When the centroid is on the left side of the image center, M x and right x The line connecting the two is the x-axis, and the line passing through M y The perpendicular line is the y-axis; Step 6, traverse all non-black pixels in the foreground image of the current frame, calculate the distances position1 and position2 from all non-black pixels to the two coordinate axes of the dynamic coordinate system, and reassign new colors to all non-black pixels through the color matrix to establish a color regionalized coordinate system. The color matrix color is as follows: The method to assign new colors to all non-black pixels is as follows: Among them, color(k) is the element in the color matrix, and k is the index value; Step 7, extract the color corresponding to the paw and calculate the coordinates of the maximum circumscribed rectangle of each paw through the expansion area calculation comparison method.

[0013] Furthermore, the method of calculating the coordinates of the maximum circumscribed rectangle of each claw by the expansion area calculation comparison method includes:

[0014] Take out the j-th color pixel area in each frame of the foreground image and dilate it, where the dilated area is larger than the area of ​​the largest claw; calculate the claw landing coordinate rec of the j-th color pixel area in the t-th frame new The area and the paw landing coordinates rec of the j-th color pixel area in the previous frame old The intersection of areas and union ratio iou, if iou>0.1, it is considered that rec new With rec old Same, using rec old As the paw landing coordinates of the j-th color pixel area in the t-th frame, otherwise rec new This is a new claw landing coordinate, using rec new The paw landing coordinates of the j-th color pixel area in the t-th frame.

[0015] Furthermore, the method of distinguishing the mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs in combination with the spatial logic algorithm includes: re-establishing the coordinate system for the mouse entering and leaving the region of interest, when the horizontal coordinate M of the center of mass is x When the distance from the boundary is less than N pixels, the horizontal coordinate M of the centroid x The line connecting the horizontal coordinates of the convex hull points on the side of the boundary farthest from the mouse is the distinguishing line;

[0016] The method of obtaining the final classification result and the coordinates of the footprint outer rectangular box through the short-term interference elimination algorithm and the four-phase cross-validation algorithm includes:

[0017] The short-term interference elimination algorithm is as follows:

[0018] Calculate the color matrix of the first detection record, and when the number of consecutive appearances of the j-th color is less than n frames, determine that the j-th color is a short-term interference, and eliminate the j-th color;

[0019] The four-phase cross-validation algorithm is as follows:

[0020] The adjacent rectangles of the coordinate system of each frame of the foreground image are cross-validated. If the overlapping area ratio of the adjacent rectangles is greater than 0.1, the rectangle with a larger area is retained and the rectangle with a smaller area is eliminated as a false positive.

[0021] Furthermore, the method of performing regional coloring for each frame of image according to the result of the first detection includes:

[0022] In the foreground image area of ​​each frame, the non-black pixels corresponding to the mouse paw are replaced with the specified color according to the color matrix, and then the original image is converted into the HSV space to obtain a colored image containing only the paw, and the regionalized image with only the pixels of the i-th color is extracted. The regionalized image is binarized and converted into a binary regionalized image, in which the pixel part containing the i-th color is white and the background is black. The noise in the binary regionalized image is removed to obtain the eroded image, and the coordinates of the maximum circumscribed rectangle of the specified paw in the eroded image are calculated by the expansion area calculation comparison method.

[0023] Furthermore, the method of calculating the coordinates of the maximum circumscribed rectangle of the designated claw in the erosion image by the expansion area calculation comparison method includes:

[0024] Dilate each frame of the eroded image so that the expanded area is larger than the area of ​​the specified claw; find the largest contour and calculate the maximum enclosing rectangle; set the maximum enclosing rectangle of the eroded image of the current frame rec cur =[x,y,w,h] and the maximum enclosing rectangle of the previous eroded image recold =[x old ,y old ,w old ,h old ] to compare and calculate the intersection-and-union ratio iou; if iou>0.1, it means that the mouse's designated paw has touched the ground, and the maximum enclosing rectangle of the current frame becomes rec cur =[x old ,y old ,w old ,h old ]; add 1 to the number of frames of the specified claw’s landing time; if iou≤0.1, record the maximum bounding rectangle of the frame where the specified claw appears in the current frame’s eroded image as rec old =[x, y, w, h], and analyze the motion data of the specified paw; the motion data include maximum touchdown time, support time, footprint area, maximum footprint area, braking time, propulsion time, swing time, swing speed, maximum footprint length, maximum footprint width, step length of walking cycle and instantaneous speed.

[0025] Furthermore, the method for analyzing the maximum contact time of the specified paw includes:

[0026] Initialize the maximum landing time of the mouse to 0, count the number of corrosion image frames in which the designated paw of the mouse is continuously in the landing state, divide the number of corrosion image frames by the frame rate to obtain the landing time, use the landing time as the support duration, compare the landing time with the recorded maximum landing time of the designated paw, if the landing time is greater than the maximum landing time, update the landing time to the maximum landing time, otherwise do not update the maximum landing time;

[0027] Methods for analyzing the maximum footprint area for a given paw include:

[0028] Initialize the maximum footprint area of ​​the mouse's designated paw to 0, count the number of non-black pixels in the maximum circumscribed rectangle, and multiply the number of non-black pixels by the eroded image scale to obtain the value as the footprint area of ​​the mouse's designated paw; compare the footprint area with the maximum footprint area, if the footprint area is larger than the maximum footprint area, mark the mouse in the propulsion state, and update the footprint area to the maximum footprint area, otherwise mark the mouse in the braking state, and do not update the maximum footprint area;

[0029] Methods for analyzing the braking and propulsion durations of a given claw include:

[0030] The number of corrosion image frames continuously marked as the advancing state is counted, and the number of corrosion image frames continuously marked as the advancing state is divided by the frame rate to obtain the advancing time; the number of corrosion image frames continuously marked as the braking state is counted, and the number of corrosion image frames continuously marked as the braking state is divided by the frame rate to obtain the braking time;

[0031] Methods for analyzing the swing duration and swing speed of a given paw include:

[0032] The time when the designated paw leaves the ground for the s+1th time is recorded, and subtracted from the time when the designated paw touches the ground for the sth time. The time difference obtained is recorded as the swing duration, and the value obtained by dividing the step length by the swing time is taken as the swing speed.

[0033] Further, the method of analyzing the length of the maximum footprint and the width of the maximum footprint of the specified paw includes:

[0034] Initialize the maximum contour of the specified claw to 0, count the number of non-black pixels on the outermost side of the eroded image, compare the number of non-black pixels on the outermost side with the recorded maximum contour, if the number of non-black pixels on the outermost side is greater than the maximum contour, update the number of non-black pixels on the outermost side to the maximum contour, otherwise do not update the maximum contour; traverse all non-black pixels on the maximum contour, calculate the Euclidean distance between any two points on the maximum contour, mark the two non-black pixels with the largest Euclidean distance as length points, and use the Euclidean distance between the length points as the length of the maximum footprint; connect the length points to obtain the length point direction vector, calculate the perpendicular vector of the length point direction vector in the plane where the maximum contour is located by the vector method, mark the two non-black pixels with the largest Euclidean distance between the two non-black pixels passing through the perpendicular vector as width points, and use the Euclidean distance between the width points as the width of the maximum footprint;

[0035] Methods for analyzing step length and instantaneous velocity of a given paw’s walk cycle include;

[0036] Record the first landing time of the specified paw in the current landing state, subtract it from the first landing time of the specified paw in the previous landing state, obtain the walking cycle, and record the position coordinates of the two landing states; the value obtained by subtracting the two position coordinates is used as the step length; the value obtained by dividing the step length by the walking cycle is used as the instantaneous speed.

[0037] Further, the method of calculating the gait analysis data of the specified paw by the specified color includes:

[0038] Extract all motion data corresponding to the specified color, count all support time, footprint area, braking time, propulsion time, swing time, swing speed, step length and instantaneous speed of the walking cycle, and calculate the average value to obtain the average support time, average footprint area, average braking time, average propulsion time, average swing time, average swing speed, average step length and average instantaneous speed.

[0039] A mouse gait analysis method based on color regionalization, implementing a mouse gait analysis method based on color regionalization, comprising:

[0040] Data acquisition module: used to preset a closed walking platform, with a side-light footprint enhancement device built in the bottom of the closed walking platform, demarcate the area of ​​interest of the mouse, and use a high-definition camera to collect a background picture of the closed walking platform excluding the mouse. The shooting angle and shooting height of the background picture are the same as the shooting angle and shooting height of the subsequent natural gait video. No coercive measures are taken on the mouse, and the mouse is allowed to move freely in the closed walking platform. The high-definition camera is turned on to continuously shoot the mouse, and a green LED light is used as the light source of the side-light footprint enhancement device, which enters the glass side of the side-light footprint enhancement device, and a natural gait video containing the green fluorescent footprint of the mouse is obtained based on the footprint light refraction technology;

[0041] The first detection module is used to perform the first detection on the natural gait video. Each frame of the natural gait video is subtracted from the background image pixel by pixel through the background subtraction technology to obtain a foreground image set containing only mice. A color regionalization coordinate system is established based on the foreground image set. The mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs is distinguished by the spatial logic algorithm. The short-term interference rejection algorithm and the four-phase cross-validation algorithm are used for detection to obtain the final classification result and the coordinates of the footprint circumscribed rectangular box.

[0042] The second detection module is used to perform a second detection on the natural gait video, regionalize and color each frame of the image based on the results of the first detection, and calculate the gait analysis data of the specified paw by specifying the color.

[0043] Technical effects and advantages of the mouse gait analysis method and system based on color regionalization of the present invention:

[0044] The present invention adopts a method based on color regionalization. First, the gait area of ​​the mouse is marked and segmented by color information, which can effectively distinguish the body area of ​​the mouse from the background, reduce the influence of fur color and background, and thus better locate the gait area of ​​the mouse; by processing each frame of gait image, the gait changes of the mouse can be tracked in continuous video frames. At the same time, it can also calculate the gait analysis data of the mouse, and can analyze the gait characteristics and behavior of the mouse in more depth, which can be widely used in the fields of mouse behavior research and pharmacology. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of the mouse gait analysis method based on color regionalization of the present invention;

[0046] Figure 2 A schematic diagram of re-establishing a coordinate system for a mouse entering and leaving a region of interest in combination with a spatial logic algorithm of the present invention;

[0047] Figure 3 It is a schematic diagram of the detection process of the present invention;

[0048] Figure 4 This is a block diagram of the mouse gait analysis system based on color regionalization of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Example 1

[0051] See also Figure 1 As shown, the mouse gait analysis method based on color regionalization described in this embodiment includes the following steps:

[0052] A closed walking platform is preset, and a side-light footprint enhancement device is built into the bottom of the closed walking platform. The area of ​​interest of the mouse is delineated, and a background picture of the closed walking platform excluding the mouse is collected using a high-definition camera. The shooting angle and shooting height of the background picture are the same as the shooting angle and shooting height of the subsequent natural gait video. No coercive measures are taken on the mouse, and the mouse is allowed to move freely in the closed walking platform. The high-definition camera is turned on to continuously shoot the mouse, and a natural gait video containing the green fluorescent footprint of the mouse is obtained based on the footprint light refraction technology;

[0053] The method for obtaining a natural gait video containing green fluorescent footprints of mice based on footprint light refraction technology includes:

[0054] Green LED light is used as the light source of the side-light footprint enhancement device, and the green LED light is projected into the glass side of the side-light footprint enhancement device; when the mouse footprint is located on the glass plane, the footprint will reflect and refract with the green fluorescence, forming a light and dark contrast, thereby producing a clearly visible footprint image. This side-lighting method helps to reduce background interference, making the boundaries of the footprint clearer, which is convenient for subsequent image processing and analysis. Through the side-light footprint enhancement device, the mouse footprint in the footprint image is significantly enhanced, the outline is clearer, and the footprint can be identified and tracked more accurately.

[0055] The natural gait video is first detected. Each frame of the natural gait video is subtracted from the background image pixel by pixel through the background subtraction technology to obtain a foreground image set containing only mice. A color regionalization coordinate system is established based on the foreground image set to assign a color to each paw of the mouse. The green fluorescent footprint in each frame of the foreground image is distinguished by a spatial logic algorithm. The first detection result is tested by a short-term interference rejection algorithm and a four-phase cross-validation algorithm to obtain the final classification result and the coordinates of the footprint circumscribed rectangular box.

[0056] Reference Figure 3 , the method of establishing a color regionalization coordinate system based on the foreground image set includes:

[0057] Step 1, binarize the foreground image set so that the foreground area where the mouse is located in the foreground image set is white and the background area is black;

[0058] Step 2: Remove the mouse's tail region through corrosion and expansion operations, leaving only the mouse's body region and marking it as A. body ;

[0059] Step 3: Calculate the maximum contour of the body area and mark it as counter max , if the maximum contour satisfies counter max >1000, then calculate the centroid corresponding to the maximum contour (M x ,M y );

[0060] Step 4: Compare the horizontal coordinate M of the center of mass x The size of the horizontal coordinate of the center point of the image, if M x If the horizontal coordinate is greater than the center point of the image, it is judged that the mouse is walking from right to left, otherwise it is judged that the mouse is walking from left to right;

[0061] Step 5: Calculate the maximum contour counter max The horizontal coordinate of the left convex hull point is left x and the horizontal coordinate of the right convex hull point right x , and establish a dynamic coordinate system with the centroid, left convex hull points and right convex hull points. When the centroid is on the right side of the image center, M x and left x The line connecting the two is the x-axis, and the line passing through M y The perpendicular line is the y-axis. When the centroid is on the left side of the image center, M x and right x The line connecting the two is the x-axis, and the line passing through M y The perpendicular line is the y-axis.

[0062] Step 6: Traverse all non-black pixels in the foreground image of the current frame, calculate the distances position1 and position2 from all non-black pixels to the two coordinate axes of the dynamic coordinate system, and reassign new colors to all non-black pixels through the color matrix to establish a color regionalized coordinate system. The color matrix color is as follows:

[0063]

[0064] The method to assign new colors to all non-black pixels is as follows:

[0065]

[0066] Among them, color(k) is the element in the color matrix used to assign colors to the mouse paws, and k is the index value used to select the color in the color matrix.

[0067] Step 7. Extract the color corresponding to the paw and calculate the coordinates of the maximum circumscribed rectangle of each paw through the expansion area calculation comparison method. Although each frame of the paw will generate a new external rectangular coordinate, since it is the same mouse, only one rectangular coordinate needs to be retained.

[0068] The method of calculating the coordinates of the maximum circumscribed rectangle of each claw by the expansion area calculation comparison method includes:

[0069] Take out the j-th color pixel area in each frame of the foreground image and dilate it, where the dilated area is larger than the area of ​​the largest claw;

[0070] Calculate the paw landing coordinates rec of the j-th color pixel area in the t-th frame new The area and the paw landing coordinates rec of the j-th color pixel area in the previous frame old The intersection of areas and union ratio iou, if iou>0.1, it is considered that rec new With rec old Same, using rec old As the paw landing coordinates of the j-th color pixel area in the t-th frame, otherwise rec new This is a new claw landing coordinate, using rec new The paw landing coordinates of the j-th color pixel area are taken as the t-th frame, so that only one rectangular frame coordinate is retained each time the paw lands.

[0071] The method of distinguishing the mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs by combining the spatial logic algorithm includes:

[0072] Reference Figure 2 , re-establish the coordinate system for mice entering and leaving the region of interest,

[0073] When the horizontal coordinate M of the center of mass x When the distance from the boundary is less than N (N is preferably 50) pixels, the horizontal coordinate M of the centroid is x The line connecting the horizontal coordinates of the convex hull points on the side of the boundary farthest from the mouse is a distinguishing straight line, which is used to distinguish the two paws on the side of the boundary farthest from the mouse.

[0074] Take the mouse walking from right to left as an example. When the horizontal coordinate M of the center of mass is x When the distance to the right border is less than 50 pixels, M x and left x The line connecting the left and right front paws is the distinguishing line. By default, it only distinguishes between the left and right front paws. Similarly, when the horizontal coordinate M of the center of mass is x When the distance from the left border is less than 50 pixels, M x and right x The line connecting the two is the distinguishing straight line. By default, it only distinguishes between the left hind paw and the right hind paw.

[0075] The method of obtaining the final classification result and the coordinates of the footprint outer rectangular box through the short-term interference elimination algorithm and the four-phase cross-validation algorithm includes:

[0076] The short-term interference elimination algorithm is as follows:

[0077] The color matrix of the first detection record is calculated. When the number of consecutive appearances of the jth color is less than n frames, the jth color is determined to be a short-term interference and the jth color is eliminated. Taking the left front paw as an example, when the number of consecutive appearances of yellow is less than 5 frames, yellow is determined to be a short-term interference and yellow is eliminated. Eliminating short-term interference in the color matrix can improve the accuracy of mouse gait recognition analysis.

[0078] The four-phase cross-validation algorithm is as follows:

[0079] Cross-validate the adjacent rectangles of the coordinate system of each frame of the foreground image. If the overlapping area ratio of the adjacent rectangles is greater than 0.1, the rectangle with the larger area is retained and the rectangle with the smaller area is eliminated as the part of the false detection. Cross-validating the rectangular matrix can avoid the situation where the same paw is recognized as both the left front paw and the right front paw, and can improve the accuracy of mouse gait recognition analysis.

[0080] The natural gait video is tested a second time, and each frame is regionally colored according to the result of the first test, and the gait analysis data of the specified paw is calculated by specifying the color.

[0081] Reference Figure 3, the method of regionalizing and coloring each frame of image by the result of the first detection includes:

[0082] In the foreground image area of ​​each frame, the non-black pixels corresponding to the mouse paw are replaced with the specified color according to the color matrix, and then the original image is converted into the HSV space to obtain a colored image containing only the paw, and the regionalized image with only the pixels of the i-th color is extracted (the regionalized image with only the pixels of the i-th color corresponds to all images containing the specified paw of a mouse. Taking the yellow corresponding to the left front paw as an example, the regionalized image only contains the left front paw of the mouse), and the regionalized image is binarized to convert the regionalized image into a binary regionalized image, in which the pixel part containing the i-th color is white and the background is black. The noise in the binary regionalized image is removed to obtain the eroded image, and the coordinates of the maximum circumscribed rectangle of the specified paw in the eroded image are calculated by the expansion area calculation comparison method.

[0083] The method of calculating the coordinates of the maximum circumscribed rectangle of a specified claw in the eroded image by the expansion area calculation comparison method includes:

[0084] Dilate each frame of the eroded image so that the expanded area is larger than the area of ​​the specified claw; find the largest contour and calculate the maximum enclosing rectangle; set the maximum enclosing rectangle of the eroded image of the current frame rec cur =[x,y,w,h] and the maximum enclosing rectangle of the previous eroded image rec old =[x old ,y old ,w old ,h old ] to compare and calculate the intersection-and-union ratio iou; if iou>0.1, it means that the mouse's designated paw has touched the ground, and the maximum enclosing rectangle of the current frame becomes rec cur =[x old ,y old ,w old ,h old ]; add 1 to the number of frames of the specified claw’s landing time; if iou≤0.1, record the maximum bounding rectangle of the frame where the specified claw appears in the current frame’s eroded image as rec old =[x, y, w, h], and analyze the motion data of the specified paw; the motion data include maximum touchdown time, support time, footprint area, maximum footprint area, braking time, propulsion time, swing time, swing speed, maximum footprint length, maximum footprint width, step length of walking cycle and instantaneous speed.

[0085] Methods for analyzing the maximum ground contact time for a given paw include:

[0086] Initialize the maximum landing time of the mouse to 0, count the number of corrosion image frames in which the specified paw of the mouse is continuously in the landing state, divide the number of corrosion image frames by the frame rate to obtain the landing time, use the landing time as the support time, compare the landing time with the recorded maximum landing time of the specified paw, if the landing time is greater than the maximum landing time, update the landing time to the maximum landing time, otherwise do not update the maximum landing time.

[0087] Methods for analyzing the maximum footprint area for a given paw include:

[0088] Initialize the maximum footprint area of ​​the mouse's designated paw to 0, count the number of non-black pixels in the maximum circumscribed rectangle, and multiply the number of non-black pixels by the eroded image scale to obtain the value as the footprint area of ​​the mouse's designated paw; compare the footprint area with the maximum footprint area, if the footprint area is larger than the maximum footprint area, mark the mouse in the propulsion state, and update the footprint area to the maximum footprint area, otherwise mark the mouse in the braking state, and do not update the maximum footprint area;

[0089] Methods for analyzing the braking and propulsion durations of a given claw include:

[0090] The number of corrosion image frames continuously marked as the advancing state is counted, and the number of corrosion image frames continuously marked as the advancing state is divided by the frame rate to obtain the advancing duration; the number of corrosion image frames continuously marked as the braking state is counted, and the number of corrosion image frames continuously marked as the braking state is divided by the frame rate to obtain the braking duration.

[0091] Methods for analyzing the length of the largest footprint and the width of the largest footprint of a given paw include;

[0092] Initialize the maximum contour of the specified claw to 0, count the number of the outermost non-black pixels in the eroded image, compare the number of the outermost non-black pixels with the recorded maximum contour, if the number of the outermost non-black pixels is greater than the maximum contour, update the number of the outermost non-black pixels to the maximum contour, otherwise do not update the maximum contour; traverse all non-black pixels on the maximum contour, calculate the Euclidean distance between any two points on the maximum contour, mark the two non-black pixels with the largest Euclidean distance as length points, and use the Euclidean distance between the length points as the length of the maximum footprint; connect the length points to obtain the length point direction vector, calculate the perpendicular vector of the length point direction vector in the plane where the maximum contour is located by the vector method, mark the two non-black pixels with the largest Euclidean distance between the two non-black pixels passing through the perpendicular vector as width points, and use the Euclidean distance between the width points as the width of the maximum footprint.

[0093] Methods for analyzing the swing duration and swing speed of a given paw include:

[0094] The time when the designated paw leaves the ground for the s+1th time is recorded, and subtracted from the time when the designated paw touches the ground for the sth time. The time difference obtained is recorded as the swing duration, and the value obtained by dividing the step length by the swing time is taken as the swing speed.

[0095] Methods for analyzing step length and instantaneous velocity of a given paw’s walk cycle include;

[0096] Record the first landing time of the specified paw in the current landing state, subtract it from the first landing time of the specified paw in the previous landing state, obtain the walking cycle, and record the position coordinates of the two landing states; the value obtained by subtracting the two position coordinates is used as the step length; the value obtained by dividing the step length by the walking cycle is used as the instantaneous speed.

[0097] When all corrosion images of a specified paw are analyzed, the gait analysis data of the specified paw are calculated, and the gait analysis data include average support time, average swing time, average braking time, average propulsion time, average footprint area, average swing speed, and average instantaneous speed.

[0098] Methods for calculating gait analysis data for a specified paw by specifying a color include:

[0099] Extract all motion data corresponding to the specified color, count all support time, footprint area, braking time, propulsion time, swing time, swing speed, step length and instantaneous speed of the walking cycle, and calculate the average value to obtain the average support time, average footprint area, average braking time, average propulsion time, average swing time, average swing speed, average step length and average instantaneous speed.

[0100] Example 2

[0101] See also Figure 4 As shown, the mouse gait analysis system based on color regionalization described in this embodiment includes:

[0102] Data acquisition module: used to preset a closed walking platform, with a side-light footprint enhancement device built in the bottom of the closed walking platform, demarcate the area of ​​interest of the mouse, and use a high-definition camera to collect a background picture of the closed walking platform excluding the mouse. The shooting angle and shooting height of the background picture are the same as the shooting angle and shooting height of the subsequent natural gait video. No coercive measures are taken on the mouse, and the mouse is allowed to move freely in the closed walking platform. The high-definition camera is turned on to continuously shoot the mouse, and a green LED light is used as the light source of the side-light footprint enhancement device, which enters the glass side of the side-light footprint enhancement device, and a natural gait video containing the green fluorescent footprint of the mouse is obtained based on the footprint light refraction technology;

[0103] The first detection module is used to perform the first detection on the natural gait video. Each frame of the natural gait video is subtracted from the background image pixel by pixel through the background subtraction technology to obtain a foreground image set containing only mice. A color regionalization coordinate system is established based on the foreground image set. The mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs is distinguished by the spatial logic algorithm. The short-term interference rejection algorithm and the four-phase cross-validation algorithm are used for detection to obtain the final classification result and the coordinates of the footprint circumscribed rectangular box.

[0104] The second detection module is used to perform a second detection on the natural gait video, regionalize and color each frame of the image based on the results of the first detection, and calculate the gait analysis data of the specified paw by specifying the color.

[0105] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0106] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A mouse gait analysis method based on color regionalization, characterized in that: The steps include: A closed walking platform is preset, and a side-light footprint enhancement device is built into the bottom of the closed walking platform. The area of ​​interest of the mouse is demarcated, and a background picture of the closed walking platform excluding the mouse is collected by using a high-definition camera. The shooting angle and shooting height of the background picture are the same as the shooting angle and shooting height of the subsequent natural gait video collection. No coercive measures are taken on the mouse, and the mouse is allowed to move freely in the closed walking platform. The high-definition camera is turned on to continuously shoot the mouse, and a green LED light is used as the light source of the side-light footprint enhancement device to enter the glass side of the side-light footprint enhancement device, and a natural gait video containing green fluorescent footprints of the mouse is obtained based on the footprint light refraction technology; The natural gait video is first detected. Each frame of the natural gait video is subtracted from the background image pixel by pixel through the background subtraction technology to obtain a foreground image set containing only mice. A color regionalization coordinate system is established based on the foreground image set. The mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs is distinguished by the spatial logic algorithm. The short-term interference rejection algorithm and the four-phase cross-validation algorithm are used for detection to obtain the final classification result and the coordinates of the footprint circumscribed rectangular box. The natural gait video is tested a second time, and each frame is regionally colored according to the result of the first test, and the gait analysis data of the specified paw is calculated by specifying the color.

2. The mouse gait analysis method based on color regionalization according to claim 1, characterized in that: The method of establishing a color regionalization coordinate system based on the foreground image set includes: Step 1: Binarize the foreground image set so that the foreground area where the mouse is located is white and the background area is black. Step 2: Remove the mouse's tail area through corrosion and expansion operations, and only keep the mouse's body area, which is marked as A. body ; Step 3: Calculate the maximum contour of the body area and mark it as counter max , if the maximum contour satisfies counter max >1000, then calculate the centroid corresponding to the maximum contour (M x ,M y ); Step 4, compare the horizontal coordinate M of the center of mass x The size of the horizontal coordinate of the center point of the image, if M x If the horizontal coordinate is greater than the horizontal coordinate of the center point of the image, it is judged that the mouse is walking from right to left, otherwise it is judged that the mouse is walking from left to right; Step 5, calculate the maximum contour counter max The horizontal coordinate of the left convex hull point is left x and the horizontal coordinate of the right convex hull point right x , and establish a dynamic coordinate system with the centroid, left convex hull points and right convex hull points. When the centroid is on the right side of the image center, M x and left x The line connecting the two is the x-axis, and the line passing through M y The perpendicular line is the y-axis. When the centroid is on the left side of the image center, M x and right x The line connecting the two is the x-axis, and the line passing through M y The perpendicular line is the y-axis; Step 6, traverse all non-black pixels in the foreground image of the current frame, calculate the distances position1 and position2 from all non-black pixels to the two coordinate axes of the dynamic coordinate system, and reassign new colors to all non-black pixels through the color matrix to establish a color regionalized coordinate system. The color matrix color is as follows: The method to assign new colors to all non-black pixels is as follows: Among them, color(k) is the element in the color matrix, and k is the index value; Step 7, extract the color corresponding to the paw and calculate the coordinates of the maximum circumscribed rectangle of each paw through the expansion area calculation comparison method.

3. The mouse gait analysis method based on color regionalization according to claim 2, characterized in that: The method for calculating the coordinates of the maximum circumscribed rectangle of each claw by the expansion area calculation comparison method includes: Take out the j-th color pixel area in each frame of the foreground image and dilate it, where the dilated area is larger than the area of ​​the largest claw; calculate the claw landing coordinate rec of the j-th color pixel area in the t-th frame new The area and the paw landing coordinates rec of the j-th color pixel area in the previous frame old The intersection of areas and union ratio iou, if iou>0.1, it is considered that rec new With rec old Same, using rec old As the paw landing coordinates of the j-th color pixel area in the t-th frame, otherwise rec new This is a new claw landing coordinate, using rec new The paw landing coordinates of the j-th color pixel area in the t-th frame.

4. The mouse gait analysis method based on color regionalization according to claim 3, characterized in that: The method for distinguishing the mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs in combination with the spatial logic algorithm includes: re-establishing the coordinate system for the mouse entering and leaving the region of interest, when the horizontal coordinate M of the center of mass is x When the distance from the boundary is less than N pixels, the horizontal coordinate M of the centroid x The line connecting the horizontal coordinates of the convex hull points on the side of the boundary farthest from the mouse is the distinguishing line; The method of obtaining the final classification result and the coordinates of the footprint outer rectangular box through the short-term interference elimination algorithm and the four-phase cross-validation algorithm includes: The short-term interference elimination algorithm is as follows: Calculate the color matrix of the first detection record, and when the number of consecutive appearances of the j-th color is less than n frames, determine that the j-th color is a short-term interference, and eliminate the j-th color; The four-phase cross-validation algorithm is as follows: The adjacent rectangles of the coordinate system of each frame of the foreground image are cross-validated. If the overlapping area ratio of the adjacent rectangles is greater than 0.1, the rectangle with a larger area is retained and the rectangle with a smaller area is eliminated as a false positive.

5. The mouse gait analysis method based on color regionalization according to claim 4, characterized in that: The method for performing regional coloring for each frame of image according to the result of the first detection includes: In the foreground image area of ​​each frame, the non-black pixels corresponding to the mouse paw are replaced with the specified color according to the color matrix, and then the original image is converted into the HSV space to obtain a colored image containing only the paw, and the regionalized image with only the pixels of the i-th color is extracted. The regionalized image is binarized and converted into a binary regionalized image, in which the pixel part containing the i-th color is white and the background is black. The noise in the binary regionalized image is removed to obtain the eroded image, and the coordinates of the maximum circumscribed rectangle of the specified paw in the eroded image are calculated by the expansion area calculation comparison method.

6. The mouse gait analysis method based on color regionalization according to claim 5, characterized in that: The method for calculating the coordinates of the maximum circumscribed rectangle of a designated claw in the eroded image by the expansion area calculation comparison method includes: Dilate each frame of the eroded image so that the expanded area is larger than the area of ​​the specified claw; find the largest contour and calculate the maximum enclosing rectangle; set the maximum enclosing rectangle of the eroded image of the current frame rec cur =[x,y,w,h] and the maximum enclosing rectangle of the previous eroded image rec old =[x old ,y old ,w old ,h old ] to compare and calculate the intersection-and-union ratio iou; if iou>0.1, it means that the mouse's designated paw has touched the ground, and the maximum enclosing rectangle of the current frame becomes rec cur =[x old ,y old ,w old ,h old ]; add 1 to the number of frames of the specified claw’s landing time; if iou≤0.1, record the maximum bounding rectangle of the frame where the specified claw appears in the current frame’s eroded image as rec old =[x, y, w, h], and analyze the motion data of the specified paw; the motion data include maximum touchdown time, support time, footprint area, maximum footprint area, braking time, propulsion time, swing time, swing speed, maximum footprint length, maximum footprint width, step length of walking cycle and instantaneous speed.

7. The mouse gait analysis method based on color regionalization according to claim 6, characterized in that: The method of analyzing the maximum ground contact time of the specified paw comprises: Initialize the maximum landing time of the mouse to 0, count the number of corrosion image frames in which the designated paw of the mouse is continuously in the landing state, divide the number of corrosion image frames by the frame rate to obtain the landing time, use the landing time as the support duration, compare the landing time with the recorded maximum landing time of the designated paw, if the landing time is greater than the maximum landing time, update the landing time to the maximum landing time, otherwise do not update the maximum landing time; Methods for analyzing the maximum footprint area for a given paw include: Initialize the maximum footprint area of ​​the mouse's designated paw to 0, count the number of non-black pixels in the maximum circumscribed rectangle, and multiply the number of non-black pixels by the eroded image scale to obtain the value as the footprint area of ​​the mouse's designated paw; compare the footprint area with the maximum footprint area, if the footprint area is larger than the maximum footprint area, mark the mouse in the propulsion state, and update the footprint area to the maximum footprint area, otherwise mark the mouse in the braking state, and do not update the maximum footprint area; Methods for analyzing the braking and propulsion durations of a given claw include: The number of corrosion image frames continuously marked as the advancing state is counted, and the number of corrosion image frames continuously marked as the advancing state is divided by the frame rate to obtain the advancing time; the number of corrosion image frames continuously marked as the braking state is counted, and the number of corrosion image frames continuously marked as the braking state is divided by the frame rate to obtain the braking time; Methods for analyzing the swing duration and swing speed of a given paw include: The time when the designated paw leaves the ground for the s+1th time is recorded, and subtracted from the time when the designated paw touches the ground for the sth time. The time difference obtained is recorded as the swing duration, and the value obtained by dividing the step length by the swing time is taken as the swing speed.

8. The mouse gait analysis method based on color regionalization according to claim 7, characterized in that: The method of analyzing the length of the maximum footprint and the width of the maximum footprint of the specified paw includes: Initialize the maximum contour of the specified claw to 0, count the number of non-black pixels on the outermost edge of the eroded image, compare the number of non-black pixels on the outermost edge with the recorded maximum contour, if the number of non-black pixels on the outermost edge is greater than the maximum contour, update the number of non-black pixels on the outermost edge to the maximum contour, otherwise do not update the maximum contour; traverse all non-black pixels on the maximum contour, calculate the Euclidean distance between any two points on the maximum contour, mark the two non-black pixels with the largest Euclidean distance as length points, and use the Euclidean distance between the length points as the length of the maximum footprint; Connect the length points to obtain the length point direction vector, calculate the perpendicular vector of the length point direction vector in the plane where the maximum contour is located by the vector method, mark the two non-black pixel points with the largest Euclidean distance between the two non-black pixel points passing through the perpendicular vector as width points, and use the Euclidean distance between the width points as the width of the maximum footprint; Methods for analyzing step length and instantaneous velocity of a given paw’s walk cycle include; Record the first landing time of the specified paw in the current landing state, subtract it from the first landing time of the specified paw in the previous landing state, obtain the walking cycle, and record the position coordinates of the two landing states; the value obtained by subtracting the two position coordinates is used as the step length; the value obtained by dividing the step length by the walking cycle is used as the instantaneous speed.

9. The mouse gait analysis method based on color regionalization according to claim 8, characterized in that: The method for calculating the gait analysis data of the specified paw by the specified color comprises: Extract all motion data corresponding to the specified color, count all support time, footprint area, braking time, propulsion time, swing time, swing speed, step length and instantaneous speed of the walking cycle, and calculate the average value to obtain the average support time, average footprint area, average braking time, average propulsion time, average swing time, average swing speed, average step length and average instantaneous speed.

10. A mouse gait analysis method based on color regionalization, implementing the mouse gait analysis method based on color regionalization according to any one of claims 1 to 9, characterized in that: include: Data acquisition module: used to preset a closed walking platform, with a side-light footprint enhancement device built in the bottom of the closed walking platform, demarcate the area of ​​interest of the mouse, and use a high-definition camera to collect a background picture of the closed walking platform excluding the mouse. The shooting angle and shooting height of the background picture are the same as the shooting angle and shooting height of the subsequent natural gait video. No coercive measures are taken on the mouse, and the mouse is allowed to move freely in the closed walking platform. The high-definition camera is turned on to continuously shoot the mouse, and a green LED light is used as the light source of the side-light footprint enhancement device, which enters the glass side of the side-light footprint enhancement device, and a natural gait video containing the green fluorescent footprint of the mouse is obtained based on the footprint light refraction technology; The first detection module is used to perform the first detection on the natural gait video. Each frame of the natural gait video is subtracted from the background image pixel by pixel through the background subtraction technology to obtain a foreground image set containing only mice. A color regionalization coordinate system is established based on the foreground image set. The mouse paw to which the green fluorescent footprint in each frame of the foreground image belongs is distinguished by the spatial logic algorithm. The short-term interference rejection algorithm and the four-phase cross-validation algorithm are used for detection to obtain the final classification result and the coordinates of the footprint circumscribed rectangular box. The second detection module is used to perform a second detection on the natural gait video, regionalize and color each frame of the image based on the results of the first detection, and calculate the gait analysis data of the specified paw by specifying the color.

Images