Computation device of detecting scratching behavior for mice and method thereof

Abstract

A computation method of detecting scratching behavior for mice includes inputting a plurality of frames of a video to an artificial neural network model to obtain a scratching condition of each frame, and taking the frame having the scratching condition being positive as a scratching frame; computing a section-frame quantity of a frame section between a starting frame to a current frame and a scratching-frame quantity of the scratching frames during the frame section; determining a scratching event of the frame section according to a ratio between the scratching-frame quantity and the section-frame quantity; and computing a total scratching quantity of the scratching behavior of mice based on the scratching event detected in the video.

Description

Technical Field

[0001] This case relates to a computing device and a computing method, and more particularly to a computing device and a computing method for detecting mouse scratching behavior. Prior Art

[0002] In current biological and medical research, behavioral models using laboratory mice are often used to study neurological, dermatological, and other pathological conditions. One behavioral pattern researchers often use is the scratching behavior of laboratory mice. For example, laboratory mice scratch in response to itch or pain, and the frequency of scratching varies depending on their physiological condition. Changes in scratching frequency after administration of a test drug can help researchers understand the drug's effect on the pathology of a specific research topic.

[0003] To determine the number of scratches by laboratory mice, manual counting or image detection are the main methods. However, manual counting is time-consuming, and different personnel use different counting standards. While image detection can save manual counting time, the current image detection mechanism cannot accurately determine the number of scratches by laboratory mice, resulting in poor accuracy of detection results.

[0004] Therefore, how to improve the accuracy of detecting the number of scratching times of laboratory mice is an issue worthy of attention. Summary of the Invention

[0005] One embodiment of the present invention provides a computing device for detecting mouse scratching behavior, including a memory and a processor. The memory is configured to store a video including multiple image frames, a neural network model, and multiple program codes. The processor is coupled to the memory and configured to execute the multiple program codes to perform the following operations: input the multiple image frames into the neural network model, determine the scratching status of each of the multiple image frames using the neural network model, and determine the image frames with a scratching status as yes as scratch frames; calculate the interval frame number of the image interval between the starting frame and the current frame and the scratch frame number of the scratch frames in the image interval, where the starting frame is the first of N consecutive scratch frames, the current frame is the currently determined image frame, and N is a positive integer; determine the scratching events in the image interval based on the ratio of the scratching frame number to the interval frame number; and calculate the total number of scratching events of the mouse scratching behavior from the scratching events detected in the video.

[0006] Another embodiment of the present invention provides a computational method for detecting mouse scratching behavior, executed on a computing device. The computational method includes: inputting multiple video frames into a neural network model, determining the scratching status of each of the multiple frames using the neural network model, and determining image frames with a positive scratching status as scratch frames; calculating the interval frame number of an image interval between a starting frame and a current frame, and the scratch frame number of each scratch frame in the image interval, where the starting frame is the first of N consecutive scratch frames, the current frame is the currently detected image frame, and N is a positive integer; determining scratching events in the image interval based on the ratio of the scratching frame number to the interval frame number; and calculating the total number of scratching events in the mouse scratching behavior from the scratching events detected in the video. Simple diagram description

[0007] FIG1 is a block diagram of a computing device for detecting mouse scratching behavior according to one embodiment of the present invention. FIG. 2 is a schematic diagram illustrating detecting a mouse scratching behavior using multiple image frames in a video according to an embodiment of the present invention. FIG. 3 is a flow chart of a method for detecting mouse scratching behavior according to an embodiment of the present invention. FIG. 4 is a schematic diagram illustrating evaluating the training results of a neural network model through sample videos according to another embodiment of the present invention. FIG. 5 is a flow chart of a computational method for detecting mouse scratching behavior according to another embodiment of the present invention. Implementation Method

[0008] The present invention will be further described below with reference to the drawings and examples so that those skilled in the art can better understand the present invention and implement it accordingly. However, the examples are not intended to limit the present invention.

[0009] To facilitate understanding of the present case, the environment for detecting mouse scratching behavior can be one in which each laboratory mouse (hereinafter referred to as a mouse) is housed in a single cage. In one embodiment, a camera is positioned approximately above the cage, shooting downward, at an angle that allows for a complete capture of the mouse's limb movements.

[0010] In one embodiment, the user can place four cages adjacent to each other so that a single frame captures all four cages (i.e., one mouse in each of the four quadrants of the frame). Before detecting mouse scratching behavior, four segments of the four cages are segmented to create four frame segments, each containing only one mouse. Subsequent detection of mouse scratching behavior is performed on each of the four segments. During interpretation, the scratching behavior of the same mouse is determined using the detection results from segments in the same quadrant. In this embodiment, a single video (covering four mice) is divided into four segments for separate detection, resulting in the scratching behavior of each of the four mice. This allows users to capture multiple mice from the same camera, saving hardware costs while detecting the scratching behavior of multiple mice, providing a highly efficient detection environment.

[0011] To simplify the description, the following embodiments are based on a video that covers a mouse or a frame block that covers a mouse after the video is segmented.

[0012] FIG1 is a block diagram of a computing device for detecting mouse scratching behavior according to one embodiment of the present invention.

[0013] As shown in FIG1 , a computing device 100 for detecting mouse scratching behavior includes a processor 110 and a memory 120 . The processor 110 is coupled to the memory 120 .

[0014] In one embodiment, the storage 120 is configured to store a video of scratching behavior to be detected, a neural network model for detecting the scratching behavior of a mouse, and a plurality of program codes.

[0015] In one embodiment, the processor 110 is configured to execute a neural network model and a plurality of program codes to perform scratching behavior calculations and obtain the scratching status of all image frames in the video.

[0016] In one embodiment, the neural network model is a trained model. In practical applications, the neural network model receives a video and, after performing model calculations, outputs a detection result for each frame of the video (e.g., a "yes" scratching status indicates that the mouse in that frame has scratched; a "no" scratching status indicates that the mouse in that frame has not scratched) and a corresponding confidence value for that detection result.

[0017] Neural network-like models include object detection algorithms (such as You Only Look Once (including various versions of YOLO, optionally using YOLOv4, YOLOv7, or other versions) and Deformable Transformers (including Deformable DETR). For detailed descriptions, please refer to the document Zhu et al. (2021). Deformable DETR: Deformable Transformers for End-to-End Object Detection).

[0018] Due to biological mechanisms, a mouse's scratching behavior occurs randomly. Furthermore, once a mouse begins scratching, it may persist for several seconds or occur intermittently. In some cases, algorithms treat intermittent scratching as a single instance of scratching; in other cases, algorithms use the total number of image frames in which scratching is detected to determine a single instance of scratching. However, these methods cannot definitively determine whether a mouse's scratching behavior is triggered by the same or multiple consecutive neural signals, or by intermittent neural signals, and therefore cannot achieve accurate detection results.

[0019] To improve the accuracy of the detection results, the processor 110 of the computing device 100 of the present invention performs the following operations to achieve accurate detection.

[0020] In one embodiment, processor 110 inputs a video of scratching behavior to be detected into a neural network model. The neural network model then determines the scratching status of each of the multiple frames in the video. A "yes" scratching status indicates scratching behavior; a "no" scratching status indicates no scratching behavior.

[0021] The processor 110 will, based on the detection result of each frame, identify the image frame with a scratch condition as a scratch frame. Therefore, the processor 110 obtains the timestamps of the frames marked as scratch frames in the video.

[0022] Please refer to FIG. 2 , which is a schematic diagram illustrating detecting a mouse scratching behavior using multiple image frames in a video according to an embodiment of the present invention.

[0023] As shown in Figure 2, after the neural network model is run, the video 200 is labeled with multiple scratching frames. The straight line in the video 200 represents the scratching condition, which is the position marked as the scratching frame.

[0024] The processor 110 reads the scratch status of the image frames one by one based on the order of the time axis, and determines whether there are multiple consecutive image frames that are scratch frames.

[0025] In one embodiment, the processor 110 uses the current frame to indicate the position of the currently interpreted image frame. For example, the processor 110 uses the current frame to sequentially interpret image frame 211, starting from image frame 201 (the starting frame) based on the timeline. In the embodiment of FIG2 , the scratching condition changes from "no" to "yes" at image frame 211. In this case, image frame 211 is used as the starting frame, meaning that the processor 110 sets the image frame at which the scratching condition changes from "no" to "yes" as the starting frame.

[0026] In one embodiment, when the processor 110 determines that N consecutive scratch frames have occurred (using the current frame to determine based on the advancement of the time axis), the scratch event confirmation procedure is initiated.

[0027] In the scratch event confirmation process, the first of N consecutive scratch frames (N is a positive integer) is used as the starting frame. Then, the processor 110 calculates the interval frame number of the image interval between the starting frame and the current frame and the scratch frame number of the scratch frames in the image interval.

[0028] For example, if the current frame is at image frame 213 (i.e., the fifth frame from image frame 211), processor 110 can calculate the number of frames in image interval 231 between starting frame 211 and the current frame (image frame 213) as 5, and count the number of scratched frames in image interval 231 as 5. After the current frame has been advanced several times, if the current frame is at image frame 215 (i.e., the 23rd frame from image frame 211), processor 110 can calculate the number of frames in image interval 233 between starting frame 211 and the current frame (image frame 215) as 23, and count the number of scratched frames in image interval 231 as 17. After the current frame is advanced several times, if the current frame is at the position of image frame 217 (i.e., the 33rd frame starting from image frame 211), the processor 110 can calculate the number of interval frames of image interval 235 between the starting frame 211 and the current frame (image frame 217) as 33 frames, and count the number of scratch frames in image interval 231 as 17 frames.

[0029] Next, the processor 110 calculates the ratio of the scratching frame number to the interval frame number to determine the scratching events in the image interval. The processor 110 calculates the total number of scratching behaviors of the mouse from the scratching events detected in the video.

[0030] For example, if the current frame is at image frame 213, the ratio of the number of scratched frames to the number of interval frames is 1 (i.e., 5 / 5). If the current frame is at image frame 215, the ratio of the number of scratched frames to the number of interval frames is 0.73 (i.e., 17 / 23). If the current frame is at image frame 217, the ratio of the number of scratched frames to the number of interval frames is 0.51 (i.e., 17 / 33).

[0031] In one embodiment, processor 110 calculates the ratio of the number of scratched frames to the number of frames in the interval. When the ratio is greater than a scratch threshold, processor 110 advances the current frame to the next image frame. This ratio is calculated for the image interval between the start frame and the current frame, and the ratio between the number of scratched frames and the number of frames in the interval is calculated until the ratio is no greater than the threshold. If the ratio is no greater than the scratch threshold, processor 110 determines the last detected scratched frame in the image interval as the end frame, and the image interval between the start frame and the end frame as a scratch event.

[0032] For example, assume the scratch threshold is 0.52. As shown in FIG2 , if the current frame is at the position of the frame preceding image frame 217, the ratio of the number of scratch frames to the number of interval frames is 0.53 (i.e., 17 / 32). Since this ratio is 0.53, which is greater than the scratch threshold, the current frame advances to image frame 217. However, when the current frame is at image frame 217, the ratio is 0.51, which is less than the scratch threshold. At this point, processor 110 uses the last detected scratch frame in image interval 235, image frame 215, as the ending frame and deems image interval 233 between image frames 211 and 215 a scratch event. In other words, in this scratch event, there are 17 frames determined to contain scratches and 23 frames determined to contain no scratches, and this scratch event occurs in image interval 233 of video 200.

[0033] Processor 110 continues detecting the next frame of the video until the video ends. For example, following image frame 217, processor 110 continues performing the above-described operations. To avoid excessive repetition, in this embodiment, processor 110 detects another scratch event, with image frame 221 being the initial frame. The scratch event confirmation process is initiated at image frame 223. At image frame 227, the ratio is less than the scratch threshold. The last scratch frame detected in image interval 245, image frame 225, is used as the termination frame, and image interval 243 between image frames 221 and 225 is considered another scratch event. In other words, processor 110 detects two scratch events in video 200, one in image interval 233 and one in image interval 243. Therefore, processor 110 calculates the total number of scratches as two.

[0034] In one embodiment, the computing device 100 further includes a graphical user interface (GUI). The GUI is configured to display the scratching status of the video 200 on a timeline. For example, the scratching status can be represented by a line as shown in FIG. 2 , or by displaying scratched frames in yellow and non-scratched frames in purple.

[0035] In another embodiment, the video 200 can be stretched or shortened in the graphical user interface. The scratching status displayed in the graphical user interface includes displaying the scratching frequency of the merged multiple video frames using different colors. For example, the video 200 is displayed in the graphical user interface as five frames merged into one. If all five merged frames are scratched, the merged position can be displayed in a brighter yellow color.

[0036] The graphical user interface can be an interface provided by a web application, and the user uploads the video or connects to the camera through the browser program. After the computing device 100 performs all the operations, the final results are presented through the graphical user interface.

[0037] In the aforementioned embodiment where a single frame covers four cages, before detecting scratching behavior, four sections of the frame representing the four cages are cut out to produce four frame blocks. The image of each frame block is then used as a video input into the neural network model to calculate the total number of scratches.

[0038] In another embodiment, the computing device 100 further includes a camera (not shown). The camera is coupled to the storage 120 and temporarily stores the captured video in the storage 120. The processor 110 can detect the scratching behavior of the mouse in the video in near real time and calculate the total number of scratches.

[0039] FIG3 is a flow chart of a method for detecting a mouse scratching behavior according to an embodiment of the present invention. The method for detecting a mouse scratching behavior in FIG3 can be executed by the computing device 100 in FIG1 .

[0040] In step S310, the processor 110 inputs a plurality of image frames of a video into the neural network model, and determines the scratching status of each image frame by the neural network model.

[0041] If the neural network model determines that the scratching condition is yes, it means that the mouse in the image frame has scratching behavior; if the neural network model determines that the scratching condition is no, it means that the mouse in the image frame has no scratching behavior.

[0042] In step S320, the processor 110 calculates the interval frame number of the image interval from the start frame to the current frame and the scratch frame number of the scratch frame in the image interval.

[0043] In one embodiment, the processor 110 labels the image frames with a scratch status as scratch frames according to the detection result of each frame, and calculates the number of scratch frames in the video 200 that are labeled as scratch frames.

[0044] In step S330, the processor 110 determines a scratch event in the image interval according to the ratio of the scratch frame number to the interval frame number.

[0045] Processor 110 interprets the scratching status of each image frame in a timeline order. The image frame currently being interpreted by processor 110 is called the current frame. Processor 110 begins interpreting the image from the initial frame and switches to the next frame after each interpretation.

[0046] In one embodiment, the current frame read by processor 110 is the Kth frame from the start frame, where K is a positive integer. Processor 110 calculates the number S (number of scratch frames) marked as scratch frames within these K frames (number of interval frames) and calculates the ratio S / K of the number of scratch frames to the number of interval frames.

[0047] When processor 110 determines that the ratio is greater than the scratch threshold, it advances the current frame to the next image frame and calculates the ratio of the number of scratch frames to the number of interval frames in the image interval between the start frame and the current frame (e.g., the K+1 frame) until the ratio is no greater than the threshold. If processor 110 determines that the ratio is no greater than the scratch threshold, it sets the last detected scratch frame in the image interval as the end frame and treats the image interval between the start and end frames as a scratch event.

[0048] The processor 110 continues to detect the next frame of the video until the video ends.

[0049] In step S340, the processor 110 calculates the total number of scratching events from the scratching events detected in the video.

[0050] In one embodiment, one scratching event detected in step S330 is counted as one scratching, and thus the total number of scratching events can be calculated from all detected scratching events.

[0051] In order to detect the mouse scratching behavior and calculate the correct number of scratches, the computing device 100 of this case uses a plurality of sample videos to train and test the neural network model before detection.

[0052] In one embodiment, the storage 120 stores a plurality of sample videos, such as pre-recorded videos of mice.

[0053] Processor 110 pre-sets each sample video with a real scratching event. A sample video is a video of a mouse with a real scratching event labeled. A real scratching event refers to a sequence of consecutive image frames within a real scratching segment of the video that are assigned to the mouse's scratching behavior. Each sample video may contain one or more real scratching events depending on the actual situation. To simplify the description, this application assumes that each sample video has one real scratching event. However, this application does not limit the number of real scratching events in each sample video.

[0054] FIG. 4 is a schematic diagram illustrating evaluating the training results of a neural network model through sample videos according to another embodiment of the present invention.

[0055] As shown in FIG4 , sample video 410 includes multiple sample image frames (the horizontal axis corresponds to the time axis, and the vertical axis corresponds to a single image frame). Sample video 410 is pre-set with a real scratching event, such as a real scratching segment between time T1 and time T2. The scratching behavior of the mouse in the real scratching event is annotated as event image frame 413 , with the scratching-annotated image frames represented by straight lines in FIG4 .

[0056] In one embodiment, the processor 110 inputs a plurality of sample image frames of the sample video 410 into the neural network model to output a test result 420 for each sample image frame, wherein the test result includes a confidence value of the neural network model for the test result.

[0057] In one embodiment, when the test result is scratching and the confidence value is greater than a standard value, the processor 110 marks the sample image frame as a detected scratching image frame 423, as shown by the straight line stripes in FIG4. The standard value can be a preset value used to evaluate the reliability of the test result.

[0058] Furthermore, to evaluate the operation of the neural network model, the processor 110 calculates the number of scratching image frames 423 detected in the actual scratching segment labeled as scratching, such as from time T1 to time T2, and the number of event image frames 413 of the actual scratching event to obtain a detection rate for the actual scratching event. When this detection rate exceeds a detection threshold, the processor 110 counts the number of detected scratches once.

[0059] In one embodiment, the detection rate may be a ratio between the number of the capture image frames 423 and the number of the event image frames 413 .

[0060] For example, the number of event image frames 413 in the actual scratch segment is 58, and the number of scratch image frames 423 is 45, so the detection rate is 0.78. If the detection threshold is 0.7, the processor 110 counts the number of detected scratches as 1.

[0061] It's worth noting that a true scratching segment refers to image frames within a specific time interval within a sample video. In other words, the detection rate is a reference value used to evaluate whether the neural network model correctly detects scratching within the same image frame. Specifically, if the correct detection rate in a true scratching segment exceeds the detection threshold, the processor 110 will classify the detection as a scratch.

[0062] In one embodiment, the processor 110 performs the same test on all sample videos to obtain the number of scratches in all sample videos to obtain the total number of detected scratches. The processor 110 evaluates the training results of the neural network based on the total number of detected scratches.

[0063] In one embodiment, processor 110 calculates the ratio of the total number of detected scratches to the actual scratching base number and determines whether the ratio exceeds a sample threshold. If the ratio exceeds the sample threshold, it indicates that the neural network model's detection quality for the multiple sample videos inputted this time meets the requirements. Therefore, processor 110 determines that the training results are excellent and can deploy the neural network model in a real-world setting. Otherwise, processor 110 continues training the neural network model until excellent training results are achieved.

[0064] In one embodiment, the neural network model outputs scratch detection results for each sample image frame in each sample video (e.g., whether each frame indicates scratching or not). Because the actual scratching segments in the sample videos are pre-labeled, the processor 110 can obtain the locations of the image frames with actual scratching. Therefore, the processor 110 can use this information to compare the actual scratching segments with the output of the neural network model frame by frame to obtain the actual prediction performance of the neural network model. This actual prediction performance can be used as another parameter to evaluate the training success of the neural network model.

[0065] For example, a sample video contains 10 frames, and the actual scratching status in the timeline order is NNNYYYNYNN (where Y represents frames with actual scratching and N represents frames without actual scratching). If the same sample video is input into a neural network model, and the neural network model's detected scratching information is NNYYYYNNNN, the processor 110 will know that two frames (frames 3 and 8) were incorrectly detected. Therefore, the actual prediction performance of the neural network model is 0.8.

[0066] FIG5 is a flow chart of a computational method for detecting mouse scratching behavior according to another embodiment of the present invention. The computational method of FIG5 can be executed by the computing device 100 of FIG1 .

[0067] In step S510, the storage 120 stores a plurality of sample videos and pre-sets a real scratching event for each sample video.

[0068] In one embodiment, the real scratching segment of the real scratching event is at least N consecutive image frames that have been pre-labeled with a scratching condition.

[0069] In step S520, the processor 110 inputs a plurality of sample videos into the neural network model to output test results of a plurality of sample image frames of each sample video.

[0070] For example, the neural network model detects the scratching condition of each sample image frame of the sample video as a test result, such as scratching or no scratching.

[0071] In step S530, the processor 110 marks the sample image frames whose test results indicate scratching and whose confidence values ​​are greater than the standard value as detected scratching image frames.

[0072] In one embodiment, the processor 110 labels all sample image frames of the sample video to obtain a test result for each sample image frame.

[0073] In step S540, the processor 110 calculates the number of scratch detection image frames and the number of event image frames of real scratch events in the real scratch segment to obtain a detection rate of real scratch events.

[0074] In one embodiment, the actual scratch segment is a known time interval in the sample video. Processor 110 uses the same time interval as the basis for comparing actual scratch events with detected scratch events. For example, the actual scratch segment is from the 5th to the 7th second of the sample video. Therefore, processor 110 calculates the ratio of the number of scratch image frames in the detected scratch event to the number of event image frames in the actual scratch segment, using the time interval from the 5th to the 7th second. This ratio is the detection rate.

[0075] In step S550, the processor 110 determines whether the detection rate is greater than the training threshold value. If so, step S560 is executed; if not, step S570 is executed.

[0076] In step S560, the processor 110 counts a number of detected scratches.

[0077] In one embodiment, if the correct detection in the actual scratching section is higher than the detection threshold, the processor 110 may classify the detection as a scratch.

[0078] In step S570, the processor 110 determines whether the inspection of all sample videos has been completed. If not, execute step S580;

[0079] In step S580, the processor 110 sums up the number of detection scratches of multiple sample videos to obtain the total number of detected scratches.

[0080] In step S590, the processor 110 evaluates the training results of the neural network-like model based on the total number of detected scratches. If the training results are excellent (e.g., the ratio of the total number of detected scratches calculated by the processor 110 to the true scratch base is greater than the sample threshold value), the neural network-like model can be employed in the real field domain.

[0081] In one embodiment, the training of a neural network-like model can be implemented using a training mechanism using k-fold cross-validation. For example, multiple sample videos are divided into three groups of videos, namely training video, validation video and test video, and each sample video will be the role of training video, validation video and test video in different cross-validation rounds to effectively use the limited dataset, avoid overfitting problems and improve model generalization.

[0082] In one embodiment, the computing method may be implemented as a computer program and stored in a non-transient computer-readable record media, and such that the processor 110 performs the computing method for detecting mouse scratching behavior after reading this non-transient computer-readable record media.

[0083] In one embodiment, the processor 110, for example, is, but is not limited to, a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a central processing unit (CPU), a graphics processor (Graphic Processing Unit, GPU), a system array single-chip (System on Chip, SoC gateable). FPGA), Network Processor chip or a combination of the above components.

[0084] In one embodiment, the storage 120 is, for example, but not limited to Random Access Memory (RAM), Flash memory (Flash memory), Read Only Memory (ROM), Hard Disk Drive (HDD), Solid State Drive (SSD), optical storage, or a combination of the above components.

[0085] In summary, a computing device for detecting mouse scratching behavior and a computing method for detecting mouse scratching behavior is proposed to detect the scratching status of all image frames of the video. Compared to existing practices, this case can solve the problem of the number of miscalculations.

[0086] The above are only specific examples of this case, and do not therefore limit the scope of the patent application in this case. Therefore, all equivalent changes made using the contents of this case are similarly included in the scope of this case and are subject to statement.

[0087] 100: Computing Devices 110: Processor 120: reservoir 200: Video 201, 211, 213, 215, 217: image frame 231, 233, 235: image interval 221, 223, 225, 227: image frames 243, 245: image interval 410: Sample video 413: Event image frame 420: Test results 423: Testing the scratch image frame T1, T2: time S310~S340, S510~S590: Steps

[0088] Domestic Hosting Information (Please note in order of hosting institution, date, number) None Overseas hosting information (please note in order of hosting country, institution, date, number) None

Claims

1. A computing device for detecting mouse scratching behavior, comprising: A memory configured to store a video including a plurality of image frames, a neural network model, and a plurality of program codes; And a processor coupled to the memory and configured to execute the multiple program codes to perform the following operations: input the multiple image frames into the neural network model to determine a scratching status of each frame of the multiple image frames by the neural network model, and use the image frame with the scratching status as yes as a scratch frame; calculate an interval frame number of an image interval between a starting frame and a current frame and a scratch frame number of the scratch frame in the image interval, wherein the starting frame is the first of N consecutive scratch frames, the current frame is the image frame currently judged and N is a positive integer; determine a scratching event in the image interval based on a ratio of the scratching frame number to the interval frame number; and calculate a total scratching count of the mouse scratching behavior from the scratching event detected in the video.

2. A computing device as described in claim 1, wherein the processor is further configured to perform the following operations: when the ratio is greater than a scratch threshold value, advance the current frame to the next image frame, and calculate the ratio of the number of scratch frames in the image interval between the start frame and the current frame to the number of frames in the interval until the ratio is no greater than the scratch threshold value; and when the ratio is no greater than the scratch threshold value, treat the last scratch frame detected in the image interval as an end frame, and treat the image interval between the start frame and the end frame as a scratch event.

3. The computing device of claim 1, further comprising a graphical user interface (GUI), wherein the processor is further configured to perform the following operations: display the scratching status of the video on a timeline on the GUI.

4. The computing device of claim 3, wherein the scratching status displayed on the graphical user interface comprises displaying a scratching frequency of the plurality of image frames in different colors.

5. A computing device as described in claim 1, wherein the processor is further configured to perform the following operations: store a plurality of sample videos and pre-set a real scratching event for each of the plurality of sample videos; input the plurality of sample videos into the neural network model respectively to output a test result for each of the plurality of sample image frames of each of the plurality of sample videos, wherein the test result includes a confidence value; and among the plurality of sample image frames of each of the plurality of sample videos, mark the sample image frame whose test result is scratching and whose confidence value is greater than a standard value as a detected scratching image frame.

6. A computing device as described in claim 5, wherein the processor is further configured to perform the following operations: based on each of the multiple sample videos, based on a real scratch segment marked as scratching in the real scratch event, calculate the number of detected scratch image frames and the number of event image frames of the real scratch event in the real scratch segment to obtain a detection rate of the multiple real scratch events; and when the detection rate is greater than a detection threshold value, count a number of detected scratches.

7. The computing device of claim 6, wherein the processor is further configured to: sum the number of detected scratches of the plurality of sample videos to obtain a total number of detected scratches; and evaluate the training results of the neural network model based on the total number of detected scratches.

8. A computing method for detecting mouse scratching behavior, executed on a computing device, comprising: A plurality of image frames of a video are input to a neural network model, and a scratching status of each of the plurality of image frames is determined by the neural network model, and the image frame in which the scratching status is yes is regarded as a scratch frame; an interval frame number of an image interval between a starting frame and a current frame and a scratch frame number of the scratch frame in the image interval are calculated, wherein the starting frame is the first of N consecutive scratch frames, the current frame is the image frame currently read, and N is a positive integer; a scratching event is determined in the image interval based on a ratio of the scratching frame number to the interval frame number; and a total scratching count of the mouse scratching behavior is calculated from the scratching event detected in the video.

9. The computing method of claim 8, wherein the step of determining the scratch event in the image interval according to the ratio of the scratch frame number to the interval frame number comprises: When the ratio is greater than a scratch threshold, the current frame is advanced to the next image frame, and the ratio of the number of scratch frames in the image interval between the starting frame and the current frame to the number of frames in the interval is calculated until the ratio is no greater than the scratch threshold; and when the ratio is no greater than the scratch threshold, the last scratch frame detected in the image interval is taken as an ending frame, and the image interval between the starting frame and the ending frame is taken as a scratch event.

10. The computing method according to claim 8, further comprising: The scratching status of the video is displayed on a timeline in a graphical user interface.

11. The computing method according to claim 10, further comprising: The scratching status displayed on the graphical user interface includes displaying a scratching frequency of the plurality of image frames in different colors.

12. The computing method of claim 8, further comprising: storing a plurality of sample videos and presetting a real scratching event for each of the plurality of sample videos; The plurality of sample videos are respectively inputted into the neural network model to output a test result for each of the plurality of sample image frames of each of the plurality of sample videos, wherein the test result includes a confidence value; and among the plurality of sample image frames of each of the plurality of sample videos, the sample image frame whose test result is scratching and whose confidence value is greater than a standard value is marked as a detected scratching image frame.

13. The computing method according to claim 12, further comprising: after marking the sample image frame as the scratch detection image frame; Based on each of the multiple sample videos, based on a real scratch segment marked as scratching in the real scratch event, the number of detected scratch image frames and the number of event image frames of the real scratch event in the real scratch segment are calculated to obtain a detection rate of the multiple real scratch events; and when the detection rate is greater than a detection threshold, the number of detected scratches is counted.

14. The algorithm according to claim 13, further comprising: summing up the detected scratching times of the plurality of sample videos to obtain a total detected scratching time; And the training results of the neural network model are evaluated based on the total number of detected scratches.

Images