Construction method and equipment of behavior logic anomaly auxiliary judgment model and storage medium

Abstract

The embodiment of the invention relates to the technical field of auxiliary judgment, and discloses a method and device for constructing a behavior logic anomaly auxiliary judgment model and a storage medium, and the method comprises the steps: carrying out the data analysis of video data of a living body, so as to obtain a plurality of time-continuous three-dimensional skeletons; performing action recognition on the three-dimensional skeleton, obtaining an action combination conversion probability parameter according to a time sequence, and marking the action combination conversion probability parameter as a corresponding emotion or disease state so as to establish a living body behavior feature training set; training a living body behavior logic anomaly auxiliary judgment model through the living body behavior feature training set to obtain a single living body behavior logic anomaly auxiliary judgment model corresponding to the current species, so that an emotion or disease state can be connected with an action combination conversion probability parameter; and determining the emotion or disease state of the living body through the motion combination conversion probability parameter.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of auxiliary judgment, and particularly to a method, device and storage medium for constructing an auxiliary judgment model for abnormal behavioral logic. Background Art

[0002] In the prior art, humans or animals will involuntarily exhibit some abnormal posture and body position characteristics due to pain, fear, developmental disorders, and some disease states. According to theory, it can be known that the state of humans or animals can be judged based on the above abnormal posture and body position characteristics. In animal experimental groups, the existing evaluation of animal body position and abnormal body postures requires experienced experimental personnel to apply different degrees of stimuli and describe according to the responses of animals to nociceptive stimuli, which highly depends on experience and is difficult to quantify, and there is currently a lack of objective quantitative evaluation means for abnormal behaviors. Summary of the Invention

[0003] In view of the above problems, the embodiments of the present invention provide a method, device and storage medium for constructing an auxiliary judgment model for abnormal behavioral logic, which are used to solve the technical problem of the lack of effective quantitative evaluation means for abnormal behaviors of living bodies in the prior art.

[0004] According to one aspect of the embodiments of the present invention, a method for constructing an auxiliary judgment model for abnormal behavioral logic is provided, and the method includes:

[0005] Step S1: For the same species, obtain video data of a living body in different emotions or different disease states for a first preset duration;

[0006] Step S2: Analyze the video data to obtain a plurality of temporally continuous three-dimensional skeletons;

[0007] Step S3: Perform action recognition on a plurality of temporally continuous three-dimensional skeletons to obtain an action recognition result, calculate action combination conversion probability parameters of various action combinations according to the time series, and mark the action combination conversion probability parameters as corresponding emotions or disease states, where the action combination is a combination of each action and another action that appears after each action;

[0008] Step S4: Repeat steps S1 - S3 to obtain the action combination conversion probability parameters marked as corresponding emotions and the action combination conversion probability parameters marked as corresponding disease states, and establish a training set of living body behavior characteristics based on this;

[0009] Step S5: Using the action combination conversion probability parameter as the input and the corresponding emotion or disease state as the output, train the auxiliary judgment model for abnormal living behavior logic through the living behavior feature training set to obtain the single-species auxiliary judgment model for abnormal living behavior logic corresponding to the current species.

[0010] In an alternative approach, the step of performing data analysis on the video data to obtain multiple temporally consecutive three-dimensional skeletons further includes:

[0011] Step S21: Perform ethological analysis on the video data to obtain multiple living three-dimensional skeletons arranged in chronological order;

[0012] Step S22: Extract pose features from the multiple three-dimensional skeletons and adjust the format to obtain multiple three-dimensional skeletons with unified size and orientation, and the multiple three-dimensional skeletons are temporally consecutive.

[0013] In an alternative approach, after the step of using the action combination conversion probability parameters of the multiple action combinations as the input and the corresponding emotion or corresponding disease state as the output to train the auxiliary judgment model for abnormal living behavior logic to obtain the single-species auxiliary judgment model for abnormal living behavior logic corresponding to the current species, it further includes:

[0014] Replace the species type for collecting the video data, repeat steps S1 - S5, and establish a single-species auxiliary judgment model for abnormal living behavior logic corresponding to another species;

[0015] Combine the single-species auxiliary judgment models for abnormal living behavior logic corresponding to different species to obtain a multi-species auxiliary judgment model for abnormal living behavior logic.

[0016] In an alternative approach, after the step of using the action combination conversion probability parameters of the multiple action combinations as the input and the corresponding emotion or corresponding disease state as the output to train the auxiliary judgment model for abnormal living behavior logic through the living behavior feature training set to obtain the single-species auxiliary judgment model for abnormal living behavior logic corresponding to the current species, it further includes:

[0017] Step S6: Repeat steps S1 - S3 to obtain the action combination conversion probability parameters of the multiple action combinations marked with the corresponding emotion and the action combination conversion probability parameters of the multiple action combinations marked with the corresponding disease state, and establish a living behavior feature verification set with the action combination conversion probability parameters of each group of multiple action combinations in the living behavior feature verification set being different from those in the living behavior feature training set;

[0018] Step S7: Verify the trained single-living-being behavior logic anomaly auxiliary judgment model according to the living-being behavior feature verification set until the loss function meets the preset threshold.

[0019] In an alternative manner, the action combination conversion probability parameters of the multiple action combinations include multiple different action combinations and the frequency of occurrence of each of the action combinations.

[0020] In an alternative manner, the multi-living-being behavior logic anomaly auxiliary judgment model includes:

[0021] A data preprocessing layer, configured to obtain video data of a living being to be analyzed, perform data processing on the video data to obtain real-time action combination conversion probability parameters of multiple real-time action combinations, and call the single-living-being behavior logic anomaly auxiliary judgment model corresponding to the species according to the species of the living being to be analyzed; and,

[0022] The single-living-being behavior logic anomaly auxiliary judgment model;

[0023] The single-living-being behavior logic anomaly auxiliary judgment model includes:

[0024] An input layer, configured to obtain the species of the living being to be analyzed and the real-time action combination conversion probability parameters of the multiple real-time action combinations;

[0025] A hidden layer, configured to input the real-time action combination conversion probability parameters of the multiple real-time action combinations into the called single-living-being behavior logic anomaly auxiliary judgment model;

[0026] An output layer, configured to output a corresponding emotion or disease state according to the single-living-being behavior logic anomaly auxiliary judgment model.

[0027] In an alternative manner, the emotion includes any one of fear, tension, anxiety, and depression, and the disease state includes any one of sensory deficits, neuropsychiatric diseases, and neurodegenerative diseases.

[0028] According to another aspect of the embodiments of the present invention, there is provided a device for constructing a behavior logic anomaly auxiliary judgment model, including:

[0029] A processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0030] The memory is configured to store at least one executable instruction, and the executable instruction causes the processor to execute the operations of the method for constructing the behavior logic anomaly auxiliary judgment model as described above.

[0031] According to another aspect of the embodiments of the present invention, a storage medium is provided. At least one executable instruction is stored in the storage medium. When the executable instruction runs on a device / apparatus for constructing a behavior logic anomaly auxiliary judgment model, it causes the device / apparatus for constructing the behavior logic anomaly auxiliary judgment model to perform the operations of the method for constructing the behavior logic anomaly auxiliary judgment model as described above.

[0032] The present invention constructs a brand-new correspondence between video data and 3D skeletons, analyzes action combination conversion probability parameters of multiple action combinations based on the 3D skeletons, and corresponds the action combination conversion probability parameters of multiple action combinations to the emotions or disease states of a living body, thereby constituting a training set of living body behavior characteristics with the action combination conversion probability parameters as the input and the corresponding emotions or disease states as the output. And based on the above training set of living body behavior characteristics, the behavior logic anomaly auxiliary judgment model for a living body is trained to obtain a single-living-body behavior logic anomaly auxiliary judgment model corresponding to the current species. Thus, through the above data analysis and model training process, emotions or disease states can be linked to behavior characteristics, and a single-living-body behavior logic anomaly auxiliary judgment model for determining the emotions or disease states of a living body through behavior logic anomaly analysis can be obtained, thereby solving the technical problem in the prior art of lacking effective quantitative evaluation means for the behavior anomalies of a living body.

[0033] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to be able to understand the technical means of the embodiments of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically given below. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings are only used to illustrate the embodiments and are not considered as a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0035] Figure 1 It shows a schematic flowchart of the first embodiment of the method for constructing a behavior logic anomaly auxiliary judgment model provided by the present invention;

[0036] Figure 2 It shows a schematic diagram of the process of video data collection and behavior analysis in the method for constructing a behavior logic anomaly auxiliary judgment model provided by the present invention;

[0037] Figure 2 -a shows a schematic diagram of the process of video data collection in the method for constructing a behavior logic anomaly auxiliary judgment model provided by the present invention;

[0038] Figure 2-b shows the behavioral analysis schematic diagram of determining the living body behavior according to the video data in the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0039] Figure 3 shows the schematic diagram of the three-dimensional skeleton reconstruction process in the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0040] Figure 3 -a shows the structural schematic diagram of the three-dimensional skeletons collected from multiple angles in the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0041] Figure 3 -b shows the statistical schematic diagram of determining the frequency of the sniffing action according to the video data in the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0042] Figure 3 -c shows the conversion schematic diagram of each action combination in the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0043] Figure 4 shows the schematic diagram of the process of the second embodiment of the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0044] Figure 5 shows the schematic diagram of the process of the third embodiment of the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0045] Figure 6 shows the structural schematic diagram of the living body behavioral logic anomaly auxiliary judgment model in the method for constructing the behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0046] Figure 7 shows the structural schematic diagram of the embodiment of the device for constructing the living body behavioral logic anomaly auxiliary judgment model provided by the present invention. Detailed Embodiments

[0047] Hereinafter, the exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0048] The following analyzes the method for judging behavioral anomalies in the prior art:

[0049] The normal behaviors of humans and animals follow a certain organizational logic. However, in cases of emotional abnormalities, neurological diseases, or brain injuries, the behavioral logic will be disrupted to varying degrees, especially the disruption of the basic behavioral logic for maintaining life. Due to the difficulty in defining and identifying animal movement actions, there is still a lack of detailed research on the organizational logic of animal basic behaviors.

[0050] Actions need to be arranged and combined according to a certain logic to form meaningful behaviors. For example, basketball includes actions such as dribbling, passing, and shooting. The organizational logic of behaviors will be affected by many factors such as past experience, one's own physiological state, and the environment, and involves species habits and decision-making, etc. However, for behaviors that meet the needs of basic survival and reproduction, they follow a relatively conservative behavioral logic. For example, animals will follow a certain logic to conduct spontaneous exploration in an open field; the behavior of avoiding natural enemies usually sequentially makes actions such as threat detection, risk assessment, escape initiation, re-assessing risks in a safe area, and then exploring out of the nest again. However, in some cases of brain injury or sensory loss, the behavioral logic of humans and animals will be disrupted. For example, some Alzheimer's patients will repeatedly ask medical staff for food due to forgetting; normal mice will immediately explore the environment after entering a strange open field, while autistic mice will quickly lose interest and groom themselves in the corner. However, due to the limitations of animal behavior analysis methods, previous studies have not analyzed animal behaviors at the action level, let alone analyzed the behavioral logic of animals based on actions.

[0051] This application provides a method, device, and storage medium for constructing an auxiliary judgment model for abnormal behavioral logic. By analyzing and processing image data and using the processed data for model training, it is possible to link the emotional or disease state with the action combination conversion probability parameter, so as to obtain a single-living-body behavioral logic abnormal auxiliary judgment model for determining the emotional or disease state of a living body through action combination analysis, thereby solving the technical problem in the prior art of lacking effective quantitative evaluation means for abnormal behaviors of living bodies.

[0052] In an implementation scenario, this solution is implemented based on a camera component and a computer component.

[0053] Among them, the camera component is used to obtain video data, and the computer component is used to execute the method for constructing an auxiliary judgment model for abnormal behavioral logic.

[0054] Figure 1 The flowchart of the first embodiment of the method for constructing an auxiliary judgment model for abnormal behavioral logic of the present invention is shown. This method for constructing an auxiliary judgment model for abnormal behavioral logic is executed by a device for constructing an auxiliary judgment model for abnormal living body behavioral logic. As Figure 1 shown, the method includes the following steps:

[0055] The construction method of the single-behavior logical anomaly auxiliary judgment model includes:

[0056] Step S1: For the same species, obtain video data of the living body in different emotions or different disease states within a first preset duration.

[0057] The above process can be realized by using multiple high-definition resolution cameras or high-definition resolution cameras with an infrared shooting mode to separately shoot the spontaneous behaviors of the same species from multiple angles. At this time, different living bodies have different emotions or different disease states. By replacing the acquisition object, the acquisition of different emotions or different disease states can be realized. The duration of the video data collected at this time can be set as needed. For example, the data acquisition time of a single living body, that is, the first preset duration, can be set between 15 min and 60 min, and multiple-angle acquisitions can be performed. As shown in the 4 perspectives shown in Figure 2 -a, in actual data acquisition, the number of perspectives can also be increased or decreased as needed.

[0058] For different species, based on morphological analysis, when each species faces different emotions or different disease states, due to differences in physiological structures, there are slight differences in the characteristics of the external environment perceived, and their behavioral tendencies will show obvious species limitations. For example: in a pain state, mice mostly show curling up, while humans mostly show covering the corresponding pain area. There are significant differences in their manifestation forms and video data. Therefore, only obtaining video data of the same species in different emotions or different disease states can ensure the consistency of the data.

[0059] In addition, when the species is an animal, the above steps can be carried out by constructing animal experimental models in different states on the premise of conforming to animal ethics, constructing experimental animals in different emotions or different disease states through the above scheme, and collecting relevant video data from them.

[0060] When the species is a human, with the consent of the acquisition object, different emotions or different disease states of the currently acquired object can be defined by collecting personal statements and doctor-assisted judgments, and relevant video data can be collected from them.

[0061] Optionally, the emotion includes any one of fear, tension, anxiety, and depression, and the disease state includes any one of sensory deficits, neuropsychiatric diseases, and neurodegenerative diseases.

[0062] Step S2: Perform data analysis on the video data to obtain multiple temporally continuous three-dimensional skeletons.

[0063] Since in a video, a living body is generally in a continuous motion state in the video data, its three-dimensional skeleton also continues to change over time. That is, a frame of image will obtain the information of a three-dimensional skeleton. There is a temporal sequence relationship among the multiple three-dimensional skeletons corresponding to a video data after data analysis.

[0064] Further, referring to Figure 4 as shown, the step of performing data analysis on the video data to obtain multiple temporally continuous three-dimensional skeletons further includes:

[0065] Step S21: Perform ethological analysis on the video data to obtain multiple three-dimensional skeletons of living bodies arranged in chronological order;

[0066] In the above process, methods for tracking multiple key body sites of animals, such as fine ethological analysis programs like Behavior Atlas (a new intelligent animal behavior precise analysis system), Moseq (action sequencing algorithm / behavior analysis algorithm that can recognize three-dimensional mouse body language (referred to as "syllables")), LEAP (behavior analysis model), Deeplabcut (a pose learning tool for pose estimation and motion analysis), EthoVision (animal movement trajectory tracking system), etc., can be used to automatically track multiple key body sites on the animal. The specific process of the above fine ethological analysis program is: referring to Figure 2 as shown in -b, first calculate the two-dimensional spatial coordinate information of the key body sites of the animal according to the position information calibrated by the camera. Subsequently, integrate the spatial coordinate information of the key body sites of multiple images, calculate the coordinate information of the key points in three-dimensional space, and reconstruct the three-dimensional skeleton of the animal.

[0067] Step S22: Extract pose features from multiple three-dimensional skeletons and adjust the format to obtain multiple three-dimensional skeletons with unified size and orientation, and the multiple three-dimensional skeletons are temporally continuous.

[0068] Among them, the process of pose feature extraction is to extract the information of the three-dimensional skeleton reconstructed in each frame. For example, extract the three-dimensional spatial coordinates of 16 body key points of the animal in each frame, and more body key points can also be set. Align the three-dimensional skeleton of the animal with the back point as the center; rotate the three-dimensional skeleton spatial coordinate information to unify the orientation of the three-dimensional skeleton; scale the skeleton according to the size ratio of the animal in the real three-dimensional space, scale the mouse skeleton to the same scale size, and thus obtain Figure 3 the three-dimensional stereogram as shown in -a.

[0069] Step S3: Perform action recognition on multiple consecutive 3D skeletons to obtain an action recognition result, calculate the action combination conversion probability parameters of multiple action combinations according to the time series, and label the action combination conversion probability parameters with corresponding emotions or disease states. The action combination is a combination of each action and another action that appears after each action.

[0070] Among them, calculating the action combination conversion probability parameters of multiple action combinations from the 3D information of the 3D skeleton is for the information of each reconstructed 3D skeleton frame. Therefore, labeling the corresponding emotions or disease states is also for each frame of the video image. The labeling process is carried out frame by frame by experimenters or users according to the process of obtaining experimental animals in different emotions or different disease states, or is carried out frame by frame according to the self-report of the collected individual and the auxiliary judgment of the doctor, and the label is corresponding to the action combination conversion probability parameters of multiple action combinations. Each frame corresponds to an action label. By calculating the appearance time, frequency, and duration of the action label, the corresponding action combination conversion probability parameters can be calculated.

[0071] Optionally, the action combination conversion probability parameters of the multiple action combinations include multiple different action combinations and the frequency of appearance of each action combination.

[0072] Reference Figure 3 As shown in Figure 3 -c, multiple actions include running, trotting, walking, right turning, left turning, stepping, jumping, climbing, rearing, hunching, rising, sniffing, grooming, pause. Multiple action combinations include running-sniffing, sniffing-running, running-stepping, trotting-sniffing, stepping-grooming, right turning-stepping, left turning-climbing, stepping-running, jumping-climbing, rearing-climbing, hunching-sniffing, pause-sniffing and other multiple action combination methods. Specifically, it can be referred to Figure 3 the conversion diagram shown in

[0073] Step S4: Repeat steps S1 - S3 to obtain the action combination conversion probability parameters of multiple action combinations marked with corresponding emotions and the action combination conversion probability parameters of multiple action combinations marked with corresponding disease states, and establish a training set of living body behavior characteristics based on this.

[0074] Step S5: Use the action combination conversion probability parameters of multiple said action combinations as input and the corresponding said emotion or disease state as output, and train the living body behavior logic abnormality auxiliary judgment model through the training set of living body behavior characteristics to obtain the single living body behavior logic abnormality auxiliary judgment model corresponding to the current species.

[0075] In the above solution, by constructing a brand - new correspondence between video data and 3D skeletons, the action combination conversion probability parameters of multiple action combinations with data characteristics are analyzed based on the 3D skeletons, and the action combination conversion probability parameters of multiple action combinations are corresponded to the emotions or disease states of the living body. Thus, a training set of living body behavior characteristics is constituted with the action combination conversion probability parameters as input and the corresponding emotions or disease states as output. And based on the above - mentioned training set of living body behavior characteristics, the living body behavior logic abnormality auxiliary judgment model is trained to obtain the single living body behavior logic abnormality auxiliary judgment model corresponding to the current species. Therefore, through the above - mentioned data analysis and model training process, emotions or disease states can be connected with behavior characteristics, and a single living body behavior logic abnormality auxiliary judgment model for determining the emotions or disease states of the living body through behavior abnormality analysis can be obtained, thereby solving the technical problem in the prior art of lacking effective quantitative evaluation means for the behavior abnormality of the living body.

[0076] Based on the above solution, the above - mentioned single living body behavior logic abnormality auxiliary judgment model is not only for scientific research and drug development, nor is it only used in mouse experimental animals. Any animal protection scenario that can use video recording, including hospitals, pet hospitals, zoos, wildlife protection, etc., can use the algorithm of the present invention to monitor the health status of animals or humans.

[0077] It can be used to assist in identifying the general range of potential abnormal emotions or disease states of animals, providing a data basis for doctors or experimenters to more quickly judge the state of humans or animals. It includes different pain templates such as mechanical pain, cold and heat pain, chemical pain, visceral pain, migraine, neuropathic pain, etc., and the spontaneous abnormal postures of animal models including addiction withdrawal reactions, congenital scoliosis, and some bone or muscle development diseases.

[0078] It is also possible to identify the emotional or disease state of an animal by observing the characteristics of its spontaneous behavior postures without the use of external stimuli. It is possible to observe the abnormal body postures of animals in specific emotions and diseases in a natural state. Without external stimuli causing emotional changes in the animals, it can better reflect the true spontaneous manifestations of the animals' experiences such as fear, addiction withdrawal, and pain in their real state. By digitizing the various posture characteristics of the animals through calculation, it is possible to more accurately describe the posture characteristics of the animals, providing comparable indicators that can be quantitatively analyzed for the assessment of the degree of fear, pain, addiction withdrawal pain, etc.

[0079] Taking the detection of olfactory hypofunction in living animals as an example:

[0080] Existing evaluations of the olfactory function of animals often release odor information through mineral oil, cotton swab volatilization, or directly through a catheter, and highly rely on the animals' curiosity about strange odors and their memory related to rewards and punishments. The evaluation methods are very single. They are mainly divided into two types: (1) Based on the animals' curiosity, testing their recognition of different odors; (2) Based on rewards or punishments, conducting associative learning of odors to induce the animals to make odor choices. However, these methods all have certain limitations. First, odor molecules are prone to volatilization and diffusion, and the odor molecules in the air will soon be diluted; second, humans and animals will soon become habituated to olfactory stimuli and lose their ability to recognize odors; third, releasing odors requires a certain carrier, such as a cotton swab, catheter, etc., and it is difficult to rule out that the animals show misleading behaviors due to their curiosity about the cotton swab or their memory of the exhaust sound; finally, the AD animal model has memory and cognitive defects, and poor performance in the above-mentioned behavioral tests may be due to problems with its memory or cognitive ability, and it cannot accurately reflect whether the olfactory ability of the animals is normal. Using the analysis method proposed by the present invention, by identifying the sniffing actions of animals through action recognition and analyzing the behavioral organization logic centered on sniffing animals, it can be used to evaluate whether the olfactory system of animals is working properly, because animals with olfactory loss will have disorders in the behavioral organization centered on sniffing.

[0081] Aiming at the shortcomings of the low data sites of traditional behavioral analysis and the inability to analyze the behavioral organization logic of animals, the present invention proposes a method for analyzing animal actions and conducting action conversion logic analysis based on fine behavioral analysis in a natural state, which can be used for the early screening of abnormal emotions, brain diseases, and brain injuries. This method does not require prior training of animals. It only needs to record the spontaneous behaviors of animals and conduct fine action analysis. By calculating the frequency, duration of action occurrence, and the conversion probability between actions, from a new perspective, it judges the health state of animals based on the logical relationship of the behavioral organizational structure.

[0082] The following takes the detection of whether Alzheimer's disease mice have olfactory defects as an example to elaborate the specific technical solution of the present invention in detail:

[0083] Prepare mice with normal and defective olfaction and place them in an open field that has no odor beforehand and is prepared with a specific diffused odor, including positive odor, central odor, and negative odor.

[0084] Record the spontaneous behavior of the mice in a completely dark environment to reduce the visual cue input to the mice and make the mice rely more on olfaction to explore the environment.

[0085] Use a high-resolution camera in infrared shooting mode to take pictures of the animals' spontaneous behavior in the open field from four directions without dead angles, and the recording duration is 15 minutes.

[0086] Use the Behavior Atlas fine ethology analysis software to analyze the ethological data of the animals.

[0087] In the above software, take the mouse as an example to illustrate the software analysis process: calibrate pictures of the key site information of a certain number of animal bodies, and use these pictures as the training set to train the Deeplabcut model. Subsequently, use the trained model to automatically track the two-dimensional coordinate information of multiple key sites on the animal body in each frame of the picture. According to the position information of the camera, integrate the key information of the key sites on the body of multiple images to reconstruct the three-dimensional skeleton information of the animal. Then, according to the dynamic three-dimensional skeleton movement of the animal, that is, the similarity of the three-dimensional spatial coordinates of the key points on the mouse body over time, use the dynamic time warping algorithm (dynamic aligned time kernel) to segment the continuous behavior to obtain several action segments. Use UMAP to reduce the pose features of these action segments to a two-dimensional action feature space, and combine the movement speed to form a three-dimensional action feature space. Subsequently, use the hierarchical clustering method to unsupervised cluster the action segments with similar movement characteristics and assign a digital label to the corresponding action segments. Finally, each frame of image information will obtain the corresponding digital label. Finally, manually check the content of these action segments and annotate them.

[0088] Use the analysis algorithm proposed in the present invention to calculate the sniffing behavior characteristics of the animals:

[0089] 1) Manually check the action modules divided by the fine ethology analysis system and annotate them;

[0090] 2) Extract the sniffing action, calculate basic features such as the duration distribution and average frequency of this action; 3) Extract the three-dimensional skeleton information of the sniffing action and calculate the sniffing pose characteristics under different odor cues;

[0091] 4) Based on the fine ethology action sequence, calculate the action transition probability. The olfactory mouse, as an important sense organ in the dark environment, is an important nodal action that connects various actions to form sequential behaviors.

[0092] Construct a systematic database of mouse sniffing behavior characteristics based on the sniffing action characteristics of normal mice and mice with olfactory dysfunction under different odor stimuli.

[0093] Based on feature learning with a large sample size, establish a multi-dimensional action feature model for olfactory dysfunction. By using this model to compare the spontaneous behaviors of mice in a dark environment, mice with abnormal olfactory function can be quickly identified.

[0094] Through the above solution, this application is based on a brand-new idea. By obtaining action labels through fine behavioral analysis, calculating the probability of mutual conversion between actions, and analyzing the logical relationship of behavioral organization, states such as abnormal emotions, brain diseases, and brain injuries are identified from high-dimensional logical relationships. Without introducing external stimuli, it reduces interference with the animal's state, and more objectively, uses a data-driven method to evaluate the abnormal brain state of animals. When detecting the olfactory system function of an Alzheimer's disease animal model, it is possible to judge whether the olfactory system of the animal is abnormal only from the spontaneous behavior characteristics of the animal without sacrificing the animal for pathological sectioning and without conducting complex memory and cognitive experiments for evaluation, which is used for ultra-early screening of this disease.

[0095] Taking the detection of olfactory function defects in Alzheimer's disease model mice as an example, the application of brain disease evaluation based on action conversion logic is elaborated in detail above. In fact, not only Alzheimer's disease models and olfactory function defects, but also any sensory perception defects, neuropsychiatric diseases, neurodegenerative diseases, and abnormal emotions such as fear, anxiety, and depression, and any disease states and brain states that show abnormal logical relationships with the behavioral organization of normal animals can be evaluated to a certain extent by the algorithm of the present invention.

[0096] In an optional embodiment, after the step of training the auxiliary judgment model for abnormal in vivo behavioral logic by using the action combination conversion probability parameter as the input and the corresponding emotion or the corresponding disease state as the output to obtain the auxiliary judgment model for abnormal in vivo behavioral logic corresponding to the current species, it further includes:

[0097] Replace the species type for collecting the video data, repeat steps S1 - S5, and establish an auxiliary judgment model for abnormal in vivo behavioral logic corresponding to another species;

[0098] Combine the auxiliary judgment models for abnormal in vivo behavioral logic corresponding to different species to obtain a multi - auxiliary judgment model for abnormal in vivo behavioral logic.

[0099] Through the above solution, for each species, a corresponding single-living-being behavior logic abnormality auxiliary judgment model can be trained separately, so that for each species, fine behavior analysis can be carried out. It further extracts pose features from the actions for unsupervised clustering discrimination, which can not only provide dynamic pose kinetics data, but also more objectively divide actions based on pose features in a more meaningful way, realizing objective and digital quantitative analysis indicators for each species. As a result, not only the pain state, bone and muscle development diseases of animals, but also any disease state showing abnormal spontaneous behavior characteristics of each species can be extracted and calculated through the solution constructed in this application. Moreover, by training a single-living-being behavior logic abnormality auxiliary judgment model for supervision for each species, such as cats, dogs, pigs, cows, sheep, monkeys, etc., other wild animals and humans, by constructing a single-living-being behavior logic abnormality auxiliary judgment model corresponding to the disease models of each species strain, it has important reference value for many fields such as basic scientific research, drug development, and animal health status detection.

[0100] In an alternative embodiment, with reference to Figure 5 As shown, after the step of using the action combination transition probability parameter as the input and the corresponding emotion or the corresponding disease state as the output, and training the single-living-being behavior logic abnormality auxiliary judgment model for the current species through the living behavior feature training set, the following steps are further included:

[0101] Step S6: Repeat steps S1 - S3 to obtain the action combination transition probability parameters of multiple action combinations marked with the corresponding emotion and the action combination transition probability parameters of multiple action combinations marked with the corresponding disease state, and establish a living behavior feature verification set therewith. The living behavior feature verification set is different from the action combination transition probability parameters of each group of multiple action combinations in the living behavior feature training set;

[0102] The living behavior feature verification set is different from the action combination transition probability parameters of each group of multiple action combinations in the living behavior feature training set. By replacing the collected living objects, the living objects collected in the living behavior feature verification set can be made different from those in the living behavior feature training set, so as to achieve the purpose of different action combination transition probability parameters for multiple action combinations.

[0103] Step S7: Verify the trained single-living-being behavior logic abnormality auxiliary judgment model according to the living behavior feature verification set until the loss function meets the preset threshold.

[0104] Through the above solution, the verification of the single-living body behavior logic anomaly auxiliary judgment model can be realized, ensuring the accuracy of the auxiliary judgment of the single-living body behavior logic anomaly auxiliary judgment model during subsequent use.

[0105] Optionally, the multi-living body behavior logic anomaly auxiliary judgment model includes:

[0106] A data preprocessing layer for obtaining video data of the living body to be analyzed, obtaining real-time action combination conversion probability parameters according to the video data, and calling the single-living body behavior logic anomaly auxiliary judgment model corresponding to the species according to the species of the living body to be analyzed; and,

[0107] The single-living body behavior logic anomaly auxiliary judgment model;

[0108] Reference Figure 6 As shown, the single-living body behavior logic anomaly auxiliary judgment model includes:

[0109] An input layer for obtaining the species of the living body to be analyzed and real-time action combination conversion probability parameters;

[0110] A hidden layer for inputting the action combination conversion probability parameters into the called single-living body behavior logic anomaly auxiliary judgment model;

[0111] An output layer for outputting corresponding emotions or disease states according to the single-living body behavior logic anomaly auxiliary judgment model.

[0112] Among them, the steps of obtaining video data of the living body to be analyzed and obtaining real-time action combination conversion probability parameters according to the video data are implemented with reference to steps S1-S3. The single-living body behavior logic anomaly auxiliary judgment model called according to the species of the living body to be analyzed is constructed in the construction method and verified by the living body behavior feature verification set.

[0113] In the above implementation, as shown in Figure 6 The single-living body behavior logic anomaly auxiliary judgment model is implemented using a fully connected neural network. Its input layer has two dimensions, namely x_axis and y_axis, the hidden layer is 50-dimensional, and the output layer is a 1*4 matrix, thereby constructing a fully connected neural network model with one-to-one correspondence where the input is the action combination conversion probability parameter and the output is the emotion or disease state, providing an effective auxiliary judgment model for further analyzing the precise high-dimensional correspondence relationship between animal states and abnormal behavioral phenotypes and postures. It should be noted that the matrices of the input layer and output layer can be changed according to needs, and the number of layers and dimensions of the hidden layer can also be changed according to the actual number of action combinations to obtain a single-living body behavior logic anomaly auxiliary judgment model applicable to each species.

[0114] Figure 7 FIG. 3 shows a schematic structural diagram of an embodiment of a device for constructing an auxiliary judgment model for behavioral logic anomalies. The specific embodiments of the present invention do not limit the specific implementation of the device for constructing the auxiliary judgment model for behavioral logic anomalies.

[0115] As Figure 7 shown, the device for constructing the auxiliary judgment model for behavioral logic anomalies may include: a processor 402, a communications interface 404, a memory 406, and a communication bus 408.

[0116] Among them: The processor 402, the communications interface 404, and the memory 406 communicate with each other through the communication bus 408. The communications interface 404 is used to communicate with network elements of other devices such as clients or other servers. The processor 402 is used to execute the program 410, and specifically may execute the relevant steps in the above-mentioned method embodiment for constructing the auxiliary judgment model for behavioral logic anomalies.

[0117] Specifically, the program 410 may include program code, and the program code includes computer-executable instructions.

[0118] The processor 402 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the device for constructing the auxiliary judgment model for behavioral logic anomalies may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0119] The memory 406 is used to store the program 410. The memory 406 may be a high-speed RAM memory, or may also include non-volatile memory, such as at least one disk memory.

[0120] The program 410 may specifically be called by the processor 402 to cause the device for constructing the auxiliary judgment model for behavioral logic anomalies to execute the operations of the above-mentioned method for constructing the auxiliary judgment model for behavioral logic anomalies.

[0121] It should be noted that since the device for constructing the behavior logic anomaly auxiliary judgment model of the present application can implement all embodiments of the method for constructing the behavior logic anomaly auxiliary judgment model, the device for constructing the behavior logic anomaly auxiliary judgment model of the present application has all the beneficial effects of the method for constructing the behavior logic anomaly auxiliary judgment model, which will not be elaborated here.

[0122] An embodiment of the present invention provides a storage medium, which stores at least one executable instruction. When the executable instruction runs on the device / apparatus for constructing the behavior logic anomaly auxiliary judgment model, it causes the device / apparatus for constructing the behavior logic anomaly auxiliary judgment model to execute the method for constructing the behavior logic anomaly auxiliary judgment model in any of the above method embodiments.

[0123] It should be noted that since the storage medium of the present application can implement all embodiments of the method for constructing the behavior logic anomaly auxiliary judgment model, the storage medium of the present application has all the beneficial effects of the method for constructing the behavior logic anomaly auxiliary judgment model, which will not be elaborated here.

[0124] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0125] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. Similarly, in order to streamline the present invention and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. Among them, the claims following the specific implementation mode are hereby expressly incorporated into the specific implementation mode, where each claim itself serves as a separate embodiment of the present invention.

[0126] Those skilled in the art can understand that the modules in the device in the embodiments can be adaptively changed and disposed in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive.

[0127] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for constructing an auxiliary judgment model for abnormal behavioral logic, characterized in that, The method includes: Step S1: For the same species, obtain video data of a living body in different emotional or disease states for a first preset duration; Step S2: Perform data analysis on the video data to obtain a plurality of temporally continuous three-dimensional skeletons; Step S3: Perform action recognition on the plurality of temporally continuous three-dimensional skeletons to obtain an action recognition result, calculate action combination conversion probability parameters for various action combinations according to the time series, and label the action combination conversion probability parameters with the corresponding emotional or disease state, where the action combination is a combination of each action and another action that appears after each action; Step S4: Repeat steps S1 - S3 to obtain the action combination conversion probability parameters labeled with the corresponding emotions and the action combination conversion probability parameters labeled with the corresponding disease states, and establish a training set of living body behavior characteristics based on this; Step S5: Use the action combination conversion probability parameters of various action combinations as inputs and the corresponding emotions or disease states as outputs, and train an auxiliary judgment model for abnormal living body behavior logic through the training set of living body behavior characteristics to obtain a single-living-body abnormal behavior logic auxiliary judgment model corresponding to the current species.

2. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 1, characterized in that, The step of performing data analysis on the video data to obtain a plurality of temporally continuous three-dimensional skeletons further includes: Step S21: Perform ethological analysis on the video data to obtain a plurality of three-dimensional skeletons of the living body arranged in chronological order; Step S22: Extract pose features from the plurality of three-dimensional skeletons and adjust the format to obtain a plurality of three-dimensional skeletons with unified size and orientation, and the plurality of three-dimensional skeletons are temporally continuous.

3. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 1 or 2, characterized in that After the step of using the action combination conversion probability parameters of various action combinations as inputs and the corresponding emotions or disease states as outputs to train an auxiliary judgment model for abnormal living body behavior logic to obtain a single-living-body abnormal behavior logic auxiliary judgment model corresponding to the current species, it further includes: Replace the species type for collecting the video data, repeat steps S1 - S5, and establish a single-living-body abnormal behavior logic auxiliary judgment model corresponding to another species; Combine the single-living-body abnormal behavior logic auxiliary judgment models corresponding to different species to obtain a multi-living-body abnormal behavior logic auxiliary judgment model.

4. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 1 or 2, characterized in that, After the step of using the action combination conversion probability parameters of various action combinations as inputs and the corresponding emotions or disease states as outputs to train an auxiliary judgment model for abnormal living body behavior logic through the training set of living body behavior characteristics to obtain a single-living-body abnormal behavior logic auxiliary judgment model corresponding to the current species, it further includes: Step S6: Repeat steps S1 - S3 to obtain the action combination conversion probability parameters of the multiple action combinations marked with corresponding emotions, and the action combination conversion probability parameters of the multiple action combinations marked with corresponding disease states, and establish a living behavior feature verification set with the action combination conversion probability parameters of the multiple action combinations in the living behavior feature training set being all different; Step S7: Verify the trained single living behavior logic abnormality auxiliary judgment model according to the living behavior feature verification set until the loss function meets the preset threshold.

5. The method for constructing an auxiliary judgment model for abnormal behavioral logic according to claim 1, characterized in that The action combination conversion probability parameters of the multiple action combinations include multiple different action combinations and the frequency of occurrence of each action combination.

6. The method for constructing an abnormal behavior logic auxiliary judgment model according to claim 2, wherein The multi - living behavior logic abnormality auxiliary judgment model includes: A data pre - processing layer, configured to obtain video data of a living body to be analyzed, perform data processing on the video data to obtain real - time action combination conversion probability parameters of multiple real - time action combinations, and call the single living behavior logic abnormality auxiliary judgment model corresponding to the species according to the species of the living body to be analyzed; and, The single living behavior logic abnormality auxiliary judgment model; The single living behavior logic abnormality auxiliary judgment model includes: An input layer, configured to obtain the species of the living body to be analyzed and the real - time action combination conversion probability parameters of multiple real - time action combinations; A hidden layer, configured to input the real - time action combination conversion probability parameters of the multiple real - time action combinations into the called single living behavior logic abnormality auxiliary judgment model; An output layer, configured to output the corresponding emotion or disease state according to the single living behavior logic abnormality auxiliary judgment model.

7. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 2, wherein, The emotion includes any one of fear, tension, anxiety, and depression, and the disease state includes any one of sensory deficits, neuropsychiatric diseases, and neurodegenerative diseases.

8. An apparatus for constructing an auxiliary judgment model for abnormal behavioral logic, characterized in that, It includes: A processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations of the method for constructing the behavior logic abnormality auxiliary judgment model as described in any one of claims 1 - 5.

9. A storage medium, characterized in that, At least one executable instruction is stored in the storage medium. When the executable instruction runs on the device / apparatus for constructing the behavior logic abnormality auxiliary judgment model, it causes the device / apparatus for constructing the behavior logic abnormality auxiliary judgment model to execute the operations of the method for constructing the behavior logic abnormality auxiliary judgment model as described in any one of claims 1 - 5.

Images