Method for constructing auxiliary determination model for behavior logic abnormality, and device and storage medium

Abstract

The present application relates to the technical field of auxiliary determination. Disclosed are a method for constructing an auxiliary determination model for a behavior logic abnormality, and a device and a storage medium. The method comprises: performing data analysis on video data of a living body, so as to obtain a plurality of temporally consecutive three-dimensional skeletons; performing action recognition on the three-dimensional skeletons, obtaining action combination conversion probability parameters on the basis of a time series, and marking the action combination conversion probability parameters as corresponding emotions or disease states, so as to establish a living-body behavior feature training set; and training an auxiliary determination model for a living-body behavior logic abnormality by means of the living-body behavior feature training set, so as to obtain an auxiliary determination model for a single-living-body behavior logic abnormality that corresponds to a current species. Therefore, the emotions or disease states can be connected to the action combination conversion probability parameters, and the emotions or disease states of the living body can be determined by means of the action combination conversion probability parameters.

Description

Construction method, device and storage medium of behavioral logic anomaly auxiliary judgment model Technical Field

[0001] The present application relates to the field of auxiliary judgment technology, and in particular to a method, device and storage medium for constructing an auxiliary judgment model for abnormal behavior logic. Background Art

[0002] In existing technology, humans and animals can involuntarily adopt abnormal postures and body positions due to pain, fear, developmental disorders, and certain diseases. Theoretically, these abnormal postures and body positions can be used to determine the condition of the person or animal. However, in animal experiments, the current assessment of animal posture and abnormal body positions requires experienced researchers to describe the animal's response to noxious stimulation by applying varying degrees of stimulation. This method relies heavily on experience and is difficult to quantify. Furthermore, there is currently a lack of objective quantitative methods for evaluating behavioral abnormalities.

[0003] Summary of the Invention

[0004] The purpose of this application is to provide a method, device and storage medium for constructing a behavioral logic anomaly auxiliary judgment model.

[0005] In a first aspect, an embodiment of the present application provides a method for constructing a behavioral logic anomaly auxiliary judgment model, the method comprising:

[0006] Step S1: For the same species, obtain video data of a living being in different emotions or different disease states for a first preset time period;

[0007] Step S2: performing data analysis on the video data to obtain a plurality of time-continuous three-dimensional skeletons;

[0008] Step S3, performing action recognition on multiple time-continuous three-dimensional skeletons to obtain action recognition results, and calculating action combination conversion probability parameters of multiple action combinations based on the time sequence, and marking the action combination conversion probability parameters as corresponding emotions or disease states; the action combination is a combination of each action and another action that appears after each action;

[0009] Step S4, repeating steps S1-S3 to obtain the action combination conversion probability parameters marked as corresponding to emotions and the action combination conversion probability parameters marked as corresponding to disease states, and using them to establish a living body behavior feature training set;

[0010] Step S5: Using the action combination conversion probability parameter as input and the corresponding emotion or disease state as output, the living body behavior logic abnormality auxiliary judgment model is trained through the living body behavior feature training set to obtain a single living body behavior logic abnormality auxiliary judgment model corresponding to the current species.

[0011] Optionally, the step of performing data analysis on the video data to obtain a plurality of time-continuous three-dimensional skeletons further comprises:

[0012] Step S21: performing behavioral analysis on the video data to obtain a plurality of living three-dimensional skeletons arranged in chronological order;

[0013] Step S22 : performing posture feature extraction on the plurality of three-dimensional skeletons and performing format adjustment to obtain a plurality of three-dimensional skeletons with uniform size and orientation, wherein the plurality of three-dimensional skeletons are continuous in time.

[0014] Optionally, after the step of training the living body behavior logic abnormality auxiliary judgment model using the action combination conversion probability parameters of the plurality of action combinations as input and the corresponding emotions or the corresponding disease states as output to obtain a single living body behavior logic abnormality auxiliary judgment model corresponding to the current species, the method further includes:

[0015] Change the species type of the video data collected, repeat steps S1-S5, and establish a single living body behavior logic abnormality auxiliary judgment model corresponding to the other species;

[0016] The single-living-body behavioral logic anomaly auxiliary judgment models corresponding to different species are combined to obtain a multi-living-body behavioral logic anomaly auxiliary judgment model.

[0017] Optionally, after the step of taking the action combination conversion probability parameters of the plurality of action combinations as input and the corresponding emotions or the corresponding disease states as output, and training the living body behavior logic abnormality auxiliary judgment model using the living body behavior feature training set to obtain a single living body behavior logic abnormality auxiliary judgment model corresponding to the current species, the step further includes:

[0018] Step S6, repeatedly performing steps S1-S3 to obtain the action combination conversion probability parameters of the plurality of action combinations marked as corresponding to emotions, and the action combination conversion probability parameters of the plurality of action combinations marked as corresponding to disease states, and thereby establishing a living body behavior feature verification set, wherein the action combination conversion probability parameters of each group of the plurality of action combinations in the living body behavior feature verification set and the living body behavior feature training set are different;

[0019] Step S7: verify the trained single living body behavior logic anomaly auxiliary judgment model based on the living body behavior feature verification set until the loss function meets a preset threshold.

[0020] Optionally, the action combination conversion probability parameters of the multiple action combinations include multiple different action combinations and the frequency of occurrence of each action combination.

[0021] Optionally, the multi-living body behavior logic abnormality auxiliary judgment model includes:

[0022] A data preprocessing layer is used to obtain video data of the living organism to be analyzed and perform data processing on the video data to obtain real-time action combination conversion probability parameters of multiple real-time action combinations, and call a single living organism behavior logic anomaly auxiliary judgment model corresponding to the species of the living organism to be analyzed; and

[0023] The single living body behavior logic abnormality auxiliary judgment model;

[0024] The single living body behavior logic abnormality auxiliary judgment model includes:

[0025] An input layer is used to obtain the species of the living body to be analyzed and the real-time action combination conversion probability parameters of the multiple real-time action combinations;

[0026] a hidden layer, configured to input the real-time action combination conversion probability parameters of the plurality of real-time action combinations into the called single-living-body behavior logic anomaly auxiliary judgment model;

[0027] The output layer is used to output the corresponding emotion or disease state according to the single living body behavior logic abnormality auxiliary judgment model.

[0028] Optionally, the emotion includes any one of fear, tension, anxiety, and depression, and the disease state includes any one of sensory impairment, neuropsychiatric disease, and neurodegenerative disease.

[0029] In a second aspect, an embodiment of the present application further provides a device for constructing a behavioral logic anomaly auxiliary judgment model, including:

[0030] A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0031] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operations of the method for constructing the behavioral logic abnormality auxiliary judgment model as described above.

[0032] In a third aspect, an embodiment of the present application also provides a storage medium, in which at least one executable instruction is stored. When the executable instruction is run on a device / apparatus for constructing a behavioral logic anomaly auxiliary judgment model, the device / apparatus for constructing a behavioral logic anomaly auxiliary judgment model performs the operation of the method for constructing a behavioral logic anomaly auxiliary judgment model as described above.

[0033] This solution constructs a new way of corresponding video data to three-dimensional skeletons, analyzes action combination conversion probability parameters of multiple action combinations based on the three-dimensional skeleton, and corresponds the action combination conversion probability parameters of multiple action combinations to the emotions or disease states of the living body, thereby forming a living body behavior feature training set with the action combination conversion probability parameters as input and the corresponding emotions or disease states as output, and trains the living body behavior logic anomaly auxiliary judgment model based on the above-mentioned living body behavior feature training set to obtain a single living body behavior logic anomaly auxiliary judgment model corresponding to the current species, so that the emotions or disease states can be linked to the behavioral characteristics through the above-mentioned data analysis and model training process, so as to obtain a single living body behavior logic anomaly auxiliary judgment model that determines the emotions or disease states of the living body through behavior logic anomaly analysis, thereby solving the technical problem of the lack of effective quantitative evaluation means for living body behavior anomalies in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] FIG1 is a flow chart showing a first embodiment of a method for constructing a behavioral logic abnormality auxiliary judgment model provided by the present invention;

[0036] FIG2 is a schematic diagram showing the process of video data acquisition and behavior analysis in the method for constructing a behavior logic abnormality auxiliary judgment model provided by the present invention;

[0037] FIG2-a shows a schematic diagram of the process of video data acquisition in the method for constructing a behavioral logic abnormality auxiliary judgment model provided by the present invention;

[0038] FIG2-b shows a schematic diagram of behavior analysis for determining living body behavior based on video data in the method for constructing a behavioral logic abnormality auxiliary judgment model provided by the present invention;

[0039] FIG3 is a schematic diagram showing a three-dimensional skeleton reconstruction process in the method for constructing a behavioral logic abnormality auxiliary judgment model provided by the present invention;

[0040] FIG3-a shows a schematic diagram of the structure of a three-dimensional skeleton collected from multiple angles in the method for constructing a behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0041] FIG3-b shows a statistical diagram of determining the frequency of sniffing actions based on video data in the method for constructing a behavioral logic anomaly auxiliary judgment model provided by the present invention;

[0042] FIG3-c shows a schematic diagram of the conversion of various action combinations in the method for constructing the behavioral logic abnormality auxiliary judgment model provided by the present invention;

[0043] FIG4 is a flow chart showing a second embodiment of a method for constructing a behavioral logic abnormality auxiliary judgment model provided by the present invention;

[0044] FIG5 is a flow chart showing a third embodiment of a method for constructing a behavioral logic abnormality auxiliary judgment model provided by the present invention;

[0045] FIG6 shows a schematic structural diagram of a living body behavior logic abnormality auxiliary judgment model in the method for constructing a behavior logic abnormality auxiliary judgment model provided by the present invention;

[0046] FIG7 shows a schematic structural diagram of an embodiment of a device for constructing a model for auxiliary determination of abnormal behavior logic of a living being provided by the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] The following is an analysis of the methods for determining abnormal behavior in the prior art:

[0049] Normal behavior in humans and animals follows a certain organizational logic. However, in situations of emotional distress, neurological disease, or brain damage, this behavioral logic can become disrupted to varying degrees, particularly the fundamental behavioral logic necessary to sustain life. Due to the difficulty in defining and identifying animal movements, detailed research into the organizational logic underlying animal behavior remains lacking.

[0050] Meaningful behavior can only be formed when actions are arranged and combined according to a certain logic. For example, basketball involves dribbling, passing, and shooting. The organizational logic of behavior is influenced by many factors, including past experience, physiological state, and the environment, and involves species habits and decision-making. However, behaviors necessary to maintain basic survival and reproduction adhere to a more conservative behavioral logic. For example, animals spontaneously explore in an open field according to a certain logic. Behavior to avoid predators typically involves a sequence of threat detection, risk assessment, escape initiation, reassessment of risk in a safe area, and further exploration outside the nest. However, in certain conditions of brain damage or sensory loss, the behavioral logic of humans and animals can become disrupted. For example, some Alzheimer's patients repeatedly ask medical staff for food due to amnesia. Normal mice will immediately explore the environment after entering an unfamiliar open field, while mice with autism quickly lose interest and preen in a corner. However, due to the limitations of animal behavior analysis methods, previous research has failed to analyze animal behavior at the level of movement, let alone analyze the behavioral logic of animals based on movement.

[0051] The present application provides a method, device and storage medium for constructing a behavioral logic anomaly auxiliary judgment model. By analyzing and processing image data and using the analyzed and processed data for model training, the emotion or disease state can be linked to the action combination conversion probability parameter, thereby obtaining a single living body behavioral logic anomaly auxiliary judgment model that determines the emotion or disease state of the living body through action combination analysis, thereby solving the technical problem of the lack of effective quantitative evaluation means for the behavioral anomalies of living bodies in the existing technology.

[0052] In one implementation scenario, this solution is implemented based on a camera component and a computer component.

[0053] Among them, the camera component is used to obtain video data, and the computer component is used to execute the method of constructing a behavioral logic anomaly auxiliary judgment model.

[0054] Figure 1 shows a flowchart of a first embodiment of a method for constructing a behavioral logic anomaly auxiliary judgment model according to the present invention. The method is executed by a device for constructing a living body behavioral logic anomaly auxiliary judgment model. As shown in Figure 1, the method includes the following steps:

[0055] The method for constructing the single-behavior logic anomaly auxiliary judgment model includes:

[0056] Step S1: For the same species, obtain video data of a living being in different emotions or different disease states for a first preset time period.

[0057] The above process can be achieved by using multiple high-definition resolution cameras or high-definition resolution cameras with infrared shooting mode to shoot the spontaneous behavior of the same species from multiple angles. At this time, different living organisms have different emotions or different disease states. By changing the collection object, different emotions or different disease states can be collected. The duration of the video data collected at this time can be set as needed. For example, the data collection time of a single living organism, that is, the first preset duration, can be set to between 15 minutes and 60 minutes. Collection from multiple angles can be performed. You can refer to the four perspectives shown in Figure 2-a. During actual data collection, you can also add or delete perspectives as needed.

[0058] Morphological analysis reveals that different species, when faced with different emotions or disease states, perceive subtle differences in the external environment due to differences in physiological structure. This can lead to distinct species-specific behavioral tendencies. For example, when in pain, mice tend to curl up, while humans tend to cover the painful area. These two behaviors, as well as the video data, differ significantly. Therefore, we only capture video data of the same species in different emotions or disease states to ensure data consistency.

[0059] In addition, when the species is an animal, the above steps can be carried out by constructing animal experimental models in different states under the premise of complying with animal ethics, and the above scheme can be used to construct experimental animals in different emotions or different disease states, and collect relevant video data from them.

[0060] When the species is human, with the consent of the subject, the different emotions or disease states of the subject can be defined by collecting personal statements and doctor-assisted judgments, and relevant video data can be collected.

[0061] Optionally, the emotion includes any one of fear, tension, anxiety, and depression, and the disease state includes any one of sensory impairment, neuropsychiatric disease, and neurodegenerative disease.

[0062] Step S2: performing data analysis on the video data to obtain a plurality of time-continuous three-dimensional skeletons.

[0063] Because living things are typically in continuous motion within a video, their 3D skeletons also change over time. This means that each frame of an image yields a single 3D skeleton. After analyzing a video, the resulting 3D skeletons are sequentially linked.

[0064] Furthermore, referring to FIG4 , the step of performing data analysis on the video data to obtain a plurality of time-continuous three-dimensional skeletons further includes:

[0065] Step S21: performing behavioral analysis on the video data to obtain a plurality of living three-dimensional skeletons arranged in chronological order;

[0066] In this process, multiple key body locations on the animal can be automatically tracked using sophisticated behavioral analysis programs such as BehaviorAtlas (a new intelligent animal behavior precision analysis system), Moseq (an action sequencing algorithm / behavioral analysis algorithm that can recognize three-dimensional mouse body language (called "syllables")), LEAP (a behavioral analysis model), Deeplabcut (a posture learning tool for posture estimation and motion analysis), and EthoVision (an animal motion trajectory tracking system). The specific process of the aforementioned sophisticated behavioral analysis program is as follows: Referring to Figure 2-b, the two-dimensional spatial coordinate information of the key body locations of the animal is first calculated based on the position information calibrated by the camera. The spatial coordinate information of the key body locations from multiple images is then integrated to calculate the coordinate information of the key points in three-dimensional space and reconstruct the animal's three-dimensional skeleton.

[0067] Step S22 : performing posture feature extraction on the plurality of three-dimensional skeletons and performing format adjustment to obtain a plurality of three-dimensional skeletons with uniform size and orientation, wherein the plurality of three-dimensional skeletons are continuous in time.

[0068] Among them, the process of posture feature extraction is to extract the information of the reconstructed three-dimensional skeleton of each frame, such as extracting the three-dimensional spatial coordinates of the 16 body key points of the animal in each frame. More body key points can also be set, and the three-dimensional skeleton of the animal is aligned with the back point as the center; the three-dimensional skeleton spatial coordinate information is rotated to unify the three-dimensional skeleton orientation; according to the size ratio of the animal in the real three-dimensional space, the skeleton is scaled, and the mouse skeleton is scaled to the same scale size, thereby obtaining the three-dimensional stereo image shown in Figure 3-a.

[0069] Step S3, perform action recognition on multiple time-continuous three-dimensional skeletons to obtain action recognition results, and calculate action combination conversion probability parameters of multiple action combinations based on the time series, and mark the action combination conversion probability parameters as corresponding emotions or disease states. The action combination is a combination of each action and another action that appears after each action.

[0070] The calculation of the three-dimensional information of the three-dimensional skeleton to obtain the action combination conversion probability parameters of multiple action combinations is for the information of the reconstructed three-dimensional skeleton of each frame. Therefore, the corresponding emotion or disease state is also for each frame of the video image. The labeling process is performed by the experimenter or user based on the acquired process of constructing experimental animals in different emotions or different disease states, or frame by frame based on the self-report of the collected individuals and the auxiliary judgment of the doctor, and the labels are matched with the action combination conversion probability parameters of multiple action combinations. Each frame corresponds to an action label. By calculating the time, frequency and duration of the action label, the corresponding action combination conversion probability parameter can be calculated.

[0071] Optionally, the action combination conversion probability parameters of the multiple action combinations include multiple different action combinations and the frequency of occurrence of each action combination.

[0072] As shown in Figure 3-c, the various actions include running, trotting, walking, right turning, left turning, stepping, jumping, climbing, rearing, hunching, rising, sniffing, grooming, and pausing. The various action combinations include run-sniffing, sniffing-running, run-pacing, trotting-sniffing, pacing-grooming, right turning-pacing, left turning-climbing, stepping-running, jumping-climbing, standing-climbing, hunching-sniffing, pausing-sniffing, and other action combinations. For details, see the transition diagram shown in Figure 3-c. The frequency of these actions can be calculated by statistically analyzing them with reference to Figure 3-b. By calculating the probability of the next action occurring after each action, the transition probability from each action to the next, and between different actions, can be calculated.

[0073] Step S4, repeating steps S1-S3 to obtain action combination conversion probability parameters for multiple action combinations marked as corresponding to emotions, and action combination conversion probability parameters for multiple action combinations marked as corresponding to disease states, and using these to establish a living body behavior feature training set;

[0074] Step S5: using the action combination conversion probability parameters of the plurality of action combinations as input and the corresponding emotions or disease states as output, the living body behavior logic abnormality auxiliary judgment model is trained through the living body behavior feature training set to obtain a single living body behavior logic abnormality auxiliary judgment model corresponding to the current species.

[0075] In the above scheme, by constructing a new way of corresponding video data to three-dimensional skeleton, action combination conversion probability parameters of multiple action combinations with data characteristics are analyzed based on the three-dimensional skeleton, and the action combination conversion probability parameters of multiple action combinations are corresponded to the emotions or disease states of the living body, thereby forming a living body behavior feature training set with the action combination conversion probability parameters as input and the corresponding emotions or the disease states as output, and the living body behavior logic abnormality auxiliary judgment model is trained based on the above living body behavior feature training set to obtain a single living body behavior logic abnormality auxiliary judgment model corresponding to the current species, so that the emotions or disease states can be connected with the behavior characteristics through the above data analysis and model training process, so as to obtain a single living body behavior logic abnormality auxiliary judgment model that determines the emotions or disease states of the living body through behavior abnormality analysis, thereby solving the technical problem of the lack of effective quantitative evaluation means for the behavior abnormalities of living bodies in the existing technology.

[0076] Based on the above scheme, the above-mentioned single-living behavioral logic anomaly auxiliary judgment model is not only used in scientific research and drug development, nor is it only used for mouse experimental animals. Any animal protection scene that can use video recording, including: hospitals, pet hospitals, zoos, wildlife protection, etc., can use the algorithm of the present invention to monitor the health status of animals or humans.

[0077] It can be used to help identify the approximate range of potential abnormal emotions or disease states in animals, providing a data foundation for doctors and experimenters to more quickly judge the state of humans or animals. This includes different pain templates such as mechanical pain, hot and cold pain, chemical pain, visceral pain, migraine, and neuropathic pain, as well as spontaneous abnormal postures in animal models of addiction withdrawal reactions, congenital scoliosis, and some skeletal or muscular developmental diseases.

[0078] It is also possible to identify an animal's emotional or medical condition simply by observing its spontaneous behavioral and postural characteristics, without the need for external stimuli. This allows for observing the abnormal postures exhibited by animals in their natural state when experiencing specific emotions and illnesses, without external stimuli causing emotional changes. This allows for a more realistic and spontaneous reflection of the animal's experiences of fear, addiction withdrawal, pain, and other such conditions. By digitizing the various postural characteristics of an animal, this allows for a more precise description of these characteristics, providing quantifiable comparative indicators for assessing levels of fear, pain, and the distress of addiction withdrawal.

[0079] Take the detection of hyposmia in living subjects as an example:

[0080] The current assessment of an animal's olfactory function often involves volatilizing mineral oil or cotton swabs or releasing odor information directly through a catheter. These methods are highly dependent on the animal's curiosity about unfamiliar odors and its memory related to rewards and punishments, resulting in a very simple assessment method. These methods are mainly divided into two types: (1) testing the animal's ability to identify different odors based on curiosity; and (2) inducing the animal to select an odor through odor-associated learning based on rewards or punishments. However, these methods all have certain limitations. First, odor molecules evaporate and diffuse easily, and the odor molecules in the air are quickly diluted. Second, humans and animals quickly become accustomed to olfactory stimulation and lose their ability to identify odors. Third, odor release requires a certain carrier, such as a cotton swab or catheter, and it is difficult to rule out the possibility that the animal exhibits misleading behavior due to curiosity about the cotton swab or memory of the exhaust sound. Finally, AD animal models have memory and cognitive deficits. Poor performance in the above behavioral tests may be due to problems with their memory or cognitive abilities, and cannot accurately reflect whether the animal's olfactory ability is normal. Using the analysis method proposed in the present invention, the animal's sniffing action is identified through action recognition, and the behavioral organization logic centered on the sniffing animal is analyzed. This can be used to evaluate whether the animal's olfactory system is working properly, because animals with anosmia will have behavioral organization disorders centered on sniffing.

[0081] To address the shortcomings of traditional behavioral analysis, which suffers from a low data density and inability to analyze the logical organization of animal behavior, this paper proposes a method for analyzing animal movements in their natural state based on detailed behavioral analysis and performing logical analysis of movement transitions. This method can be used for early screening of abnormal emotions, brain diseases, and brain damage. This method requires no prior animal training; it simply records the animal's spontaneous behavior and performs detailed movement analysis. By calculating the frequency, duration, and transition probability of movements, this method offers a novel perspective on the logical relationship between behavioral organization structures to determine the animal's health status.

[0082] The following uses the example of detecting whether Alzheimer's mice have olfactory defects to describe the specific technical solutions of the present invention in detail:

[0083] Mice with normal or deficient sense of smell were prepared and placed in an open field without any odor or with specific odors prepared in advance, including positive odor, central odor and negative odor.

[0084] The spontaneous behavior of mice was recorded in a completely dark environment, reducing the visual cue input to the mice and allowing them to rely more on their sense of smell to explore the environment.

[0085] A high-resolution camera in infrared shooting mode was used to film the spontaneous behavior of animals in the open field from four directions without blind spots, with a recording time of 15 minutes.

[0086] BehaviorAtlas software was used to analyze the animal's spontaneous behavioral data.

[0087] The software uses a mouse as an example to illustrate the analysis process: a certain number of images containing key body locations are calibrated and used as a training set to train the DeepLabCut model. The trained model then automatically tracks the 2D coordinates of multiple key body locations within each frame. Based on the camera's position, the key body location information from multiple images is integrated to reconstruct the animal's 3D skeleton. Based on the animal's dynamic 3D skeleton motion—that is, the similarity of the 3D coordinate time series of key points on the mouse's body—a dynamic aligned time kernel (DAT) algorithm is then used to segment continuous actions, generating several action segments. Using Unified Mapping (UMAP), the posture features of these action segments are reduced to a 2D action feature space and, combined with motion speed, a 3D action feature space is constructed. Hierarchical clustering is then used to unsupervisedly cluster action segments with similar motion features and assign a numerical label to each segment. Ultimately, each frame of image information is assigned a corresponding numerical label. Finally, the content of these action segments is manually reviewed and annotated.

[0088] The analysis algorithm proposed by the present invention is used to calculate the sniffing behavior characteristics of animals:

[0089] 1) Manually inspect and annotate the action modules divided by the refined behavior analysis system;

[0090] 2) Extract the sniffing action and calculate basic features such as the duration distribution and average frequency of the action; 3) Extract the 3D skeleton information of the sniffing action and calculate the sniffing posture characteristics under different odor cues;

[0091] 4) Calculate the probability of action transitions based on the detailed behavioral action sequence. Olfactory perception is a crucial sensory organ in dark environments and serves as a crucial node in connecting individual actions to form a sequential behavior.

[0092] Based on the sniffer action characteristics of normal mice and mice with olfactory dysfunction under different odor stimulations, a systematic mouse sniffer behavior characteristic database was constructed.

[0093] Based on large-scale feature learning, a multi-dimensional action feature model of olfactory dysfunction was established. This model was used to compare the spontaneous behavior of mice in a dark environment, enabling the rapid identification of mice with olfactory dysfunction.

[0094] Through the above scheme, this application is based on a brand-new idea. It obtains action labels through fine behavioral analysis, calculates the probability of mutual conversion between actions, analyzes the logical relationship of behavioral organization, and identifies emotional abnormalities, brain diseases, brain damage and other states from high-dimensional logical relationships. No external stimulation is introduced to reduce interference with the animal's state, and the abnormal brain state of animals is evaluated more objectively and data-driven. When detecting the olfactory system function of animal models of Alzheimer's disease, it is possible to judge whether the animal's olfactory system is abnormal only from the spontaneous behavioral characteristics of the animal without sacrificing the animal for pathological sectioning or conducting complex memory and cognitive experiments for evaluation, so as to perform ultra-early screening for the disease.

[0095] The application of action conversion logic for brain disease assessment has been described in detail above, primarily using the example of detecting olfactory deficits in Alzheimer's disease mice. In reality, the algorithm described above can be used to assess not only Alzheimer's disease models and olfactory deficits, but also any sensory deficits, neuropsychiatric disorders, neurodegenerative diseases, and emotional states such as fear, anxiety, and depression. Any disease state or brain state that exhibits behavioral logic that deviates from normal can be assessed to some extent using this algorithm.

[0096] In an optional embodiment, after the step of training the living body behavior logic abnormality auxiliary judgment model using the action combination transition probability parameter as input and the corresponding emotion or the corresponding disease state as output to obtain a single living body behavior logic abnormality auxiliary judgment model corresponding to the current species, the method further includes:

[0097] Change the species type of the video data collected, repeat steps S1-S5, and establish a single living body behavior logic abnormality auxiliary judgment model corresponding to the other species;

[0098] The single-living-body behavioral logic anomaly auxiliary judgment models corresponding to different species are combined to obtain a multi-living-body behavioral logic anomaly auxiliary judgment model.

[0099] Through the above scheme, a corresponding single-body behavioral logic abnormality auxiliary judgment model can be trained for each species, so that a detailed behavioral analysis can be performed for each species. It is to further extract posture features for the actions of unsupervised clustering discrimination, which can not only provide dynamic posture dynamics data, but also more objectively divide the actions more meaningfully based on posture features, and realize objective and digital quantitative analysis indicators for each species, so that not only the pain state of each species, bone and muscle development diseases, but also any disease state that exhibits abnormal spontaneous behavioral characteristics can be extracted and calculated through the scheme constructed by this application. Moreover, by training a single-body behavioral logic abnormality auxiliary judgment model for each species for supervision, such as cats, dogs, pigs, cattle, sheep, monkeys, other wild animals and humans, and thus by constructing a single-body behavioral logic abnormality auxiliary judgment model corresponding to the disease model of each species strain, it has important reference value for many fields such as basic scientific research, drug development, and animal health status detection.

[0100] In an optional embodiment, referring to FIG5 , after the step of using the action combination transition probability parameter as input and the corresponding emotion or the corresponding disease state as output, and training the living body behavior logic abnormality auxiliary judgment model using the living body behavior feature training set to obtain a single living body behavior logic abnormality auxiliary judgment model corresponding to the current species, the step further includes:

[0101] Step S6, repeating steps S1-S3 to obtain action combination conversion probability parameters for multiple action combinations marked as corresponding to emotions, and action combination conversion probability parameters for multiple action combinations marked as corresponding to disease states, and using these to establish a live behavior feature verification set, wherein the action combination conversion probability parameters for each group of multiple action combinations in the live behavior feature verification set and the live behavior feature training set are different;

[0102] The action combination conversion probability parameters of each group of multiple action combinations in the live behavior feature verification set and the live behavior feature training set are different. By replacing the collected living objects, the live objects collected in the live behavior feature verification set and the live behavior feature training set can be different, thereby achieving the purpose of different action combination conversion probability parameters for multiple action combinations.

[0103] Step S7: verify the trained single living body behavior logic anomaly auxiliary judgment model based on the living body behavior feature verification set until the loss function meets a preset threshold.

[0104] Through the above solution, the single living body behavior logic anomaly auxiliary judgment model can be verified, ensuring the accuracy of the auxiliary judgment of the single living body behavior logic anomaly auxiliary judgment model in subsequent use.

[0105] Optionally, the multi-living body behavior logic abnormality auxiliary judgment model includes:

[0106] A data preprocessing layer is used to obtain video data of the living organism to be analyzed and obtain real-time action combination conversion probability parameters based on the video data, and to call a single living organism behavior logic anomaly auxiliary judgment model corresponding to the species of the living organism to be analyzed; and

[0107] The single living body behavior logic abnormality auxiliary judgment model;

[0108] As shown in FIG6 , the single living body behavior logic abnormality auxiliary judgment model includes:

[0109] The input layer is used to obtain the species of the living organism to be analyzed and the real-time action combination conversion probability parameters;

[0110] A hidden layer, configured to input the action combination conversion probability parameter into the called single-living-body behavior logic anomaly auxiliary judgment model;

[0111] The output layer is used to output the corresponding emotion or disease state according to the single living body behavior logic abnormality auxiliary judgment model.

[0112] The steps of obtaining video data of the living organism to be analyzed and deriving real-time action combination conversion probability parameters based on the video data are implemented with reference to steps S1-S3. The single living organism behavior logic anomaly auxiliary judgment model corresponding to the species of the living organism to be analyzed is called based on the species. The single living organism behavior logic anomaly auxiliary judgment model called is constructed in the construction method and verified using the living organism behavior feature verification set.

[0113] In the above implementation scheme, as shown in reference Figure 6, the single-living-body behavioral logic abnormality auxiliary judgment model is implemented using a fully connected neural network, whose input layer has two dimensions, x_axis and y_axis, the hidden layer is 50 dimensions, and the output layer is a 1*4 matrix, thereby constructing a one-to-one corresponding fully connected neural network model with input as action combination conversion probability parameters and output as emotion or disease state, thereby providing an effective auxiliary judgment model for further analyzing the precise high-dimensional correspondence between animal state and abnormal behavioral phenotypes and postures. It should be noted that the matrices of the input layer and output layer can be transformed as needed, and the number of layers and dimensions of the hidden layer can also be transformed according to the actual number of action combinations, so as to obtain a single-living-body behavioral logic abnormality auxiliary judgment model suitable for each species.

[0114] FIG7 shows a schematic structural diagram of an embodiment of a device for constructing a behavioral logic anomaly auxiliary judgment model according to the present invention. The specific embodiment of the present invention does not limit the specific implementation of the device for constructing a behavioral logic anomaly auxiliary judgment model.

[0115] As shown in FIG. 7 , the device for constructing the behavioral logic abnormality auxiliary judgment model may include: a processor 402 , a communications interface 404 , a memory 406 , and a communication bus 408 .

[0116] Processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other devices, such as clients or other server network elements. Processor 402 is used to execute program 410, which may specifically perform the steps described in the aforementioned embodiment of the method for constructing a behavioral logic anomaly auxiliary judgment model.

[0117] Specifically, the program 410 may include program code including computer-executable instructions.

[0118] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the device for constructing a behavioral logic anomaly auxiliary judgment model may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0119] The memory 406 is used to store the program 410. The memory 406 may be a high-speed RAM memory, or may also include a non-volatile memory, such as at least one disk memory.

[0120] The program 410 can be specifically called by the processor 402 to enable the construction device of the behavior logic abnormality auxiliary judgment model to perform the operations of the construction method of the behavior logic abnormality auxiliary judgment model as described above.

[0121] It should be noted that since the construction device of the behavioral logic anomaly auxiliary judgment model of the present application can implement all embodiments of the construction method of the behavioral logic anomaly auxiliary judgment model, the construction device of the behavioral logic anomaly auxiliary judgment model of the present application has all the beneficial effects of the construction method of the behavioral logic anomaly auxiliary judgment model, which will not be repeated here.

[0122] An embodiment of the present invention provides a storage medium storing at least one executable instruction. When the executable instruction is executed on a device / apparatus for constructing a behavioral logic anomaly auxiliary judgment model, the device / apparatus for constructing a behavioral logic anomaly auxiliary judgment model executes the method for constructing a behavioral logic anomaly auxiliary judgment model in any of the above-mentioned method embodiments.

[0123] It should be noted that since the storage medium of the present application can implement all embodiments of the method for constructing a behavioral logic anomaly auxiliary judgment model, the storage medium of the present application has all the beneficial effects of the method for constructing a behavioral logic anomaly auxiliary judgment model, which will not be repeated here.

[0124] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0125] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present invention may be practiced without these specific details. Similarly, in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. The claims that follow the detailed description are hereby expressly incorporated into that detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0126] Those skilled in the art will appreciate that the modules in the devices of the embodiments can be adaptively changed and installed in one or more devices different from the embodiments. The modules, units, or components in the embodiments can be combined into one module, unit, or component, and furthermore, they can be divided into multiple submodules, subunits, or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0127] It should be noted that the above embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by a person skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A method for constructing an auxiliary judgment model for abnormal behavior logic, characterized in that, The method includes: Step S1: For the same species, obtain video data of a living body in different emotional states or different disease states for a first preset duration; Step S2: Perform data analysis on the video data to obtain a plurality of temporally continuous three-dimensional skeletons; Step S3: Perform action recognition on the plurality of temporally continuous three-dimensional skeletons to obtain an action recognition result, calculate action combination transition probability parameters for various action combinations according to the time series, and label the action combination transition probability parameters as corresponding emotional states or disease states, where the action combination is a combination of each action and another action that appears after each action; Step S4: Repeat steps S1 - S3 to obtain the action combination transition probability parameters labeled as corresponding emotional states and the action combination transition probability parameters labeled as corresponding disease states, and establish a training set of living body behavior characteristics based on this; Step S5: Use the action combination transition probability parameters of various action combinations as inputs, and the corresponding emotional states or disease states as outputs, and train an auxiliary judgment model for abnormal living body behavior logic through the training set of living body behavior characteristics to obtain a single-living-body abnormal behavior logic auxiliary judgment model corresponding to the current species.

2. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 1, wherein The step of performing data analysis on the video data to obtain a plurality of temporally continuous three-dimensional skeletons further includes: Step S21: Perform ethological analysis on the video data to obtain a plurality of three-dimensional skeletons of the living body arranged in chronological order; Step S22: Extract pose features from the plurality of three-dimensional skeletons and adjust the format to obtain a plurality of three-dimensional skeletons with unified size and orientation, and the plurality of three-dimensional skeletons are temporally continuous.

3. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 1 or 2, characterized in that After the step of using the action combination transition probability parameters of various action combinations as inputs, and the corresponding emotional states or corresponding disease states as outputs, and training an auxiliary judgment model for abnormal living body behavior logic through the training set of living body behavior characteristics to obtain a single-living-body abnormal behavior logic auxiliary judgment model corresponding to the current species, it further includes: Replace the species type for collecting the video data, repeat steps S1 - S5, and establish a single-living-body abnormal behavior logic auxiliary judgment model corresponding to another species; Combine the single-living-body abnormal behavior logic auxiliary judgment models corresponding to different species to obtain a multi-living-body abnormal behavior logic auxiliary judgment model.

4. The method for constructing an abnormal behavior logic auxiliary judgment model according to claim 1 or 2, characterized in that, After the step of using the action combination transition probability parameters of various action combinations as inputs, and the corresponding emotional states or corresponding disease states as outputs, and training an auxiliary judgment model for abnormal living body behavior logic through the training set of living body behavior characteristics to obtain a single-living-body abnormal behavior logic auxiliary judgment model corresponding to the current species, it further includes: Step S6: Repeat steps S1 - S3 to obtain the action combination conversion probability parameters of the multiple action combinations marked with corresponding emotions and the action combination conversion probability parameters of the multiple action combinations marked with corresponding disease states, and establish a living behavior feature verification set with the action combination conversion probability parameters of the multiple action combinations in the living behavior feature verification set being different from those in the living behavior feature training set; Step S7: Verify the trained single living behavior logic abnormality auxiliary judgment model based on the living behavior feature verification set until the loss function meets the preset threshold.

5. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 1, characterized in that The action combination conversion probability parameters of the multiple action combinations include multiple different action combinations and the frequency of occurrence of each action combination.

6. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 2, wherein The multi - living behavior logic abnormality auxiliary judgment model includes: A data pre - processing layer, configured to obtain video data of the living body to be analyzed, perform data processing on the video data to obtain real - time action combination conversion probability parameters of multiple real - time action combinations, and call the single living behavior logic abnormality auxiliary judgment model corresponding to the species according to the species of the living body to be analyzed; and, The single living behavior logic abnormality auxiliary judgment model; The single living behavior logic abnormality auxiliary judgment model includes: An input layer, configured to obtain the species of the living body to be analyzed and the real - time action combination conversion probability parameters of multiple real - time action combinations; A hidden layer, configured to input the real - time action combination conversion probability parameters of the multiple real - time action combinations into the called single living behavior logic abnormality auxiliary judgment model; An output layer, configured to output the corresponding emotion or disease state according to the single living behavior logic abnormality auxiliary judgment model.

7. The method for constructing an auxiliary judgment model for abnormal behavior logic according to claim 2, wherein, The emotion includes any one of fear, tension, anxiety, and depression, and the disease state includes any one of sensory deficits, neuropsychiatric diseases, and neurodegenerative diseases.

8. An apparatus for constructing an auxiliary judgment model for abnormal behavior logic, characterized in that, It includes: A processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations of the method for constructing the behavior logic abnormality auxiliary judgment model according to any one of claims 1 - 5.

9. A storage medium, characterized in that, At least one executable instruction is stored in the storage medium. When the executable instruction runs on the device / apparatus for constructing the behavior logic abnormality auxiliary judgment model, it causes the device / apparatus for constructing the behavior logic abnormality auxiliary judgment model to execute the operations of the method for constructing the behavior logic abnormality auxiliary judgment model according to any one of claims 1 - 5.

Images