Behavior analysis method and apparatus, and electronic device and storage medium

Abstract

The present application relates to the field of behavior analysis. Provided are a behavior analysis method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring an image sequence, wherein the image sequence comprises at least one image frame, and each image is generated during a process in which a vapor exposure apparatus delivers the same or different vapor to at least one test target; performing pose estimation on each test target in each image to obtain key point information of each test target under different exposure times; and on the basis of the key point information of each test target under different exposure times, performing quantitative analysis on the behavior of each test target under at least one vapor exposure condition to obtain a behavior difference result of each test target. The present application solves the problems in the related art of low efficiency and insufficient accuracy of behavior analysis.

Description

Behavior analysis method, device, electronic device and storage medium Technical Field

[0001] The present application relates to the technical field of behavior analysis, and more specifically, to a behavior analysis method, device, electronic device, and storage medium. Background Art

[0002] Current research on electronic nicotine delivery systems (ENDS) or e-cigarettes mainly studies the neuropsychiatric effects of e-cigarettes by understanding and evaluating the behavior of ENDS-related addicted mice.

[0003] In existing technologies, traditional behavioral tests such as reward place preference and anxiety behavior testing after smoke withdrawal, including open field tests and elevated cross tests, require a long time to observe and measure mice after exposing them to ENDS. This leads to time delays in the detection of behavioral characteristics of ENDS-related addiction mouse models, resulting in low efficiency and accuracy of existing behavioral analysis methods. Therefore, improving the efficiency of behavioral analysis remains a challenge.

[0004] Summary of the Invention

[0005] This application provides a behavior analysis method, device, electronic device, and storage medium, which can solve the problems of low efficiency and insufficient accuracy of behavior analysis in related technologies. The technical solution is as follows:

[0006] According to one aspect of the present application, a behavior analysis method under smoke exposure conditions includes: acquiring an image sequence, the image sequence including at least one frame of image, each of the images being generated during a process in which a smoke exposure device delivers the same or different smoke to at least one detection target; performing posture estimation on each of the detection targets in each of the images to obtain key point information of each of the detection targets at different exposure times; and performing quantitative analysis on the behavior of each of the detection targets under at least one smoke exposure condition based on the key point information of each of the detection targets at different exposure times to obtain behavioral difference results for each of the detection targets.

[0007] According to one aspect of the present application, a behavior analysis device includes: an image acquisition module for acquiring an image sequence, the image sequence including at least one frame of image, each of the images being generated during a smoke exposure device delivering the same or different smoke to at least one detection target; a posture estimation module for performing posture estimation on each of the detection targets in each of the images to obtain key point information of each of the detection targets at different exposure times; and an analysis module for performing quantitative analysis on the behavior of each of the detection targets under at least one smoke exposure condition based on the key point information of each of the detection targets at different exposure times to obtain behavioral difference results of each of the detection targets.

[0008] In an exemplary embodiment, the analysis module includes: a quantification unit, used to quantify the key point information of each of the detection targets at different exposure times into position parameters and body parameters of each of the detection targets at different exposure times; a behavior characteristic determination unit, used to determine the behavior characteristics of each of the detection targets at different exposure times based on the position parameters and body parameters of each of the detection targets at different exposure times; and an analysis unit, used to use statistical methods to analyze the differences in the behavior characteristics of each of the detection targets at different exposure times to obtain behavior difference results of each of the detection targets.

[0009] In an exemplary embodiment, the behavior characteristic determination unit includes: a nose touch behavior characteristic determination subunit, which is used to determine the nose touch behavior characteristics of the detection target for each detection target at different exposure times based on the nose center distance in the position parameters and the nose speed in the body parameters; or a body behavior characteristic determination unit, which is used to determine the body behavior characteristics of the detection target based on the direction index in the position parameters and the body length and body angle in the body parameters; or an exploration behavior characteristic determination unit, which is used to determine the exploration behavior characteristics of the detection target based on the direction index in the position parameters and the body speed and movement distance in the body parameters.

[0010] In an exemplary embodiment, the analysis unit includes: a first analysis subunit, used to analyze the differences in the behavioral characteristics of each of the detection targets at different exposure times, and obtain a first difference result for each of the detection targets; a second analysis subunit, used to select at least two of the detection targets from the detection targets, analyze the differences in the behavioral characteristics of at least two of the detection targets at the same exposure time, and obtain a second difference result for each of the detection targets; and a result generation subunit, used to obtain a behavioral difference result for each of the detection targets from the first difference result and the second difference result of each of the detection targets.

[0011] In an exemplary embodiment, the posture estimation module includes: an image determination unit, used to determine the current image among the images, perform feature representation extraction at different levels on the current image, and obtain local features of the current image; a mapping unit, used to map the local features of the current image to different key point categories, and perform probability distribution conversion on the mapped different key point categories, and predict the probability that the key points of each detection target in the current image belong to different key point categories; an information acquisition unit, used to obtain the key point information of each detection target at the current exposure time based on the probability that the key points of each detection target belong to different key point categories; until the posture estimation of each detection target in each image is completed, and the key point information of each detection target at different exposure times is obtained.

[0012] In an exemplary embodiment, the posture estimation of each detection target in each image is implemented based on a posture estimation model, which is a machine learning model that has been trained and has the ability to estimate the posture of the detection target in the image; the posture estimation module also includes: a training set acquisition unit, used to acquire a training set; the training set includes multiple training samples, and the training samples are images with labels; the labels are used to indicate the real key points of the detection targets in the training samples; a training unit, used to train a pre-constructed machine learning model based on each training sample in the training set to obtain a training result; the training result is used to indicate the predicted key points of the detection targets in the training samples; an updating unit, used to determine whether the training of the machine learning model is completed based on the error between the predicted key points in the training results and the real key points in the labels; if so, the posture estimation model is obtained; otherwise, the parameters of the machine learning model are updated to continue training.

[0013] In an exemplary embodiment, a behavior analysis system includes: a smoke exposure device and an analysis end; the smoke exposure device includes a smoke delivery unit and an image camera unit; wherein the smoke delivery unit delivers the same or different smoke to at least one detection target; the image camera unit captures and collects images during the process of the smoke delivery unit delivering the same or different smoke to at least one detection target, and transmits the images to the analysis end; the analysis end, based on the received image and according to the behavior analysis method as described above, performs behavior analysis on the detection target in the image under smoke exposure conditions.

[0014] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the behavioral analysis method described above.

[0015] According to one aspect of the present application, a storage medium stores computer-readable instructions thereon, wherein the computer-readable instructions are executed by one or more processors to implement the behavior analysis method as described above.

[0016] According to one aspect of the present application, a computer program product includes computer-readable instructions, which are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the behavioral analysis method described above.

[0017] The beneficial effects of the technical solution provided by this application are:

[0018] In the above technical solution, the posture of the detection target in the image is estimated through an image sequence to obtain key point information of each detection target at different exposure times. The key point information is then used to quantitatively analyze the behavior of each detection target under at least one smoke exposure condition to obtain behavioral difference results for each detection target. Because the images are generated during the process of a smoke exposure device delivering the same or different smoke to at least one detection target, the behavioral changes of the detection target under different exposure times (achieved through different images) and different smoke exposure conditions (achieved through different smoke) can be monitored in real time. Combined with machine learning methods, subtle behavioral changes can be more objectively and easily captured, thereby effectively solving the problem of low efficiency of behavior analysis existing in related technologies.

[0019] In addition, an innovative behavioral analysis method is proposed to analyze the behavior of the detection target when the detection target is just exposed to e-cigarette vapor, rather than analyzing the detection target's anxiety behavior after exposure to ENDS. This not only eliminates the need for traditional behavioral tests such as reward location preference and anxiety behavior after smoke withdrawal, but is also simpler and more time-saving, thereby further improving the accuracy of behavioral analysis and detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts.

[0021] FIG1 is a schematic diagram of an implementation environment according to the present application;

[0022] FIG2 is a flow chart of a behavior analysis method according to an exemplary embodiment;

[0023] FIG3 is a flow chart of step 250 in one embodiment of the embodiment corresponding to FIG2 ;

[0024] FIG3a is a schematic diagram of a specific implementation of determining key points involved in the embodiment corresponding to FIG3;

[0025] FIG4 is a flow chart showing a process of determining behavior characteristics according to an exemplary embodiment;

[0026] FIG5 is a flow chart of step 350 in one embodiment of the embodiment corresponding to FIG3 ;

[0027] FIG6 a is a schematic diagram of a specific implementation of a posture estimation model according to an exemplary embodiment;

[0028] FIG6 b is a flow chart of a training process of a posture estimation model according to an exemplary embodiment;

[0029] FIG7 is a flow chart of step 230 in one embodiment of the embodiment corresponding to FIG2 ;

[0030] FIG7 a is a schematic diagram of a specific implementation of obtaining key point information according to an exemplary embodiment;

[0031] FIG8 is a hardware structure diagram of a behavior analysis system according to an exemplary embodiment;

[0032] FIG9 is a structural block diagram of a behavior analysis device according to an exemplary embodiment;

[0033] FIG10 is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0034] Fig. 11 is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0035] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0036] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0037] The rise in the use of electronic nicotine delivery systems (ENDS), or e-cigarettes, is believed to increase the risk of addiction. Therefore, understanding and evaluating the behavior of ENDS-related addiction mouse models is fundamental to studying the neuropsychiatric effects of e-cigarettes. As previously mentioned, current assessments of addictive behavior are primarily based on nicotine withdrawal-induced anxiety in mice. Because existing studies lack tools to demonstrate and assess the real-time behavioral state of smoke exposure in animal models, behaviors associated with nicotine withdrawal-induced anxiety are generally tested after exposure to ENDS. The lack of behavioral testing during ENDS exposure has led to little understanding of mouse behavior during e-cigarette exposure. Consequently, relying on traditional assessment methods (such as conditioned place preference and elevated maze tests) not only fails to reveal these behavioral characteristics in mice, but also lacks efficiency and accuracy in behavioral testing of ENDS-related addiction mouse models.

[0038] From the above, it can be seen that the relevant technologies still have the defects of low efficiency and insufficient accuracy of behavior analysis.

[0039] To this end, the behavior analysis method provided in this application can effectively improve the efficiency of behavior analysis. Accordingly, the behavior analysis method is applicable to a behavior analysis device, which can be deployed in an electronic device. The electronic device can be a computer device configured with a von Neumann architecture, for example, the computer device includes a desktop computer, a laptop computer, a server, etc.

[0040] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0041] Figure 1 is a schematic diagram of an implementation environment involved in a behavior analysis method. It should be noted that this implementation environment is only an example adapted to the present invention and should not be considered to provide any limitation on the scope of application of the present invention.

[0042] The implementation environment includes a collection end 110 and a service end 130 .

[0043] Specifically, the acquisition terminal 110 can be considered an image acquisition device, including but not limited to electronic devices with a camera, still camera, or video camera function. Furthermore, the acquisition terminal 110 also has the function of delivering electronic cigarette aerosol (hereinafter referred to as smoke). For example, the acquisition terminal 110 can be a smoke exposure device, which includes a smoke delivery unit and an image capture unit. The smoke delivery unit is used to deliver the same or different smoke to at least one detection target, and the image capture unit is used to capture and image the smoke delivery process of the same or different smoke to at least one detection target.

[0044] Server 130 can be an electronic device such as a desktop computer, laptop computer, or server. It can also be a computer cluster consisting of multiple servers, or even a cloud computing center consisting of multiple servers. Server 130 is used to provide background services, such as, but not limited to, behavior analysis services.

[0045] A network communication connection is pre-established between the server 130 and the acquisition terminal 110 via a wired or wireless method, and data transmission between the server 130 and the acquisition terminal 110 is achieved via the network communication connection. The transmitted data includes but is not limited to images in an image sequence.

[0046] In one application scenario, through the interaction between the acquisition terminal 110 and the server terminal 130, the acquisition terminal 110 transmits smoke to the detection target, shoots and collects the detection target to obtain an image, and uploads the image to the server terminal 130 to request the server terminal 130 to provide behavior analysis services for the image.

[0047] For the server 130, after receiving the image sequence formed by the images of the detection target uploaded by the acquisition terminal 110, it calls the behavior analysis service to perform behavior analysis on the detection target in each image in the image sequence under smoke exposure conditions, and obtains the behavior difference results of the detection target, so as to solve the problem of low behavior analysis efficiency existing in related technologies.

[0048] Please refer to FIG. 2 . An embodiment of the present application provides a behavior analysis method. The method is applicable to an electronic device, which may be the server 130 in the implementation environment shown in FIG. 1 .

[0049] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0050] As shown in FIG2 , the method may include the following steps:

[0051] Step 210: Acquire an image sequence.

[0052] First, it should be noted that the image sequence includes at least one frame of imagery, each of which is generated by a camera capturing at least one detection target while the smoke exposure device is delivering the same or different smoke streams to the target. The camera can be fixedly positioned around the detection target, for example, or it can be a flexibly movable image capture unit within the smoke exposure device.

[0053] It is understood that an image sequence includes at least one frame of image, and the camera captures a video segment or multiple images through continuous or multiple independent captures, and behavior analysis is performed based on the captured video segment or multiple images. In other words, the image sequence can refer to a video segment, and the frame of image used for behavior analysis can be a video frame in the video segment; alternatively, the image sequence can refer to multiple images, and the frame of image used for behavior analysis can be one of the multiple images.

[0054] The image sequence may be obtained from a plurality of frames of images captured in real time by a camera device and uploaded to a computer device in real time by the camera device, or may be a plurality of frames of images captured by a camera device during a historical period pre-stored in the computer device.

[0055] Then, the subsequent behavior analysis can be based on several frames of images taken in real time to improve the real-time nature of the behavior analysis, or it can be based on several frames of images stored in advance to improve the efficiency of the behavior analysis. For example, it can be performed when the electronic device has a small amount of processing tasks, or it can be performed according to the operating instructions of the experimenter. This embodiment does not limit this.

[0056] Secondly, the detection target refers to any object exposed to smoke, which can also be understood as any object exposed to smoke captured in the image. The object can be a mouse, rat, human, etc. to be analyzed, and is not limited here.

[0057] The smoke exposure device can deliver either nicotine-containing or nicotine-free smoke to the detection target. Accordingly, different smoke types are considered different smoke exposure conditions. By delivering different smoke types to at least one detection target through the smoke delivery unit of the smoke exposure device, the at least one detection target can be considered to be under different smoke exposure conditions, thereby enabling analysis of the at least one detection target's behavior under different smoke exposure conditions.

[0058] Step 230 : performing posture estimation on each detection target in each image to obtain key point information of each detection target at different exposure times.

[0059] It should be noted that since the shape of each detection target in each image included in the image sequence is not fixed, in order to accurately identify the motion posture of the detection target in each frame of image, it is necessary to determine the key point information of the detection target in each frame of image.

[0060] The key point information includes the location of at least one key point in the image. That is, the key point of the detection target is uniquely identified by its location in the image. For example, the location can be described by the coordinates of each pixel in the image. The key point is a point on the detection target that can reflect the detection target's motion posture. For example, the key point can be a skeletal point on the detection target, or a point determined through experimentation that can reflect the detection target's motion state, which is not limited here.

[0061] Optionally, the key point information of this embodiment can be obtained based on a posture estimation model. The posture estimation model is a machine learning model that has been trained and has the ability to estimate the posture of the detection target in the image. For example, the posture estimation model can be a single-animal DeepLabCut model.

[0062] Step 250 : Based on the key point information of each detection target at different exposure times, quantitatively analyze the behavior of each detection target under at least one smoke exposure condition to obtain behavioral difference results of each detection target.

[0063] The behavioral difference results indicate differences in the behavioral characteristics of the target at different exposure times, for example, behavioral differences between different targets at the same exposure time, or differences in the same behavior of the same target at different exposure times. These behavioral difference results can be obtained by quantitatively analyzing the behavior of the target using statistical methods, such as, but not limited to, the test-test method and the one-way ANOVA method.

[0064] It is worth mentioning that the exposure time in this embodiment corresponds to the images in the image sequence. It can be understood that the exposure time is the shooting and acquisition time of the corresponding images in the image sequence. Of course, in other embodiments, each exposure time can be further divided into different exposure stages. The exposure times in different exposure stages can be defined according to different time points in the smoke delivery process based on the experimental purpose. For example, from the time dimension, different exposure times can be divided into several exposure stages, including early exposure, late exposure, acute exposure, subacute exposure, subchronic exposure, chronic exposure, etc. The exposure times can also be divided into exposure stages and withdrawal stages based on the different withdrawal conditions.

[0065] With the above-described embodiments, it is possible to acquire a sequence of images of the detection target in real time during smoke delivery, and automatically perform behavioral analysis using a computer device without the need for human intervention. This not only effectively improves the efficiency of behavioral analysis but also fully guarantees its accuracy. Furthermore, it eliminates the need for behavioral analysis after smoke delivery, making the behavioral analysis process simple and time-saving. Furthermore, by quantitatively analyzing the behavioral differences of each detection target, the behavioral characteristics of each detection target can be obtained in real time, allowing for accurate analysis of subtle differences in the target's behavior, further improving the accuracy and efficiency of behavioral analysis.

[0066] In an exemplary embodiment, step 250 may include the following steps:

[0067] In step 310 , the key point information of each detection target at different exposure times is quantified into position parameters and body parameters of each detection target at different exposure times.

[0068] Position parameters refer to parameters related to the position of the detection target at the corresponding exposure time, including but not limited to nose center distance and direction index. Body parameters refer to parameters related to the body changes of the detection target, including but not limited to body speed, nose speed, body angle, body length, and movement distance.

[0069] It should be noted that in order to facilitate the calculation of position parameters and body parameters, fixed key points need to be preset according to the experimental conditions. For example, appropriate positions are selected on the smoke exposure device and set as the center key point and the north key point. The positions of the center key point and the north key point on the smoke exposure device remain unchanged during the entire smoke delivery process. Accordingly, the center key point and the north key point also have corresponding pixel points in each image contained in the image sequence, so that the corresponding key point information can be obtained to assist in the behavioral analysis of each detection target, thereby improving the accuracy and efficiency of the behavioral analysis.

[0070] For example, as shown in Figure 3a, when the detection target is a mouse, the figure shows the preset center key point P_center and the mouse's key points P_nose, P_body, and P_tail. Among them, P_center is a preset center key point located at a fixed position in the smoke exposure device. This center key point is a point determined through experiments that can reflect the motion state of the detection target, such as the smoke outlet of the smoke delivery part in the smoke exposure device; P_nose, P_body, and P_tail are the key points of the mouse's nose, body, and tail, respectively, and can be considered as skeletal points on the detection target. Through these key points, position parameters and body parameters can be calculated. Among them, position parameters include nose center distance and direction index. Body parameters include body speed, nose speed, body angle, body length, and movement distance.

[0071] Specifically, each key point P_center, P_nose, P_body and P_tail is represented by the coordinates of the pixel points in the image. For the convenience of description, the calculation process uses each key point P_center, P_nose, P_body and P_tail to represent the coordinates of the corresponding pixel points in the image.

[0072] Then, the nose center distance represents the distance between the nose key point P_nose and the center key point P_center, and the calculation formula is ||P_nose-P_center||2. The direction index represents the position of the nose key point P_nose relative to other key points (such as the tail key point P_tail), and the calculation formula is

[0073] Body speed represents the movement distance of the key point of P_body in unit time, and the calculation formula is Nose speed represents the angle at which the nose key point P_nose deviates from the central axis per unit time. The calculation formula is: The body angle represents the angle at which the nose keypoint P_nose deviates from the central axis. The body length represents the distance from the nose keypoint to the body keypoint, plus the distance from the body keypoint to the tail keypoint, calculated as ||P_nose-P_body||2+||P_tail-P_body||2. The movement distance represents the integral of the distance traveled by the body keypoint P_body over time at the corresponding exposure time, calculated as Σ||ΔP_body||2.

[0074] It should be noted that in the above formula, ΔP_body is the movement distance of the P_body key point under the corresponding exposure time, ΔP_nose' is the rotation angle of the P_nose nose key point under the corresponding exposure time, Δt is the duration of the exposure phase, and the central axis refers to the line between the P_nose nose key point, the P_body body key point, and the P_tail tail key point.

[0075] Step 330 : Based on the position parameters and body parameters of each detection target at different exposure times, the behavior characteristics of each detection target at different exposure times are determined respectively.

[0076] The behavioral characteristics are used to describe the behavioral patterns and / or behavioral habits of the detection target during the smoke delivery process. Optionally, the behavioral characteristics may include nose touch behavior characteristics, exploration behavior characteristics, body behavior characteristics, etc.

[0077] The inventors discovered that behavioral changes in mice differed after one month of exposure to nicotine-containing and nicotine-free e-cigarettes. For example, considering the stress-relieving effect of nicotine, one month of exposure to nicotine-containing e-cigarettes resulted in reduced nose-touching behavior, increased body angles, decreased exploratory behavior, and decreased movement rates. In other words, the behavioral patterns or habits (nose-touching behavior, exploratory behavior) exhibited by mice under different smoke exposure conditions are related to the mice's body angles, movement rates, and other factors. Therefore, in this embodiment, the behavioral characteristics of each detection target at different exposure times are determined based on the positional and body parameters of each detection target at different exposure times, thereby facilitating analysis of the behavioral patterns or habits exhibited by each detection target under different smoke exposure conditions.

[0078] In one possible implementation, as shown in FIG4 , the process of determining the behavior characteristics may include the following different steps:

[0079] Step 410 , under different exposure times, for each detection target, determine the nose touch behavior characteristics of the detection target based on the nose center distance in the position parameter and the nose speed in the body parameter.

[0080] Among them, the nose contact behavior characteristics reflect the quantitative indicators of the target's nose contact behavior under different smoke exposure conditions. It can also be considered that the nose contact behavior characteristics are used to accurately describe the target's nose contact behavior under different smoke exposure conditions. Therefore, by determining the target's nose contact behavior characteristics, the target's nose contact behavior related to pleasantness can be identified. It can be understood that when the smoke makes the target pleasant, the number of times the target's nose contact behavior occurs will increase. That is, the stronger the target's pleasure caused by the smoke, the stronger the target's nose contact behavior.

[0081] In one possible implementation, the nose-touching behavior characteristics can be determined by the nose-center distance in the position parameter. Specifically, by presetting the center key point at the smoke outlet of the smoke exposure device, since the nose-center distance indicates the distance between the nose key point and the center key point, that is, the distance between the nose of the detection target and the smoke outlet, the nose-touching behavior of the detection target sniffing the smoke outlet can be quantified by obtaining the nose-center distance at different exposure stages. For example, a distance threshold for triggering the nose-touching behavior is preset. When the nose-center distance is less than the distance threshold, it indicates that the detection target has a nose-touching behavior of sniffing the smoke outlet. Then, by obtaining the number of times the nose-center distance of the detection target is less than the distance threshold at different exposure stages, the duration, and the total duration, etc., the nose-touching behavior of the detection target can be quantified as the nose-touching behavior characteristics of the detection target. That is, the stronger the pleasure the detection target produces in response to the smoke, the smaller the nose-center distance.

[0082] In one possible implementation, nose contact behavior characteristics can also be determined using nose speed, a physical parameter. For example, a speed threshold for triggering nose contact behavior can be preset. When the nose speed exceeds this speed threshold, it indicates that the target is sniffing the smoke outlet. In other words, the greater the target's pleasure from the smoke, the greater the nose speed.

[0083] Step 430 : determining the body behavior characteristics of the detection target based on the direction index in the position parameter and the body length and body angle in the body parameter.

[0084] Among them, the physical behavior characteristics reflect the physical changes of the target under different smoke exposure conditions. It can also be considered that the physical behavior characteristics are used to accurately describe the physical changes of the target under different smoke exposure conditions. For example, physical changes can refer to changes in the target's posture, such as stretching limbs or curling up. It is understood that when the target is exposed to smoke, it will produce corresponding physiological reactions (such as pleasure), and these physiological reactions can be reflected by the target's physical behavior. For example, whether physical behaviors such as stretching limbs and curling up occur, or whether physical behaviors such as sneezing and rapid breathing occur, these physical behaviors can be used to infer the physiological reactions of mice to smoke at different exposure stages. For example, stretching limbs reflects the target's pleasure when exposed to smoke. Therefore, by obtaining the target's directional index, body length, body angle, etc. at different exposure stages, and then quantifying the target's physical changes under different smoke exposure conditions based on these parameters, specifically quantifying the severity and frequency of changes in the target's body length, body angle, and directional index under different exposure conditions / stages, it is possible to objectively and accurately analyze the target's behavior.

[0085] Step 450 : determining the exploration behavior characteristics of the detection target based on the direction index in the position parameter and the body speed and movement distance in the body parameter.

[0086] Among them, the exploration behavior feature is used to describe the motion behavior of the detection target under smoke exposure conditions.

[0087] It can be understood that the detection target may engage in exploratory behavior at different exposure stages. For example, the detection target's movement distance becomes longer, and the length of time the detection target touches various corners in the smoke exposure device can both reflect that the detection target is exploring. Then, the movement behavior of the detection target at different exposure stages can be quantified based on the direction index in the position parameter and the body speed and movement distance in the body parameter, so as to determine the impact of the detection target's exploratory behavior at different exposure stages. For example, the duration and movement intensity of the detection target's exploratory behavior can be quantified by the movement distance and body speed. The frequency and duration of the specific position changes of the detection target's exploratory behavior can be determined by the direction index and movement distance combined with the central key point, and then the detection target's pleasure at different exposure stages can be analyzed. For example, if the detection target's movement distance becomes longer, and the specific position of the exploratory behavior changes frequently, touching various corners of the smoke exposure device, it indicates that the detection target is less affected under the corresponding exposure conditions / exposure stages.

[0088] It should be noted that the behavioral patterns / behavioral habits exhibited by different detection targets under different smoke exposure conditions are different. The position parameters and body parameters generated by the key point information are also different under different smoke exposure conditions, and the behavioral characteristics formed thereby are also different. In other words, the behavioral characteristics of each detection target under different smoke exposure conditions can uniquely describe the behavioral patterns / behavioral habits of each detection target under different smoke exposure conditions.

[0089] Step 350 , using statistical methods to analyze the differences in the behavioral characteristics of each detection target at different exposure times, and obtaining behavioral difference results of each detection target.

[0090] In a possible implementation, as shown in FIG5 , step 350 may include the following steps:

[0091] Step 510 , analyzing the differences in the behavioral characteristics of each detection target at different exposure times to obtain a first difference result for each detection target.

[0092] Among them, the first difference result is the different behavioral characteristics exhibited by the detection target at different exposure times. The first difference result can be used to evaluate the performance of the same behavioral characteristics of the detection target at different exposure stages.

[0093] Optionally, the first difference result can be achieved by a t-test method in statistical methods.

[0094] Step 530 , selecting at least two detection targets from each detection target, analyzing the differences in the behavioral characteristics of the at least two detection targets under the same exposure time, and obtaining a second difference result for each detection target.

[0095] The second difference result is the different behavioral characteristics exhibited by different detection targets at the same exposure time. The first difference result can be used to evaluate the differences in the behavioral characteristics of each detection target at the same exposure time.

[0096] Optionally, the second difference result can be achieved by a One-way ANOVA method in statistical methods.

[0097] Step 550 : Obtaining a behavior difference result of each detection target based on the first difference result and the second difference result of each detection target.

[0098] Through the above process, the behavioral differences of each detection target at different exposure stages are comprehensively evaluated through the first difference result and the second difference result, so that the behavioral differences of the detection target can be more comprehensively obtained and the accuracy of behavioral detection can be improved.

[0099] In an exemplary embodiment, the process of performing pose estimation on each detection target in each image is implemented based on a pose estimation model, wherein the pose estimation model is a machine learning model that has been trained and has the ability to estimate the pose of the detection target in the image.

[0100] As shown in Figure 6a, the pose estimation model includes an input layer 601, a convolutional layer 603, a pooling layer 605, a fully connected layer 607, and an output layer 609. The input layer 601 is used to acquire images from an image sequence, the convolutional layer 603 and the pooling layer 605 are used to extract local features from each image in the image sequence, the fully connected layer 607 is used to map the local features of the image, and the output layer 609 is used to convert the output of the fully connected layer 607 into a probability distribution, ultimately obtaining the key point information of the detected target in each image.

[0101] As shown in FIG6b , in one possible implementation, the training process of the posture estimation model may include the following steps:

[0102] Step 710: Obtain a training set.

[0103] The training set includes multiple training samples, which are images with labels. The labels are used to indicate the true key points of the detection targets in the training samples.

[0104] Step 730: Train the pre-built machine learning model based on the training samples in the training set to obtain training results.

[0105] The training results are used to indicate the predicted key points of the detection targets in the training samples.

[0106] Step 750: Determine whether the training of the machine learning model is complete based on the error between the predicted key points in the training results and the actual key points in the labels.

[0107] The error calculation is implemented based on a specified algorithm function constructed based on the training samples and the parameters of the machine learning model. The specified algorithm function includes but is not limited to a loss function.

[0108] If the error is less than the convergence condition, the posture estimation model is obtained; otherwise, the parameters of the machine learning model are updated to continue training.

[0109] Therefore, the parameters of the pre-built machine learning model are iteratively optimized through training samples, so that the error between the predicted key points in the training results and the real key points in the labels meets the convergence conditions, and then the mathematical mapping relationship between each pixel point in the image and different key point categories is optimized, and finally a posture estimation model that has completed training is obtained.

[0110] In the above process, when the machine learning model completes model training, a posture estimation model can be obtained, which enables the posture estimation model to have the ability of classification prediction, that is, the ability to estimate the posture of the detection target, thereby improving the accuracy of key point information acquisition and enabling the action recognition process to be realized.

[0111] The following describes the posture estimation process in detail based on the network structure of the posture estimation model in Figure 6a:

[0112] As shown in FIG7 , in an exemplary embodiment, step 230 may include the following steps:

[0113] Step 231 : determining a current image among the images, performing feature representation extraction at different levels on the current image, and obtaining local features of the current image.

[0114] Among them, local features can represent pixel information at different levels of the detection target in the current image. For example, when the detection object is a mouse, the pixel information can be the skeletal key point information on the mouse's body parts, such as the nose key point, body key point, etc. Correspondingly, local features can represent the mouse's nose key point, body key point, etc.

[0115] In other words, local features accurately describe the characteristics of different body parts of the detection target through digital information. It should be understood that different detection targets will have different bone key point information on the corresponding body parts in the image, and the extracted local features will also be different.

[0116] In step 233 , the local features of the current image are mapped to different key point categories, and the mapped different key point categories are converted into probability distributions to predict the probability that the key points of each detection target in the current image belong to different key point categories.

[0117] Specifically, the essence of probability distribution transformation is classification prediction, that is, the classification function of the output layer of the posture estimation model is used to calculate the probability that each pixel represented by the local features of the current image belongs to different key point categories, thereby predicting the key point category to which each pixel in the current image belongs.

[0118] For example, assuming that the current image contains a nose key point category, a body key point category, a tail key point category, and a background category, then, assuming that the probability of a pixel point in the current image belonging to the nose key point category is P0, the probability of belonging to the body key point category is P1, the probability of belonging to the tail key point is P2, and the probability of belonging to the background is P3.

[0119] If P0 is the largest, it means that the pixel belongs to the nose key point category. Similarly, if P1 is the largest, it means that the pixel belongs to the body key point category. If P2 is the largest, it means that the pixel belongs to the tail key point category. If P4 is the largest, it means that the pixel belongs to the background category.

[0120] Therefore, after completing the classification prediction of all pixels in the current image, it can be determined that the nose key point in the image is composed of pixels belonging to the nose key point category), the body key point (composed of pixels belonging to the body key point category), and the tail key point (composed of pixels belonging to the tail key point category).

[0121] In step 235, based on the probability that the key points of each detection target belong to different key point categories, the key point information of each detection target at the current exposure time is obtained, until the posture estimation of each detection target in each image is completed, and the key point information of each detection target at different exposure times is obtained.

[0122] In one possible implementation, as shown in Figure 7a, the detection targets are 8 mice. A coordinate system is established by pre-set north key points and center key points. The image sequence containing the 8 detection targets is input into the single-animalDeepLabCut model for pose estimation to obtain the key point information of each detection target. In Figure 7a, the key point information of each detection target is represented as a trajectory diagram changing with time, with the horizontal axis representing time and the vertical axis representing the number of each detection target and the different key points of the detection target under each number.

[0123] Through the above process, the key point information of the image sequence can be quickly extracted through the posture estimation model, which improves the efficiency of action recognition.

[0124] Referring to FIG8 , an embodiment of the present application provides a behavior analysis system 800 for executing the behavior analysis method of the present application. The behavior analysis system 800 includes, but is not limited to, a smoke exposure device 801 and an analysis terminal 803 . The smoke exposure device 801 includes a smoke delivery unit 810 and an image capture unit 830 .

[0125] The smoke delivery unit 810 is used to deliver the same or different smoke to at least one detection target.

[0126] The image camera unit 830 is used to capture and collect images when the smoke conveying unit conveys the same or different smoke to at least one detection target, and transmit the images to the analysis end.

[0127] The analysis terminal 803 is configured to perform a behavior analysis on the detection target in the image under smoke exposure conditions based on the received image and according to a behavior analysis method.

[0128] Compared with related technologies, behavioral analysis can be automatically performed with the help of computer equipment without the need for human intervention, which not only effectively improves the efficiency of behavioral analysis, but also fully guarantees the accuracy of behavioral analysis. By identifying the behavioral characteristics of the detection target based on the acquired position parameters and body parameters, more precise behavioral characteristics can be obtained, thereby improving the accuracy of behavioral analysis. At the same time, by quantitatively analyzing the behavioral difference results of each detection target, the behavioral characteristics of each detection target can be obtained in real time, and the subtle differences in the detection target behavior can be accurately analyzed, thereby improving the accuracy and efficiency of behavioral analysis. By comprehensively evaluating the behavioral differences of each detection target at different exposure stages through the first difference result and the second difference result, the behavioral differences of the detection target can be more comprehensively obtained, improving the accuracy of behavioral detection.

[0129] The following are embodiments of the device of the present application, which can be used to perform the behavior analysis method involved in the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the behavior analysis method involved in the present application.

[0130] Please refer to FIG. 9 . An embodiment of the present application provides a behavior analysis device 900 , including but not limited to: an image acquisition module 910 , a posture estimation module 930 , and an analysis module 950 .

[0131] The image acquisition module 910 is configured to acquire an image sequence, wherein the image sequence includes at least one frame of image, each of which is generated during a process in which the smoke exposure device delivers the same or different smoke to at least one detection target;

[0132] a posture estimation module 930 for performing posture estimation on each of the detection targets in each of the images to obtain key point information of each of the detection targets at different exposure times;

[0133] The analysis module 950 is used to quantitatively analyze the behavior of each detection target under at least one smoke exposure condition based on the key point information of each detection target at different exposure times, and obtain behavioral difference results of each detection target.

[0134] It should be noted that the behavior analysis device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing behavior analysis. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the behavior analysis device will be divided into different functional modules to complete all or part of the functions described above.

[0135] In addition, the behavior analysis device and behavior analysis method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.

[0136] Fig. 10 is a schematic diagram showing the structure of an electronic device according to an exemplary embodiment. The electronic device is applicable to the server 130 in the implementation environment shown in Fig. 1 .

[0137] It should be noted that the electronic device is only an example adapted for the present application and should not be considered to provide any limitation on the scope of use of the present application. The electronic device should not be interpreted as needing to rely on or necessarily having one or more components in the exemplary electronic device 2000 shown in FIG10.

[0138] The hardware structure of the electronic device 2000 may vary greatly due to different configurations or performances. As shown in FIG10 , the electronic device 2000 includes a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

[0139] Specifically, the power supply 210 is used to provide operating voltage for various hardware devices on the electronic device 2000 .

[0140] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices, for example, the interaction between the acquisition terminal 110 and the service terminal 130 in the implementation environment shown in FIG1 .

[0141] Of course, in other examples adapted by this application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, as shown in FIG10 , which is not specifically limited here.

[0142] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include an operating system 251, application 253 and data 255, etc. The storage method can be temporary storage or permanent storage.

[0143] Among them, the operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 2000, so as to enable the central processing unit 270 to calculate and process the massive data 255 in the memory 250. It can be Windows ServerTM, Mac OSXTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0144] Application 253 is a computer-readable instruction that performs at least one specific task based on operating system 251. It may include at least one module (not shown in FIG10 ), each of which may include computer-readable instructions for electronic device 2000. For example, a behavior analysis device may be considered as application 253 deployed on electronic device 2000.

[0145] The data 255 may be photos, pictures, etc. stored in a disk, or may be image sequences, etc. stored in the memory 250 .

[0146] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer-readable instructions stored in the memory 250, thereby performing operations and processing on the massive amount of data 255 in the memory 250. For example, the behavior analysis method may be performed by the central processing unit 270 reading a series of computer-readable instructions stored in the memory 250.

[0147] In addition, the present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.

[0148] Please refer to FIG. 11 . An electronic device 4000 is provided in an embodiment of the present application. The electronic device 4000 may include a desktop computer, a laptop computer, a server, etc.

[0149] In FIG. 11 , the electronic device 4000 includes at least one processor 4001 and at least one memory 4003 .

[0150] Data exchange between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in FIG11, but this does not mean that there is only one bus or one type of bus.

[0151] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0152] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0153] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to these.

[0154] Computer-readable instructions are stored in the memory 4003 , and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .

[0155] The computer-readable instructions are executed by one or more processors 4001 to implement the behavior analysis method in the above embodiments.

[0156] In addition, an embodiment of the present application provides a storage medium on which computer-readable instructions are stored. The computer-readable instructions are executed by one or more processors to implement the behavior analysis method as described above.

[0157] In an embodiment of the present application, a computer program product is provided. The computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the behavior analysis method described above.

[0158] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0159] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A behavior analysis method under smoke exposure conditions, characterized in that: The method comprises: Acquire an image sequence, wherein the image sequence includes at least one frame of image, each of which is generated during a process in which the smoke exposure device delivers the same or different smoke to at least one detection target; Performing posture estimation on each of the detection targets in each of the images to obtain key point information of each of the detection targets at different exposure times; According to the key point information of each detection target at different exposure times, the behavior of each detection target under at least one smoke exposure condition is quantitatively analyzed to obtain the behavior difference results of each detection target.

2. The method according to claim 1, characterized in that The quantitative analysis of the behavior of each detection target under at least one smoke exposure condition is performed according to the key point information of each detection target at different exposure times to obtain the behavior difference results of each detection target, including: quantifying the key point information of each detection target at different exposure times into position parameters and body parameters of each detection target at different exposure times; Based on the position parameters and body parameters of each of the detection targets at different exposure times, respectively determine the behavior characteristics of each of the detection targets at different exposure times; Statistical methods are used to analyze the differences in the behavioral characteristics of each detection target at different exposure times to obtain behavioral difference results of each detection target.

3. The method according to claim 2, characterized in that The determining of the behavior characteristics of each detection target at different exposure times based on the position parameters and body parameters of each detection target at different exposure times includes: Under different exposure times, for each of the detection targets, according to the nose center distance in the position parameter and the nose speed in the body parameter, determine the nose touch behavior characteristics of the detection target; or Determining the body behavior characteristics of the detection target according to the direction index in the position parameter and the body length and body angle in the body parameter; or The exploration behavior characteristics of the detection target are determined according to the direction index in the position parameter and the body speed and movement distance in the body parameters.

4. The method according to claim 2, characterized in that The statistical method is used to analyze the differences in the behavioral characteristics of each of the detection targets at different exposure times to obtain the behavioral difference results of each of the detection targets, including: Analyzing the differences in the behavioral characteristics of each of the detection targets at different exposure times to obtain a first difference result of each of the detection targets; Selecting at least two detection targets from the detection targets, analyzing the differences between the behavior characteristics of the at least two detection targets under the same exposure time, and obtaining a second difference result for each detection target; The behavior difference result of each detection target is obtained from the first difference result and the second difference result of each detection target.

5. The method according to any one of claims 1 to 4, characterized in that: The performing posture estimation on each of the detection targets in each of the images to obtain key point information of each of the detection targets at different exposure times includes: Determine a current image among the images, extract feature representations at different levels for the current image, and obtain local features of the current image; Mapping the local features of the current image to different key point categories, and performing probability distribution conversion on the mapped different key point categories, to predict the probability that the key points of each detection target in the current image belong to different key point categories; Based on the probability that the key points of each detection target belong to different key point categories, obtaining the key point information of each detection target at the current exposure time; Until the posture estimation of each detection target in each image is completed, the key point information of each detection target at different exposure times is obtained.

6. The method according to claim 5, characterized in that The posture estimation of each of the detection targets in each of the images is implemented based on a posture estimation model, where the posture estimation model is a machine learning model that has been trained and has the ability to estimate the posture of the detection targets in the images; The method further comprises: Acquire a training set; the training set includes a plurality of training samples, and the training samples are images carrying labels; the labels are used to indicate the real key points of the detection target in the training samples; Based on each of the training samples in the training set, a pre-built machine learning model is trained to obtain a training result; the training result is used to indicate the predicted key points of the detection target in the training sample; Based on the error between the predicted key points in the training result and the real key points in the label, Determining whether training of the machine learning model is complete; If yes, the posture estimation model is obtained; otherwise, the parameters of the machine learning model are updated to continue training.

7. A behavior analysis system, characterized in that: The system comprises: a smoke exposure device and an analysis end; the smoke exposure device comprises a smoke delivery unit and an image camera unit; wherein, The smoke delivery unit delivers the same or different smoke to at least one detection target; The image camera unit is configured to capture and collect images during the process in which the smoke conveying unit conveys the same or different smoke to at least one of the detection targets, and transmit the images to the analysis end; The analysis end, based on the received image, performs a behavior analysis on the detection target in the image under smoke exposure conditions according to the behavior analysis method according to any one of claims 1 to 6.

8. A behavior analysis device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image sequence, wherein the image sequence includes at least one frame of image, each of which is generated during a process in which the smoke exposure device delivers the same or different smoke to at least one detection target; A posture estimation module, used to perform posture estimation on each of the detection targets in each of the images to obtain key point information of each of the detection targets at different exposure times; The analysis module is used to quantitatively analyze the behavior of each detection target under at least one smoke exposure condition according to the key point information of each detection target at different exposure times, so as to obtain the behavior difference results of each detection target.

9. An electronic device, characterized in that: include: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more of the processors so that the electronic device implements the behavior analysis method according to any one of claims 1 to 6.

10. A storage medium having computer-readable instructions stored thereon, characterized in that: The computer-readable instructions are executed by one or more processors to implement the behavior analysis method according to any one of claims 1 to 6.

Images