Behavior recognition model training and behavior recognition method, device, system and medium

By performing preset time interval segmentation and marking of mouse behavior observation videos, an end-to-end behavior recognition model is trained, which solves the problem of high training time cost of mouse behavior recognition models in the existing technology, and achieves efficient behavior recognition results analyze.

CN114241376BActive Publication Date: 2024-11-12SHENZHEN INST OF ADVANCED TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111536765.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-15
Publication Date
2024-11-12
Estimated Expiration
2041-12-15

AI Technical Summary

Technical Problem

In the prior art, the training time of mouse behavior recognition models is high, and end-to-end training cannot be achieved, especially when dealing with various solutions, such as video understanding, pose estimation and image classification, which has high data annotation cost and model training efficiency. Low problem.

Method used

By obtaining the behavior of the target object and dividing it into video clips at preset time intervals, using the video clip marked by behavior category as the model training sample, input it into the initial behavior recognition model for training until the output result satisfies the preset classification. Evaluate the indicators and obtain the target behavior recognition model.

Benefits of technology

It reduces the time cost of model training, realizes end-to-end behavior recognition model training, and eliminates the need to label video data frame by frame, improving the efficiency and accuracy of model training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114241376B_ABST
    Figure CN114241376B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention disclose behavior recognition model training and behavior recognition methods, devices, systems and media, wherein the method includes: obtaining a behavior observation video of a target object, and segmenting the behavior observation video at preset time intervals to obtain multiple video clips; inputting each video clip labeled with a behavior category as a model training sample into an initial behavior recognition model to train the initial behavior recognition model; when the output result of the initial behavior recognition model meets the preset classification evaluation index, a target behavior recognition model is obtained. The technical solution of the embodiments of the present invention solves the problem of high time cost of training a behavior recognition model based on machine recognition in the prior art, realizes an end-to-end model training method, and does not need to annotate the collected video data frame by frame, but analyzes the input behavior video clip data to obtain the corresponding behavior recognition result, thereby reducing the time cost of model training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of machine vision technology, and in particular to a behavior recognition model training and behavior recognition method, device, system and medium. Background Art

[0002] In the existing technology, specific behaviors of mice are mostly identified by combining relevant personnel watching videos with machine recognition systems. That is, specific behaviors are first identified by machine recognition in a high-throughput and unbiased manner, and then further confirmed by humans to clarify the differences between individual mice and obtain more detailed data on a time scale.

[0003] Among them, there are three solutions that can be used to identify mouse behavior by combining computer vision technology. First, use video understanding technology (such as deepethogram) on the mouse video to analyze the behavior category within a time period; second, estimate the posture of the mouse and then perform operations such as classification on the posture sequence obtained; third, classify each frame of the shot and identify the behavior of each frame.

[0004] However, the first solution mentioned above cannot achieve end-to-end training, and in the early data preparation work, experts are required to mark the video frames one by one, and the time cost of model training is relatively high; for the second solution, it is impossible to effectively estimate the posture of two mice of the same color; for the third solution, to obtain a better classification model, experts need to label a large number of pictures, and the time cost of model training is relatively high; the training process of the mouse behavior recognition model needs to be optimized. Summary of the invention

[0005] The embodiments of the present invention provide a behavior recognition model training and behavior recognition method, device, system and medium to achieve training to obtain an end-to-end behavior recognition model, analyze the input behavior video data, obtain corresponding behavior recognition results, reduce the time cost of model training, and optimize the model training process.

[0006] In a first aspect, an embodiment of the present invention provides a behavior recognition model training method, the method comprising:

[0007] Obtaining a behavior observation video of a target object, and segmenting the behavior observation video at preset time intervals to obtain a plurality of video segments;

[0008] Inputting each video clip marked with behavior categories as a model training sample into an initial behavior recognition model to train the initial behavior recognition model;

[0009] When the output result of the initial behavior recognition model meets the preset classification evaluation index, the target behavior recognition model is obtained.

[0010] In a second aspect, an embodiment of the present invention further provides a behavior recognition method, the method comprising:

[0011] Obtain the behavior video of the target object to be analyzed;

[0012] The behavior video to be analyzed is input into the behavior recognition model trained by the behavior recognition model training method described in any embodiment to obtain the behavior recognition result of the target object.

[0013] In a third aspect, an embodiment of the present invention further provides a behavior recognition model training device, the device comprising:

[0014] A data acquisition module, used to acquire a behavior observation video of a target object, and to segment the behavior observation video into multiple video segments at preset time intervals;

[0015] A model training module, used to input each video clip marked with a behavior category as a model training sample into an initial behavior recognition model to train the initial behavior recognition model;

[0016] The model determination module is used to obtain a target behavior recognition model when the output result of the initial behavior recognition model meets the preset classification evaluation index.

[0017] In a fourth aspect, an embodiment of the present invention further provides a behavior recognition device, the device comprising:

[0018] The module for acquiring data to be identified is used to acquire the behavior video of the target object to be analyzed;

[0019] The behavior recognition module is used to input the behavior video to be analyzed into the behavior recognition model trained by the behavior recognition model training method described in any embodiment to obtain the behavior recognition result of the target object.

[0020] In a fifth aspect, an embodiment of the present invention further provides a behavior recognition system, the system comprising:

[0021] A plurality of stereoscopic experimental open fields of equal volume and adjacent in space, a plurality of video information acquisition devices and a video data processing subsystem;

[0022] Wherein, the stereoscopic experimental open field is used to accommodate the observed experimental objects;

[0023] The video information acquisition device is fixed at a preset position corresponding to the stereoscopic experimental open field, and is used to synchronously acquire the behavioral video data of the experimental subject in each of the experimental open fields within a preset time period;

[0024] The video data processing subsystem is used to identify the preset behavior of the experimental subject based on the behavior video data.

[0025] In a sixth aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0026] one or more processors;

[0027] A memory for storing one or more programs;

[0028] When the one or more programs are executed by the one or more processors, the one or more processors implement the behavior recognition model training method or behavior recognition method provided by any embodiment of the present invention.

[0029] In a seventh aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a behavior recognition model training method or a behavior recognition method as provided in any embodiment of the present invention.

[0030] The embodiments of the above invention have the following advantages or beneficial effects:

[0031] The embodiment of the present invention obtains a behavior observation video of the target object, and divides the behavior observation video at preset time intervals to obtain multiple video clips; each video clip marked with a behavior category is used as a model training sample, and is input into an initial behavior recognition model to train the initial behavior recognition model; when the result of the loss function of the behavior recognition model meets the preset conditions, the target behavior recognition model is obtained. The technical solution of the embodiment of the present invention solves the problem of high time cost of training behavior recognition models based on machine recognition in the prior art. It does not need to annotate the collected video data frame by frame, and can also train an end-to-end behavior recognition model, analyze the input behavior video data, and obtain corresponding behavior recognition results, thereby reducing the time cost of model training and optimizing the model training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a behavior recognition model training method provided in Example 1 of the present invention;

[0033] Figure 2 is a structural diagram of a behavior recognition model provided in Embodiment 1 of the present invention;

[0034] Figure 3 is a flow chart of a behavior recognition method provided by Embodiment 2 of the present invention;

[0035] Figure 4 It is a structural diagram of a behavior recognition model training device provided in Embodiment 3 of the present invention;

[0036] Figure 5 is a structural diagram of a behavior recognition device provided by Embodiment 4 of the present invention;

[0037] Figure 6 is a structural diagram of a behavior recognition system provided by Embodiment 5 of the present invention;

[0038] Figure 7 This is a diagram of an application example of a behavior recognition system provided by Embodiment 5 of the present invention;

[0039] Figure 8 It is a structural diagram of a computer device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0041] Embodiment 1

[0042] Figure 1 This is a flow chart of a behavior recognition model training method provided in Embodiment 1 of the present invention. This embodiment can be applied to the case of training an end-to-end behavior recognition model based on the behavior observation video of the observed object. The method can be executed by a behavior recognition model training device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0043] like Figure 1 As shown, the behavior recognition model training method includes the following steps:

[0044] S110, obtaining a behavior observation video of a target object, and segmenting the behavior observation video at preset time intervals to obtain a plurality of video segments.

[0045] Usually, in animal behavioral experiments, a large number of experiments are conducted to observe the behavior of the experimental subjects, and scientific research is carried out based on the observed behavioral results. In experimental research, the amount of data that needs to be collected is very large. In order to improve the efficiency of data collection and avoid subjective experimental errors caused by human identification, machine recognition can be used to identify the specific observed behaviors of the experimental subjects.

[0046] The target object is the experimental object that needs to be observed, which can be an animal that needs experimental observation, such as mice, kittens, or fish. When collecting behavioral data, the activity image of the target object in the set activity area (such as the experimental open field) can be captured by a camera, that is, a behavioral observation video of the target object can be obtained. For example, in a specific example, the target object is a mouse, and the mating behavior of the mouse needs to be observed. The characteristics of mouse mating behavior are that the mating frequency is extremely low (about 0.3% under natural conditions), the duration of a single mating is short (about 5 seconds on average), the experimental time is long (several hours of observation), and the mating characteristics are obvious (one mouse shakes rapidly on another mouse).

[0047] When the behavior observation video is segmented, the length of the video segment can be determined according to the characteristics of the specific behavior of the target object to be observed. For example, the mating time of mice is short, about 5 seconds on average, so the behavior observation video can be segmented with seconds as the preset time interval to obtain multiple video segments. Of course, the time interval can be 1 second, 2 seconds or other preset time interval lengths. Then, the behavior observation video data can be annotated in units of video segments as samples for behavior recognition model training, without the need to annotate the behavior observation video frame by frame. In a preferred embodiment, the video selection can be further streamlined to reduce the amount of data calculation. For example, for each video segment after segmentation, a certain number of video image frames can be extracted as training samples for the behavior recognition model according to a fixed step length. Assuming that the sampling rate of the video shot by the camera is 60 frames per second, 30 frames can be extracted from a video segment with a duration of 1 second as 1 training sample.

[0048] In an optional implementation, each video clip may be horizontally rotated, vertically flipped, and / or horizontally flipped to expand the model training data.

[0049] S120: Input each video clip marked with a behavior category as a model training sample into an initial behavior recognition model to train the initial behavior recognition model.

[0050] During the model training process, each training sample is input into the initial behavior recognition model, which learns the features in the training sample and outputs a classification result for the training sample. Then, the result is compared with the label corresponding to the training sample to determine the loss function of the initial behavior model. Then, the model parameters of the initial behavior recognition model are adjusted according to the value of the loss function to update the initial behavior recognition model.

[0051] Specifically, the initial behavior recognition model learns and trains the model training samples. First, the training samples are preprocessed to perform preliminary feature extraction to obtain initial features. This step uses a preset feature extraction model to extract features from each video frame in the video clip, and performs maximum pooling on the features of each video frame. The preset feature extraction model is preferably a ResNet50 model (pre-trained residual neural network). Then, the features to be learned are constructed based on the features processed by maximum pooling.

[0052] In a preferred embodiment, sampling can be further performed in each feature processed by the maximum pooling according to a preset step size, such as extracting the features corresponding to the maximum pooling of 10 frames of video images from the features of 30 frames of video images in 1 second, and stacking them to obtain a feature, i.e., the feature to be learned, so as to further reduce the computational consumption and avoid calculating redundant data. If the feature dimension extracted from each frame of the video image is 2048*1, then the dimension of the feature to be learned obtained by stacking the video image frames in time order is 10*2048*1=20480.

[0053] After the features to be learned are initially obtained, the preset convolution kernels are further used to learn the features. In a preferred real-time method, four groups of different convolution kernels are used to extract the features to be learned, and the four groups of different convolution kernels include three groups of time convolution kernels and one group of space convolution kernels. Then, the behavior recognition results are output based on the extracted features, and the parameters of the initial behavior recognition model are adjusted based on the behavior recognition results.

[0054] In a specific example, the four sets of convolution kernel structures for learning the features to be learned are as follows: Figure 2 As shown. Figure 2 In the figure, the feature to be learned (i.e., video feature) is a feature data with a dimension of 10*2048*1, and four different convolution kernels are used to further extract the feature to be learned.

[0055] In the first data link on the left, the feature to be learned is first extracted through 8 convolution kernels (conv) with a dimension of 2*49, and then 3*1 maximum pooling is performed, and then 8 convolution kernels with a dimension of 1*5 and a stride of 1*5 are further used to extract features to obtain the first feature. In the second data link on the left, the feature to be learned is first extracted through 8 convolution kernels (conv) with a dimension of 5*49, and then 2*1 maximum pooling is performed, and then 8 convolution kernels with a dimension of 1*5 and a stride of 1*5 are further used to extract features to obtain the second feature. In the third data link on the left, the feature to be learned is first extracted through 8 convolution kernels (conv) with a dimension of 8*49, and then 8 convolution kernels with a dimension of 1*5 and a stride of 1*5 are further used to extract features to obtain the third feature. In the first data link on the right, 16 convolution kernels (conv) with a dimension of 1*49 are used to extract the features to be learned, and then 1*5 maximum pooling is performed, and further 16 convolution kernels with a dimension of 1*20 are used to extract features, and then 1*5 maximum pooling is performed again, and finally 16 convolution kernels (conv) with a dimension of 1*15 are used to extract features from the second pooling result to obtain the fourth feature.

[0056] Furthermore, the first feature, the second feature, the third feature and the fourth feature finally extracted are stacked (concatenated) and input into the first fully connected layer (dense) including 512 neurons / units (units) for classification. In this fully connected layer, relu is used as the activation function, and the L2 regularization coefficient is 0.001. The output result will be filtered by the dropout layer (screening coefficient 0.5). The result output by the first fully connected layer is filtered by the dropout layer and input into the second fully connected layer (dense) containing two neurons (unit), and finally the behavior category is output, that is, the result after learning the feature to be learned. Taking the identification of mouse mating behavior as an example, the output behavior category is whether the mouse behavior in the video clip is mating behavior.

[0057] S130: When the output result of the initial behavior recognition model meets the preset classification evaluation index, a target behavior recognition model is obtained.

[0058] After the initial behavior recognition model has undergone multiple iterations of learning, when the learning output of the training sample meets the preset classification evaluation index, the model training process can be ended, indicating that the accuracy of the target behavior recognition model obtained by the final training for specific behavior recognition has reached the preset accuracy level. Among them, the classification evaluation index can be any one of the indicators such as confusion matrix, accuracy, precision, recall rate or F1 Score according to the specific model training situation.

[0059] The technical solution of this embodiment obtains a behavior observation video of the target object, and divides the behavior observation video at preset time intervals to obtain multiple video clips; each video clip marked with a behavior category is used as a model training sample, input into the initial behavior recognition model, and the initial behavior recognition model is trained; when the result of the loss function of the behavior recognition model meets the preset conditions, the target behavior recognition model is obtained. The technical solution of the embodiment of the present invention solves the problem of high time cost of training behavior recognition models based on machine recognition in the prior art. It does not need to annotate the collected video data frame by frame, and can also train an end-to-end behavior recognition model, analyze the input behavior video data, and obtain the corresponding behavior recognition results, thereby reducing the time cost of model training and optimizing the model training process.

[0060] Embodiment 2

[0061] Figure 2 This is a flow chart of a behavior recognition method provided in Embodiment 2 of the present invention. This embodiment and the behavior recognition model training method in the above embodiment belong to the same inventive concept, and further describes the process of applying the trained behavior recognition model. The method can be executed by a behavior recognition device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0062] like Figure 2 As shown, the behavior recognition method includes the following steps:

[0063] S210: Obtain the behavior video of the target object to be analyzed.

[0064] The target object is the target object that needs to be observed and analyzed. The behavior video to be analyzed is the information collected by the camera device.

[0065] S220: Input the behavior video to be analyzed into a behavior recognition model trained by the behavior recognition model training method described in any embodiment to obtain a behavior recognition result of the target object.

[0066] Of course, the training samples of the behavior recognition model are also video information collected by observing the target object. Then, according to the collected video information, the behavior recognition model training method in the above embodiment is used to train and obtain the final behavior recognition model.

[0067] The behavior recognition model trained by the method of the above embodiment is an end-to-end model. By directly inputting the collected behavior video to be analyzed into the model, the corresponding output result can be obtained to determine whether the behavior of the target object in the input video is a specific behavior.

[0068] The technical solution of this embodiment can obtain the corresponding target object behavior analysis and recognition results by inputting the collected target object behavior video to be analyzed into the behavior recognition model pre-trained by the model training method of the above embodiment. The technical solution of the embodiment of the present invention solves the problem of efficiently identifying the target object's behavior and determining the behavior analysis results, thereby improving the accuracy of the output results of the existing behavior analysis model.

[0069] Embodiment 3

[0070] Figure 4 This is a structural diagram of a behavior recognition model training device provided in Example 3 of the present invention. This embodiment can be applied to model training based on behavior observation videos of observed objects to train an end-to-end behavior recognition model. The device can be implemented by software and / or hardware and integrated into a computer terminal device with application development capabilities.

[0071] like Figure 4 As shown, the behavior recognition model training device includes: a data acquisition module 310, a model training module 320 and a model determination module 330.

[0072] Among them, the data acquisition module 310 is used to obtain the behavior observation video of the target object, and divide the behavior observation video into multiple video segments at preset time intervals; the model training module 320 is used to input each video segment marked with behavior category as a model training sample into the initial behavior recognition model to train the initial behavior recognition model; the model determination module 330 is used to obtain the target behavior recognition model when the output result of the initial behavior recognition model meets the preset classification evaluation index.

[0073] The technical solution of this embodiment obtains a behavior observation video of the target object, and divides the behavior observation video at preset time intervals to obtain multiple video clips; each video clip marked with a behavior category is used as a model training sample, input into the initial behavior recognition model, and the initial behavior recognition model is trained; when the result of the loss function of the behavior recognition model meets the preset conditions, the target behavior recognition model is obtained. The technical solution of the embodiment of the present invention solves the problem of high time cost of training behavior recognition models based on machine recognition in the prior art. It does not need to annotate the collected video data frame by frame, and can also train an end-to-end behavior recognition model, analyze the input behavior video data, and obtain the corresponding behavior recognition results, thereby reducing the time cost of model training and optimizing the model training process.

[0074] Optionally, the model training module 320 is specifically used for:

[0075] Using a preset feature extraction model to extract features from each video frame in the video clip, and performing maximum pooling processing on the features of each video frame respectively;

[0076] Construct features to be learned based on the features processed by the maximum pooling;

[0077] Four different convolution kernels are respectively used to extract the features to be learned, and behavior recognition results are output based on the extracted features, and parameters of the initial behavior recognition model are adjusted based on the behavior recognition results.

[0078] Optionally, the four different convolution kernels include three temporal convolution kernels and one spatial convolution kernel.

[0079] Optionally, the model training module 320 may also be used to:

[0080] According to the preset step size, feature sampling is performed on each feature after pooling;

[0081] The sampled and pooled features are combined to obtain the features to be learned.

[0082] Optionally, the behavior recognition model training device also includes a data augmentation module, which is used to horizontally rotate, vertically flip and / or horizontally flip the video clips before inputting each video clip marked with behavior categories as model training samples into the initial behavior recognition model to amplify the model training data.

[0083] The behavior recognition model training device provided in the embodiment of the present invention can execute the behavior recognition model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0084] Embodiment 4

[0085] Figure 5 This is a structural diagram of a behavior recognition device provided in Example 4 of the present invention. This embodiment and the behavior recognition model training device in the above-mentioned embodiment belong to the same inventive concept, and further describes the process of applying the trained behavior recognition model. The device can be implemented by software and / or hardware and integrated into a server device with application development function.

[0086] like Figure 5 As shown, the behavior recognition device includes: a to-be-recognized data acquisition module 410 and a behavior recognition module 420 .

[0087] Among them, the data acquisition module 410 for identifying is used to obtain the behavior video to be analyzed of the target object; the behavior recognition module 420 is used to input the behavior video to be analyzed into the behavior recognition model trained by the behavior recognition model training method described in any embodiment to obtain the behavior recognition result of the target object.

[0088] The technical solution of this embodiment can obtain the corresponding target object behavior analysis and recognition results by inputting the collected target object behavior video to be analyzed into the behavior recognition model pre-trained by the model training method of the above embodiment. The technical solution of the embodiment of the present invention solves the problem of efficiently identifying the target object's behavior and determining the behavior analysis results, thereby improving the accuracy of the output results of the existing behavior analysis model.

[0089] The behavior recognition device provided in the embodiment of the present invention can execute the behavior recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0090] Embodiment 5

[0091] Figure 6 This is a schematic diagram of the structure of a behavior recognition system provided in Embodiment 5 of the present invention. This embodiment and the behavior recognition method in the above embodiment belong to the same inventive concept, and further describes the device for collecting observation videos of experimental subjects, and the specific process of collecting videos and analyzing the behavior of experimental subjects based on the videos. The system can be configured in a server device with application development function.

[0092] like Figure 6 As shown, the behavior recognition system includes the following structure: a plurality of stereoscopic experimental open fields with the same volume and adjacent in space, a plurality of video information acquisition devices and a video data processing subsystem.

[0093] in, Figure 6What is shown in the figure is a top view of the experimental open field. Each grid represents an experimental open field space. The number of grids in 6 is only an example and does not limit the number of experimental open fields in actual applications. The experimental open field is a three-dimensional spatial structure used to accommodate the observed experimental subjects. For example, in an experiment to observe the mating behavior of mice, two mice are accommodated in each open field space. Setting up multiple experimental open fields can collect video information of multiple groups of experimental subjects at the same time, thereby improving the efficiency of data collection.

[0094] The video information acquisition device is fixed at a preset position corresponding to the stereoscopic experimental open field, and is used to synchronously collect the behavioral video data of the experimental subjects in each experimental open field within a preset time period. The video information acquisition device can be a camera or other equipment, and the preset position is a position where the activities of the experimental subjects in the experimental open field can be clearly captured. The shooting field of view (FOV) of each video information acquisition device is different, and the shooting field of view of multiple cameras can cover the area where the experimental open field is located as a whole, and the number of video information acquisition devices can be adjusted according to this condition.

[0095] The behavior video data captured by each video information acquisition device will be transmitted to the video data processing subsystem. The video data processing subsystem segments each frame of the behavior video data according to the positional relationship of each experimental field to obtain behavior video data matching the experimental object in each experimental field; and based on each video data obtained after segmentation, the preset behavior of the experimental object is identified or a behavior recognition model is trained. The specific behavior recognition method and the training of the behavior recognition model can refer to the specific execution process in the aforementioned embodiment.

[0096] In a specific example, a behavior recognition system such as Figure 7 As shown. Figure 7 In the experiment, the open field is a three-dimensional space structure of a nine-square grid, and each grid is a space with a length, width, and height of 46, 46, and 50 cm respectively. The outer partition of the nine-square grid is 6 mm thick, which is strong enough, and the inner partition is 3 mm thick, which is transparent enough and has low reflection. The video information acquisition equipment uses a single-view industrial camera. For the collected mouse behavior video, each frame image in the video can be cut into nine parts according to the size ratio of the nine-square grid. Then the cutting results of the same position of each frame image are connected together to obtain nine pieces of behavior video data. Finally, the obtained behavior video data is applied to the data analysis of behavior types or the training of behavior analysis models.

[0097] The technical solution of this embodiment constructs a behavior recognition system through multiple stereoscopic experimental open fields of the same volume and adjacent in space, multiple video information acquisition devices and video data processing subsystems, which can simultaneously collect behavior video data of multiple groups of experimental subjects, thereby improving the efficiency of experimental data acquisition. Moreover, by performing model training or behavior category recognition through the video data processing subsystem, the problem of high time cost of training behavior recognition models based on machine recognition in the prior art can be solved. Without the need to annotate the collected video data frame by frame, an end-to-end behavior recognition model can be trained, and the input behavior video data can be analyzed to obtain the corresponding behavior recognition results, thereby reducing the time cost of model training and optimizing the model training process.

[0098] Embodiment 6

[0099] Figure 8 A schematic diagram of the structure of a computer device provided in Embodiment 6 of the present invention. Figure 8 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 8 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as an intelligent controller and server, a mobile phone and other terminal devices.

[0100] like Figure 8 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0101] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0102] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0103] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0104] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0105] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Figure 8 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0106] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the behavior recognition model training method or behavior recognition method provided in the embodiment of the present invention.

[0107] The behavior recognition model training method includes:

[0108] Obtaining a behavior observation video of a target object, and segmenting the behavior observation video at preset time intervals to obtain a plurality of video segments;

[0109] Inputting each video clip marked with behavior categories as a model training sample into an initial behavior recognition model to train the initial behavior recognition model;

[0110] When the output result of the initial behavior recognition model meets the preset classification evaluation index, the target behavior recognition model is obtained.

[0111] The behavior recognition method includes:

[0112] Obtain the behavior video of the target object to be analyzed;

[0113] The behavior video to be analyzed is input into the behavior recognition model trained by the behavior recognition model training method described in any embodiment to obtain the behavior recognition result of the target object.

[0114] Embodiment 7

[0115] This embodiment seven provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the behavior recognition model training method or behavior recognition method provided in any embodiment of the present invention is implemented.

[0116] The behavior recognition model training method includes:

[0117] Obtaining a behavior observation video of a target object, and segmenting the behavior observation video at preset time intervals to obtain a plurality of video segments;

[0118] Inputting each video clip marked with behavior categories as a model training sample into an initial behavior recognition model to train the initial behavior recognition model;

[0119] When the output result of the initial behavior recognition model meets the preset classification evaluation index, the target behavior recognition model is obtained.

[0120] The behavior recognition method includes:

[0121] Obtain the behavior video of the target object to be analyzed;

[0122] The behavior video to be analyzed is input into the behavior recognition model trained by the behavior recognition model training method described in any embodiment to obtain the behavior recognition result of the target object.

[0123] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0124] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0125] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0126] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0128] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A behavior recognition model training method, characterized in that: The method comprises: Acquire a behavior observation video of a target object, and segment the behavior observation video at preset time intervals to obtain a plurality of video segments, wherein the length of the video segment is determined according to the behavior characteristics of the target object; Inputting each video clip marked with behavior categories as a model training sample into an initial behavior recognition model to train the initial behavior recognition model; When the output result of the initial behavior recognition model meets the preset classification evaluation index, a target behavior recognition model is obtained; The process of the initial behavior recognition model learning and training the model training samples includes: Using a preset feature extraction model to extract features from each video frame in the video clip, and performing maximum pooling processing on the features of each video frame respectively; Construct features to be learned based on the features processed by the maximum pooling; Four groups of different convolution kernels are respectively used to extract the features to be learned, and behavior recognition results are output based on the extracted features, and the parameters of the initial behavior recognition model are adjusted based on the behavior recognition results, wherein the four groups of different convolution kernels include three groups of temporal convolution kernels and one group of spatial convolution kernels.

2. The method according to claim 1, characterized in that The constructing of features to be learned based on the features processed by the maximum pooling includes: According to the preset step size, feature sampling is performed on each feature processed by the maximum pooling; The sampled features processed by maximum pooling are stacked to obtain the features to be learned.

3. The method according to claim 1, characterized in that Before inputting each video clip marked with a behavior category as a model training sample into the initial behavior recognition model, the method further includes: The video clips are respectively subjected to horizontal rotation, vertical flipping and / or horizontal water quality flipping processing to expand the model training data.

4. A behavior recognition method, characterized in that: The method comprises: Obtain the behavior video of the target object to be analyzed; The behavior video to be analyzed is input into a behavior recognition model trained by the behavior recognition model training method described in any one of claims 1 to 3 to obtain a behavior recognition result of the target object.

5. A behavior recognition model training device, characterized in that: The device comprises: A data acquisition module, used to acquire a behavior observation video of a target object, and to segment the behavior observation video at preset time intervals to obtain a plurality of video segments, wherein the length of the video segment is determined according to the behavior characteristics of the target object; A model training module, used to input each video clip marked with a behavior category as a model training sample into an initial behavior recognition model to train the initial behavior recognition model; A model determination module, used to obtain a target behavior recognition model when the output result of the initial behavior recognition model meets the preset classification evaluation index; Wherein, the model training module is configured to: Using a preset feature extraction model to extract features from each video frame in the video clip, and performing maximum pooling processing on the features of each video frame respectively; Construct features to be learned based on the features processed by the maximum pooling; Four groups of different convolution kernels are respectively used to extract the features to be learned, and behavior recognition results are output based on the extracted features, and the parameters of the initial behavior recognition model are adjusted based on the behavior recognition results, wherein the four groups of different convolution kernels include three groups of temporal convolution kernels and one group of spatial convolution kernels.

6. A behavior recognition device, characterized in that: The device comprises: The module for acquiring data to be identified is used to acquire the behavior video of the target object to be analyzed; A behavior recognition module is used to input the behavior video to be analyzed into a behavior recognition model trained by the behavior recognition model training method described in any one of claims 1-3 to obtain a behavior recognition result of the target object.

7. A behavior recognition system, characterized in that: The system comprises: A plurality of stereoscopic experimental open fields of equal volume and adjacent in space, a plurality of video information acquisition devices and a video data processing subsystem; Wherein, the stereoscopic experimental open field is used to accommodate the observed experimental objects; The video information acquisition device is fixed at a preset position corresponding to the stereoscopic experimental open field, and is used to synchronously acquire the behavioral video data of the experimental subject in each of the experimental open fields within a preset time period; The video data processing subsystem is used to identify the preset behavior of the experimental subject based on the behavior video data; The video data processing subsystem is specifically used for: According to the positional relationship of each of the experimental open fields, each frame of the video image in the behavioral video data is segmented to obtain behavioral video data matching the experimental object in each of the experimental open fields; For the behavior video data matching the experimental subjects in each experimental open field, the behavior video data is input into the behavior recognition model training method described in any one of claims 1 to 3 or the behavior recognition method described in claim 4.

8. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the behavior recognition model training method as described in any one of claims 1-3 or the behavior recognition method as described in claim 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the behavior recognition model training method as described in any one of claims 1 to 3 or the behavior recognition method as described in claim 4.