Behavior detection method and device, equipment and storage medium
By using skeleton detection models and mapping relationships to identify behavior types in corporate outlets, the problem of security personnel difficulty in time to detect illegal behavior is solved, and accurate identification and timely response to behavior types in videos is achieved.
Patent Information
- Application Number
- CN202411607982.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-02-21
AI Technical Summary
In corporate outlets with large flow of people, it is difficult for security personnel to detect illegal behavior in a timely manner, resulting in the inability to take corresponding measures in a timely manner.
By obtaining the video data to be detected, input it to the pre-trained skeleton detection model, obtaining the target skeleton data, and determining the behavior type in the preset mapping relationship, and executing the corresponding processing strategy.
Accurate identification of the types of behaviors present in the video, assisting relevant personnel to respond quickly, and improving the timeliness and effectiveness of safety measures.
Smart Images
Figure CN119495123A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a behavior detection method, device, equipment, and storage medium. Background Art
[0002] In some corporate outlets, there is often a large flow of people, and there may be some violent behaviors, which may affect the safety of life and property to a certain extent. It is of great significance to identify these behaviors in real time and take corresponding measures in time.
[0003] In existing implementations, security personnel are generally deployed to conduct real-time inspections.
[0004] However, the above method covers a large area and security personnel cannot detect illegal activities in time, which leads to the inability to take corresponding measures in time. Summary of the invention
[0005] The embodiments of the present application provide a behavior detection method, device, equipment, and storage medium to achieve the effect of timely discovering existing illegal behaviors.
[0006] In a first aspect, an embodiment of the present application provides a behavior detection method, comprising:
[0007] Obtain the video data to be detected;
[0008] Inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, wherein the skeleton detection model is obtained by training an OpenPose model based on at least one video data and the skeleton data corresponding to the person in the at least one video data;
[0009] The target behavior type corresponding to the target skeleton data is determined in a preset first mapping relationship, and the first mapping relationship records the correspondence between different skeleton data and different behavior types.
[0010] In a possible implementation, the step of inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to a person in the video data to be detected includes:
[0011] Determine the key points of the person in the video data to be detected and the confidence of the key points;
[0012] The target skeleton data of the person is determined according to the key points and the confidence level.
[0013] In a possible implementation manner, before determining the key points of the person in the video data to be detected and the confidence levels of the key points, the method further includes:
[0014] The video data to be detected is converted into a target format corresponding to the OpenPose model, wherein the conversion includes: size adjustment and color conversion.
[0015] In a possible implementation, the method further includes:
[0016] When the actual skeleton data corresponding to the video data to be detected is consistent with the target skeleton data, obtaining a type of annotation behavior based on the user annotating the video data to be detected;
[0017] The first mapping relationship is updated according to the labeled behavior type and the target skeleton data.
[0018] In a possible implementation, the method further includes:
[0019] When the actual skeleton data corresponding to the video data to be detected is inconsistent with the target skeleton data, the skeleton detection model is updated according to the actual skeleton data.
[0020] In a possible implementation, the method further includes:
[0021] Determining a target processing strategy corresponding to the target behavior type in a preset second mapping relationship, wherein the mapping relationship records a correspondence between at least one behavior type and at least one processing strategy;
[0022] The target processing strategy is executed.
[0023] In a possible implementation manner, before inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to a person in the video data to be detected, the method further includes:
[0024] Acquire at least one video data and skeleton data corresponding to a person in the at least one video data from an open source space;
[0025] Based on the at least one video data and the skeleton data corresponding to the person in the at least one video data, the OpenPose model is trained to obtain the skeleton detection model.
[0026] In a second aspect, an embodiment of the present application provides a behavior detection device, including:
[0027] An acquisition module, used to acquire video data to be detected;
[0028] A processing module, used for inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, wherein the skeleton detection model is obtained by training an OpenPose model based on at least one video data and the skeleton data corresponding to the person in the at least one video data;
[0029] The determination module is used to determine the target behavior type corresponding to the target skeleton data in a preset first mapping relationship, wherein the first mapping relationship records the correspondence between different skeleton data and different behavior types.
[0030] In a possible implementation manner, the processing module is specifically configured to:
[0031] Determine the key points of the person in the video data to be detected and the confidence of the key points;
[0032] The target skeleton data of the person is determined according to the key points and the confidence level.
[0033] In a possible implementation manner, before determining the key points of the person in the video data to be detected and the confidence levels of the key points, the processing module is further configured to:
[0034] The video data to be detected is converted into a target format corresponding to the OpenPose model, wherein the conversion includes: size adjustment and color conversion.
[0035] In a possible implementation manner, the processing module is further used to:
[0036] When the actual skeleton data corresponding to the video data to be detected is consistent with the target skeleton data, obtaining a type of annotation behavior based on the user annotating the video data to be detected;
[0037] The first mapping relationship is updated according to the labeled behavior type and the target skeleton data.
[0038] In a possible implementation manner, the processing module is further used to:
[0039] When the actual skeleton data corresponding to the video data to be detected is inconsistent with the target skeleton data, the skeleton detection model is updated according to the actual skeleton data.
[0040] In a possible implementation manner, the processing module is further used to:
[0041] Determining a target processing strategy corresponding to the target behavior type in a preset second mapping relationship, wherein the mapping relationship records a correspondence between at least one behavior type and at least one processing strategy;
[0042] The target processing strategy is executed.
[0043] In a possible implementation manner, before inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to a person in the video data to be detected, the processing module is further used to:
[0044] Acquire at least one video data and skeleton data corresponding to a person in the at least one video data from an open source space;
[0045] Based on the at least one video data and the skeleton data corresponding to the person in the at least one video data, the OpenPose model is trained to obtain the skeleton detection model.
[0046] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;
[0047] The memory stores computer-executable instructions;
[0048] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.
[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.
[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.
[0051] The behavior detection method, device, equipment, and storage medium provided in the embodiments of the present application. It can be used in the field of artificial intelligence. The method includes: obtaining video data to be detected; inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, and the skeleton detection model is based on at least one video data and the skeleton data corresponding to the person in at least one video data, and the OpenPose model is trained; the target behavior type corresponding to the target skeleton data is determined in a preset first mapping relationship, and the correspondence between different skeleton data and different behavior types is recorded in the first mapping relationship. The method is used to accurately identify the behavior type existing in the video to assist relevant personnel to respond quickly. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0053] Figure 1 Schematic diagram of the behavior detection method provided in this application Figure 1 ;
[0054] Figure 2 Schematic diagram of the behavior detection method provided in this application Figure 2 ;
[0055] Figure 3 Schematic diagram of the behavior detection method provided in this application Figure 3 ;
[0056] Figure 4 A schematic diagram of the structure of a behavior detection device provided in this application;
[0057] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.
[0058] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0059] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0060] The technical background involved in this application is described as follows:
[0061] In some corporate outlets, there is often a large flow of people, and there may be some violent behaviors, which may affect the safety of life and property to a certain extent. It is of great significance to identify these behaviors in real time and take corresponding measures in time.
[0062] In existing implementations, security personnel are generally deployed to conduct real-time inspections.
[0063] However, the above method covers a large area and security personnel cannot detect illegal activities in time, which leads to the inability to take corresponding measures in time.
[0064] Some existing human posture estimation methods can provide some ideas for determining violent behavior of people. For example, pedestrian detection is performed first, then each person is segmented, and finally, based on each independent individual, the respective limbs (i.e., key points of the human body) are found.
[0065] This approach also has the following problems:
[0066] 1. Relying on the results of the first pedestrian detector, if the person cannot be found, it is impossible to find information such as the key points of the human body.
[0067] 2. The calculation time of the above process is positively correlated with the number of people. The more people there are, the more time it takes.
[0068] In response to the technical problems existing in the prior art, the present application provides a method for detecting behavior, which includes: obtaining video data to be detected; inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, the skeleton detection model is obtained by training the OpenPose model based on at least one video data and the skeleton data corresponding to the person in at least one video data; determining the target behavior type corresponding to the target skeleton data in a preset first mapping relationship, and the first mapping relationship records the correspondence between different skeleton data and different behavior types. This method is used to accurately identify the behavior type existing in the video to assist relevant personnel in responding quickly.
[0069] It should be noted that the behavior detection method and behavior detection device of the present application can be used in the field of artificial intelligence, and can also be used in any field except the field of artificial intelligence. The application field of the behavior detection method and behavior detection device of the present application is not limited.
[0070] Figure 1 Schematic diagram of the behavior detection method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:
[0071] Step 11: Obtain the video data to be detected;
[0072] In this step, the video data to be detected may be video data acquired in real time by camera units in many public places, streets, shopping malls, business halls, and the like.
[0073] Furthermore, the electronic device can obtain the video data to be detected from the camera unit in real time by configuring the streaming media protocol.
[0074] Optionally, after obtaining the video data to be detected, the video data to be detected may be preprocessed to increase the accuracy of subsequent processing, such as:
[0075] Decompose the video data to be detected into continuous image frames, such as extracting 30 frames per second; remove the noise in the video frames through filters (such as Gaussian filtering, median filtering, etc.); enhance the contrast and brightness of the video, or use edge detection and other technologies to improve the target recognition rate in subsequent detection.
[0076] Step 12: Input the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected;
[0077] The skeleton detection model is obtained by training the OpenPose model based on at least one video data and skeleton data corresponding to a person in at least one video data;
[0078] In this step, each frame of the video data to be detected is input into the skeleton detection model for inference, and the skeleton data corresponding to the person in the video data to be detected, that is, the target skeleton data, can be obtained.
[0079] In a possible implementation, the target skeleton data may be the coordinates of each key point of the person.
[0080] Optionally, the training for the skeleton detection model can be:
[0081] Step 1: Obtain at least one video data and skeleton data corresponding to a person in at least one video data from an open source space;
[0082] In this implementation, the open source space may be: downloading at least one video data from a public website. In some implementations, the annotated data for annotating the video data may also be downloaded, that is, the skeleton data corresponding to the person in the video data.
[0083] The video data may also be labeled using a preset algorithm or based on human effort to obtain skeleton data corresponding to the person in the video data.
[0084] Step 2: Based on at least one video data and skeleton data corresponding to a person in at least one video data, the OpenPose model is trained to obtain a skeleton detection model.
[0085] In this step, based on at least one video data and skeleton data corresponding to a person in at least one video data, a training set for training the OpenPose model and a validation set for validation can be constructed, thereby training the OpenPose model and obtaining a skeleton detection model.
[0086] Among them, OpenPose is an open source library based on convolutional neural networks and supervised learning and written in the Caffe framework. It can track people's facial expressions, torsos, limbs and even fingers. It is suitable not only for single people but also for multiple people, and has good robustness.
[0087] In other possible implementations, at least one time series of video data can be added to detect the position changes of key points of the human body within a certain time range to train the skeleton detection model, so as to more accurately detect the posture, estimate the posture of the target at future moments, and perform more abstract human behavior analysis.
[0088] Step 13: Determine the target behavior type corresponding to the target skeleton data in the preset first mapping relationship;
[0089] The first mapping relationship records the correspondence between different skeleton data and different behavior types.
[0090] In this step, different skeleton data have behavior types that express people in reality and can be acquired and classified in advance. After obtaining the target skeleton data output by the skeleton detection model, the behavior type corresponding to the target skeleton data can be obtained based on the target skeleton data, which is recorded as the target behavior type.
[0091] Furthermore, the following may be performed: when the actual skeleton data corresponding to the video data to be detected is inconsistent with the target skeleton data, the skeleton detection model is updated according to the actual skeleton data.
[0092] In this implementation, the video data to be detected can be manually annotated to determine whether the target skeleton data determined by the skeleton detection model is consistent with the annotated actual skeleton data, so as to verify whether the skeleton detection model is accurate.
[0093] When it is determined that the actual skeleton data is inconsistent with the target skeleton data, the skeleton detection model can be trained based on the actual skeleton data to update the parameters in the skeleton detection model to achieve the purpose of updating the skeleton detection model.
[0094] Furthermore, the following may be performed: determining a target processing strategy corresponding to a target behavior type in a preset second mapping relationship, wherein the mapping relationship records a corresponding relationship between at least one behavior type and at least one processing strategy; and executing the target processing strategy.
[0095] In this implementation, different behavior types correspond to different processing strategies.
[0096] For example:
[0097] 1. When the target behavior type is determined to be a violent incident, the corresponding target handling strategy can be: sending reminders to security personnel and calling the police at the same time, which helps to control the violent incident in the first time and maintain the order and environmental safety of the business outlets;
[0098] 2. When the target behavior type is determined to be a quarrel, the corresponding target processing strategy may be: sending a reminder to the security personnel.
[0099] The behavior detection method provided by the present application is applied to the field of artificial intelligence technology. The method includes: obtaining video data to be detected; inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, and the skeleton detection model is obtained by training the OpenPose model based on at least one video data and skeleton data corresponding to the person in at least one video data; determining the target behavior type corresponding to the target skeleton data in a preset first mapping relationship, and the first mapping relationship records the correspondence between different skeleton data and different behavior types. The method is used to accurately identify the behavior type existing in the video to assist relevant personnel to respond quickly.
[0100] Based on the above embodiments, Figure 2 Schematic diagram of the behavior detection method provided in this application Figure 2 ,like Figure 2 As shown, the above step 12 may include the following steps:
[0101] Step 21: determine the key points of the person in the video data to be detected and the confidence of the key points;
[0102] In this step, the number of key points of a person can be determined based on the skeleton detection model. The key points contain two-dimensional coordinate data. The confidence of the key point is used to represent the confidence value of the key point, and the range is usually [0,1]. The closer to 1, the more confident the key point is detected.
[0103] Optionally, before step 21, the following is further performed: converting the video data to be detected into a target format corresponding to the OpenPose model, where the conversion includes: size adjustment and color conversion.
[0104] In this implementation, the video data to be detected is converted into the target format corresponding to the OpenPose model, so that the skeleton detection model can accurately analyze the video data to be detected and obtain more accurate target skeleton data of the person.
[0105] Specifically, the Openpose algorithm uses the relationship between parts (Part Affinity Fields, PAF) and partial area affinity. It is responsible for encoding the 2D vector of the position and direction of the limbs in the image domain. At the same time, the confidence of each key point is marked with the part detection confidence map (Part Detection Confidence Maps, CMP) (commonly known as the "heat map"). Through two branches, the key point positions and the relationship between them are jointly learned. At the same time, these bottom-up detection and association methods are inferred, and the greedy analysis algorithm (English: Greedy parsing Algorithm) is used to sufficiently encode the global context and obtain high-quality results, while consuming only a small part of the computational cost. In parallel, it basically reaches real time, and the time consumption is not strongly related to the number of people in the picture.
[0106] Step 22: Determine the target skeleton data of the person based on the key points and confidence levels.
[0107] In this step, according to the confidence corresponding to the key points, the key points with confidence lower than the preset threshold can be deleted to obtain the trustworthy key points, and then the target skeleton data of the person is determined based on the two-dimensional coordinates of the key points.
[0108] The behavior detection method provided by the present application includes: determining the key points of a person in the video data to be detected and the confidence of the key points, and determining the target skeleton data of the person based on the key points and the confidence. In this technical solution, the key points of the person in the video data to be detected and the confidence of the key points are gradually obtained using a skeleton detection model to determine the target skeleton data of the person, which provides a basis for the subsequent derivation of the behavior type.
[0109] Based on the above embodiments, Figure 3 Schematic diagram of the behavior detection method provided in this application Figure 3 ,like Figure 3 As shown, the method may also include the following steps:
[0110] Step 31: when the actual skeleton data corresponding to the video data to be detected is consistent with the target skeleton data, obtaining a type of annotation behavior based on the user annotating the video data to be detected;
[0111] In this step, after the actual skeleton data and the target skeleton data output by the skeleton detection model are obtained, when it is determined that the actual skeleton data is consistent with the target skeleton data, the video data to be detected can be labeled to obtain the labeled behavior type corresponding to the video data to be detected.
[0112] Furthermore, when the marked behavior type is inconsistent with the target behavior type determined above, the following steps may be performed to increase the accuracy of subsequent behavior type determination.
[0113] Step 32: Update the first mapping relationship according to the labeled behavior type and the target skeleton data.
[0114] In this step, any of the following implementations can exist:
[0115] 1. A correspondence relationship between the target skeleton data and the annotated behavior type may be set, updated to the first mapping relationship, and the correspondence relationship between the target skeleton data and the target behavior type may be deleted;
[0116] 2. In the first mapping relationship, any behavior type corresponds to different skeleton data, that is, one behavior type corresponds to a skeleton data range. At this time, the corresponding skeleton data range in the annotated behavior type can be modified based on the target skeleton data.
[0117] The behavior detection method provided by the present application includes obtaining the annotated behavior type based on the user's annotation of the video data to be detected when the actual skeleton data corresponding to the video data to be detected is consistent with the target skeleton data, and updating the first mapping relationship according to the annotated behavior type and the target skeleton data. The technical solution can correct the first mapping relationship in real time to increase the accuracy of subsequent judgments on the video data to be detected.
[0118] Based on the above embodiments, the following are device embodiments involved in the embodiments of the present application:
[0119] Figure 4 A schematic diagram of the structure of the behavior detection device provided in this application, such as Figure 4 As shown, the device comprises:
[0120] An acquisition module 41 is used to acquire video data to be detected;
[0121] A processing module 42 is used to input the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, where the skeleton detection model is obtained by training the OpenPose model based on at least one video data and the skeleton data corresponding to the person in at least one video data;
[0122] The determination module 43 is used to determine the target behavior type corresponding to the target skeleton data in a preset first mapping relationship, and the first mapping relationship records the correspondence between different skeleton data and different behavior types.
[0123] In a possible implementation, the processing module 42 is specifically configured to:
[0124] Determine the key points of the person in the video data to be detected and the confidence of the key points;
[0125] Based on the key points and confidence levels, the target skeleton data of the person is determined.
[0126] In a possible implementation manner, before determining the key points of the person in the video data to be detected and the confidence of the key points, the processing module 42 is further used to:
[0127] Convert the video data to be detected into the target format corresponding to the OpenPose model. The conversion includes: size adjustment and color conversion.
[0128] In a possible implementation, the processing module 42 is further configured to:
[0129] When the actual skeleton data corresponding to the video data to be detected is consistent with the target skeleton data, obtaining a type of annotation behavior based on the user annotating the video data to be detected;
[0130] The first mapping relationship is updated according to the labeled behavior type and the target skeleton data.
[0131] In a possible implementation, the processing module 42 is further configured to:
[0132] When the actual skeleton data corresponding to the video data to be detected is inconsistent with the target skeleton data, the skeleton detection model is updated according to the actual skeleton data.
[0133] In a possible implementation, the processing module 42 is further configured to:
[0134] Determining a target processing strategy corresponding to the target behavior type in a preset second mapping relationship, wherein the mapping relationship records a correspondence between at least one behavior type and at least one processing strategy;
[0135] Execute the target processing strategy.
[0136] In a possible implementation, before inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to a person in the video data to be detected, the processing module 42 is further used to:
[0137] Acquire at least one video data and skeleton data corresponding to a person in at least one video data from an open source space;
[0138] Based on at least one video data and skeleton data corresponding to a person in at least one video data, an OpenPose model is trained to obtain a skeleton detection model.
[0139] The behavior detection device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.
[0140] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device provided in this embodiment includes: at least one processor 51 and a memory 52. Optionally, the device also includes a communication component 53. The processor 51, the memory 52 and the communication component 53 are connected via a bus 54.
[0141] In a specific implementation process, at least one processor 51 executes the computer-executable instructions stored in the memory 52, so that at least one processor 51 executes the above method.
[0142] The specific implementation process of the processor 51 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0143] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0144] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.
[0145] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0146] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0147] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0148] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0149] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0150] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0151] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0152] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0153] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
[0154] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0155] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application, are not limited to the precise structures described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A behavior detection method, characterized in that: The method comprises: Obtain the video data to be detected; Inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, wherein the skeleton detection model is obtained by training an OpenPose model based on at least one video data and the skeleton data corresponding to the person in the at least one video data; The target behavior type corresponding to the target skeleton data is determined in a preset first mapping relationship, and the first mapping relationship records the correspondence between different skeleton data and different behavior types.
2. The method according to claim 1, characterized in that The step of inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to a person in the video data to be detected includes: Determine the key points of the person in the video data to be detected and the confidence of the key points; The target skeleton data of the person is determined according to the key points and the confidence level.
3. The method according to claim 2, characterized in that Before determining the key points of the person in the video data to be detected and the confidence levels of the key points, the method further includes: The video data to be detected is converted into a target format corresponding to the OpenPose model, wherein the conversion includes: size adjustment and color conversion.
4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When the actual skeleton data corresponding to the video data to be detected is consistent with the target skeleton data, obtaining a type of annotation behavior based on the user annotating the video data to be detected; The first mapping relationship is updated according to the labeled behavior type and the target skeleton data.
5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the actual skeleton data corresponding to the video data to be detected is inconsistent with the target skeleton data, the skeleton detection model is updated according to the actual skeleton data.
6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Determining a target processing strategy corresponding to the target behavior type in a preset second mapping relationship, wherein the mapping relationship records a correspondence between at least one behavior type and at least one processing strategy; The target processing strategy is executed.
7. The method according to any one of claims 1 to 3, characterized in that Before inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to a person in the video data to be detected, the method further includes: Acquire at least one video data and skeleton data corresponding to a person in the at least one video data from an open source space; Based on the at least one video data and the skeleton data corresponding to the person in the at least one video data, the OpenPose model is trained to obtain the skeleton detection model.
8. A behavior detection device, characterized in that: The device comprises: An acquisition module, used to acquire video data to be detected; A processing module, used for inputting the video data to be detected into a pre-trained skeleton detection model to obtain target skeleton data corresponding to the person in the video data to be detected, wherein the skeleton detection model is obtained by training an OpenPose model based on at least one video data and the skeleton data corresponding to the person in the at least one video data; The determination module is used to determine the target behavior type corresponding to the target skeleton data in a preset first mapping relationship, wherein the first mapping relationship records the correspondence between different skeleton data and different behavior types.
9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.