Behavior feature recognition method and device, server and storage medium

Abstract

The invention discloses a behavior feature recognition method and device, a server and a storage medium, and the method comprises the steps: obtaining a reference image sequence approaching a target object, inputting the reference image sequence into a trained semantic segmentation neural network, and obtaining a recognition result of a key part of a human body and a corresponding position; creating a human body key part topological graph sequence according to the recognition result and the corresponding position; obtaining the position change of each part according to the topological graph sequence of the key parts of the human body, and determining associated parts according to the position change degree; setting a weight value between topological graph nodes of the associated parts according to the mapping value of the change degree of the associated parts; and inputting the topological graph sequence of the key parts of the human body into a trained graph neural network model to obtain behavior characteristics output by the graph neural network. The accuracy of behavior feature recognition can be improved, meanwhile, prediction can be conducted in advance, and early warning of abnormal operation is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a behavior feature recognition method, device, server and storage medium. Background Art

[0002] With the implementation of artificial intelligence technology, intelligent management of the coal mining industry has also become a trend. Based on video images, by deploying AI algorithm models, intelligent recognition and analysis of underground coal mine monitoring scenes can be achieved, and real-time monitoring and early warning of abnormal events, illegal operations, safety hazards, etc. in the coal mine safety production process can be achieved.

[0003] In the prior art, a human body image is usually identified through a neural network model, and the operation performed by the human body is judged based on the movement relationship between the various parts identified in the human body image, and then it is judged whether it is an illegal or normal operation. However, this method is only based on some basic operation action rules. Not only is the accuracy of the judgment poor, but also, because it can only give a judgment result after the human body makes an action, it has a certain lag and cannot achieve the purpose of preventing and stopping illegal operations. Summary of the invention

[0004] The embodiments of the present invention provide a behavior feature recognition method, device, server and storage medium to solve the technical problems in the prior art that the behavior feature recognition has low accuracy and certain hysteresis.

[0005] In a first aspect, an embodiment of the present invention provides a behavior feature recognition method, comprising:

[0006] Acquire a reference image sequence close to the target object, input the reference image sequence into the trained semantic segmentation neural network, and obtain recognition results and corresponding positions of key parts of the human body;

[0007] Creating a topological map sequence of key parts of the human body according to the recognition results and corresponding positions;

[0008] Acquire the position change of each part according to the topological map sequence of the key parts of the human body, and determine the associated parts according to the degree of the position change;

[0009] Setting weight values ​​between nodes in the topological graph of the associated parts according to the mapping value of the degree of change of the associated parts;

[0010] The topological map sequence of key parts of the human body is input into the trained graph neural network model to obtain the behavioral characteristics of the graph neural network output.

[0011] In a second aspect, an embodiment of the present invention further provides a behavior feature recognition device, including:

[0012] An acquisition module is used to acquire a reference image sequence close to the target object, and input the reference image sequence into the trained semantic segmentation neural network to obtain the recognition results and corresponding positions of key parts of the human body;

[0013] A creation module, used for creating a topological map sequence of key parts of the human body according to the recognition results and corresponding positions;

[0014] An associated part determination module is used to obtain the position change of each part according to the topological map sequence of the key parts of the human body, and determine the associated parts according to the degree of position change;

[0015] A setting module, used for setting weight values ​​between nodes of the topological map of the associated parts according to the mapping value of the degree of change of the associated parts;

[0016] The input module is used to input the topological map sequence of key parts of the human body into the trained graph neural network model to obtain the behavioral characteristics of the graph neural network output.

[0017] In a third aspect, an embodiment of the present invention provides a server, wherein the server includes:

[0018] one or more processors;

[0019] a storage device for storing one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the behavior feature recognition methods provided in the above embodiments.

[0021] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the behavior feature recognition method provided in the above embodiment.

[0022] The behavior feature recognition method, device, server and storage medium provided by the embodiment of the present invention obtain a reference image sequence close to the target object, input the reference image sequence into a trained semantic segmentation neural network, obtain the recognition results and corresponding positions of key parts of the human body; create a topological map sequence of key parts of the human body according to the recognition results and corresponding positions; obtain the position change of each part according to the topological map sequence of key parts of the human body, and determine the associated parts according to the degree of position change; set the weight value between the nodes of the topological map of the associated parts according to the mapping value of the degree of change of the associated parts; input the topological map sequence of key parts of the human body into a trained graph neural network model, and obtain the behavior features output by the graph neural network. By generating a topological map sequence from the positions of the human body parts recognized by the semantic segmentation neural network, and setting weights using the correlation between the moving parts of the human body, the trained graph neural network is used to accurately obtain the recognition of the behavior features, and then the prediction of the operation is realized, which can not only improve the accuracy of the behavior feature recognition, but also make predictions in advance, and realize early warning of abnormal operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0024] Figure 1 is a flow chart of a behavior feature recognition method provided in Embodiment 1 of the present invention;

[0025] Figure 2 is a flow chart of a behavior feature recognition method provided in Embodiment 2 of the present invention;

[0026] Figure 3 is a schematic diagram of the structure of a behavior feature recognition device provided in Embodiment 3 of the present invention;

[0027] Figure 4 It is a schematic diagram of the structure of the server provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0029] Embodiment 1

[0030] Figure 1: is a flow chart of a behavior feature recognition method provided in the first embodiment of the present invention. This embodiment is applicable to the situation of identifying behavior features underground. The method can be executed by a behavior feature recognition device, and specifically includes the following steps:

[0031] Step 110, obtaining a reference image sequence close to the target object, inputting the reference image sequence into the trained semantic segmentation neural network, and obtaining the recognition results and corresponding positions of key parts of the human body.

[0032] In this embodiment, the target object may be various equipment that can be operated or controlled underground, or a work surface, etc. Since the walking or other actions of underground personnel are irrelevant to the operation, there is no possibility of illegal operation. Therefore, only images close to the target object are considered, and the images can be collected by an image acquisition device set in the underground, and sorted according to the acquisition time to generate a reference image sequence.

[0033] Furthermore, since there may be multiple cameras in the well, and they are set at different positions and angles. Therefore, in this embodiment, the acquisition of an image sequence close to the target object includes: extracting a fixed object in each image in the image sequence; setting a reference coordinate system based on the fixed object in the image; adjusting the human body proportions in each image in the image sequence according to the reference coordinate system; and generating a reference image proportion based on the adjusted human body proportions. Exemplarily, the fixed object may be a target object, and the target object has fixed length, width, and height parameters. Based on this characteristic, the center of the fixed object can be used as the origin of the three-dimensional coordinates, and the reference coordinate system can be set. The human body proportions in the image are adjusted according to the reference coordinate system, so that the characters in the images captured by different image acquisition devices can be unified.

[0034] In this embodiment, the main features of the semantic segmentation neural network include: the core of the semantic segmentation neural network is to assign a semantic category to each pixel in the image to achieve pixel-level classification. The semantic segmentation neural network can accurately identify each moving part of the human body and the corresponding positional relationship.

[0035] Step 120: Create a topological map sequence of key parts of the human body according to the recognition results and the corresponding positions.

[0036] In this embodiment, key parts can be respectively intercepted from the original image according to the recognition results and the corresponding positions, and are sorted according to time to form a topological map sequence of key parts of the human body. The topological map is used to simplify the parts into nodes, and the motion characteristics are expressed in a mathematical way.

[0037] Step 130, obtaining the position change of each part according to the topological map sequence of the key parts of the human body, and determining the associated parts according to the degree of the position change.

[0038] The sequence of topological images of the key parts of the human body in the above steps can determine the change of each part, and the degree of change can be used to determine the active part. At the same time, according to the characteristics of human body movement, the parts that move in a direction related to the key parts can be found from the image as associated parts.

[0039] Step 140: setting weight values ​​between nodes in the topological graph of the associated parts according to the mapping value of the degree of change of the associated parts.

[0040] Exemplarily, since there may be multiple associated parts for a key part, and the movement degree of the associated parts is different when performing a special action, it is necessary to set the associated parts according to the degree of change of the associated parts. Optionally, one way is to set the basic action vector of each associated part according to various possible operations, including: the minimum change amplitude vector of the action amplitude and angle. Subtract the basic action vector from the vector of the currently acquired associated part, and calculate the ratio of the difference to the maximum amplitude vector, obtain the degree of change, and use the mapping value corresponding to the preset degree of change as the weight value between the nodes of the associated part topology map.

[0041] Step 150, input the topological map sequence of key parts of the human body into the trained graph neural network model to obtain the behavioral characteristics of the graph neural network output.

[0042] The topological map sequence of the key parts of the human body and the corresponding weight values ​​are input into the trained graph neural network model, and the different ranges of the topological map are determined according to the images of the part movement relationship of different operations by using the characteristics of the graph neural network. The interaction of weights at different layers can reflect the relationship between the instantaneous behavior and position of the operation and the parts of the human body, and the features are interacted to different degrees according to the relationship between the activities of the parts corresponding to the operation, so as to better extract the feature changes of the instantaneous part movement and the corresponding relationship of the operation, so as to further improve the accuracy of operation behavior recognition and prediction.

[0043] This embodiment obtains a reference image sequence close to the target object, inputs the reference image sequence into a trained semantic segmentation neural network, obtains the recognition results and corresponding positions of key parts of the human body; creates a topological map sequence of key parts of the human body according to the recognition results and corresponding positions; obtains the position change of each part according to the topological map sequence of key parts of the human body, and determines the associated parts according to the degree of position change; sets the weight values ​​between the nodes of the topological map of the associated parts according to the mapping value of the degree of change of the associated parts; inputs the topological map sequence of key parts of the human body into a trained graph neural network model, and obtains the behavioral features output by the graph neural network. By generating a topological map sequence from the positions of the human body parts recognized by the semantic segmentation neural network, and setting weights using the correlation between the moving parts of the human body, and using the trained graph neural network to accurately obtain the recognition of behavioral features, and then realizing the prediction of operations, it can not only improve the accuracy of behavioral feature recognition, but also make predictions in advance, and realize early warning of abnormal operations.

[0044] In a preferred implementation of this embodiment, the method may further add the following steps: when images of key human parts are missing in some images in the reference image sequence, simulated images of key human parts are generated based on the front and back images of the reference image sequence; simulated images are generated based on the simulated images of key human parts to replace the original images and fill in the reference image sequence. Due to the limitations of the position and angle of the image acquisition device, some of the acquired images may contain missing images of key human parts. When missing images exist, images acquired by cameras at other angles may be used to form simulated images through spatial position calculation. If images acquired by other cameras still contain missing images of key parts, the front and back images in the sequence that are closest in time and contain images of key human parts can be used to estimate simulated images of key human parts based on their positions through a compromise method. This allows the graph neural network model to output accurate behavioral features.

[0045] Embodiment 2

[0046] Figure 2 It is a flow chart of the behavior feature recognition method provided in the second embodiment of the present invention. This embodiment is optimized on the basis of the above embodiment, and the weight values ​​between the nodes of the topological map of the associated parts are set according to the mapping values ​​of the degree of change of the associated parts. The specific optimization is as follows: based on the maximum motion range of each associated part, a mapping value of the degree of change is set for the current position of each associated part; feature points are selected from the associated parts based on the mapping values; multiple feature sequences are established based on the positions of the feature points; and the weight values ​​between the nodes of the topological map of the associated parts are determined according to a preset motion space aggregation point rule.

[0047] See also Figure 2 , the behavior feature recognition method comprises:

[0048] Step 210, obtaining a reference image sequence close to the target object, inputting the reference image sequence into the trained semantic segmentation neural network, and obtaining recognition results of key parts of the human body and corresponding positions.

[0049] Step 220: Create a topological map sequence of key parts of the human body according to the recognition results and the corresponding positions.

[0050] Step 230, obtaining the position change of each part according to the topological map sequence of the key parts of the human body, and determining the associated parts according to the degree of the position change.

[0051] Step 240 : setting a change degree mapping value for the current position of each associated part based on the maximum motion range of each associated part.

[0052] In this embodiment, the mapping value may be determined corresponding to the normalized value of the motion change of the associated part, that is, the mapping value is the largest based on the maximum motion range, and the corresponding mapping value is determined based on the range from 0 to the maximum motion range.

[0053] Step 250: Select feature points from the associated parts based on the mapping values, and establish multiple feature sequences based on the positions of the feature points.

[0054] The corresponding feature points can be determined according to the mapping values, and the feature points can be representative points in the image that can reflect the motion characteristics. Based on the position of each feature point in the image, a feature sequence of the image is generated. For multiple images, multiple feature sequences are generated.

[0055] Step 260 , performing aggregation processing on the plurality of feature sequences, and determining weight values ​​between nodes of the associated part topology map according to a preset motion space aggregation point rule.

[0056] By using the above method, the feature points are mapped to the multi-dimensional space, and the corresponding weight value is determined according to the distance between the corresponding position in the multi-dimensional space and the position where the preset aggregation point rule is located. By using the above method, the minute features of the part movement are fully considered and the weight value is set based on this.

[0057] Step 270, input the topological map sequence of key parts of the human body into the trained graph neural network model to obtain the behavioral characteristics of the graph neural network output.

[0058] For example, two thresholds may be set to correspond to the sequence and mean square error, respectively. When both are less than the two thresholds, it can be determined that the reconstruction requirements are met. Alternatively, different weight values ​​may be set for the correlations of the two, and the correlations of the two are added using the weight values, and a comprehensive correlation threshold is set to determine whether the reconstruction requirements are met.

[0059] This embodiment sets the weight values ​​between the nodes of the topology map of the associated parts according to the mapping value of the degree of change of the associated parts, and specifically optimizes the following: setting the mapping value of the degree of change for the current position of each associated part based on the maximum motion range of each associated part; selecting feature points from the associated parts based on the mapping value; establishing multiple feature sequences based on the positions of the feature points; and determining the weight values ​​between the nodes of the topology map of the associated parts according to the preset motion space aggregation point rule. The features of the moving parts can be further extracted, and the weight values ​​can be set more scientifically and accurately.

[0060] In a preferred implementation of this embodiment, the method may add the following steps: when a tool is detected in the semantic segmentation neural network, the corresponding motion feature set is determined according to the attributes of the tool; accordingly, the weight values ​​between the nodes of the topological map of the associated parts are determined according to the preset motion space aggregation point rule, which is specifically optimized as follows: the preset motion space aggregation point rule is determined according to the motion feature set; the weight values ​​between the nodes of the topological map of the associated parts are determined according to the preset motion space aggregation point rule. By using the tools identified in the image, the scope of identification and prediction of the behavior features corresponding to the operation can be narrowed, and the amount of calculation can be reduced without affecting the accuracy of the set weight value. Improve the calculation efficiency.

[0061] Embodiment 3

[0062] Figure 3 is a schematic diagram of the structure of the behavior feature recognition device provided by the third embodiment of the present invention, see Figure 3 , the behavior feature recognition device comprises:

[0063] An acquisition module 310 is used to acquire a reference image sequence close to the target object, and input the reference image sequence into the trained semantic segmentation neural network to obtain recognition results and corresponding positions of key parts of the human body;

[0064] A creation module 320, for creating a topological map sequence of key parts of a human body according to the recognition results and corresponding positions;

[0065] The associated part determination module 330 is used to obtain the position change of each part according to the topological map sequence of the key parts of the human body, and determine the associated parts according to the degree of position change;

[0066] A setting module 340, used to set weight values ​​between nodes of the topological map of the associated parts according to the mapping value of the degree of change of the associated parts;

[0067] The input module 350 is used to input the topological map sequence of key parts of the human body into the trained graph neural network model to obtain the behavioral characteristics of the graph neural network output.

[0068] The behavior feature recognition device provided in this embodiment obtains a reference image sequence close to the target object, inputs the reference image sequence into a trained semantic segmentation neural network, obtains the recognition results and corresponding positions of key parts of the human body; creates a topological map sequence of key parts of the human body according to the recognition results and corresponding positions; obtains the position change of each part according to the topological map sequence of key parts of the human body, and determines the associated part according to the degree of position change; sets the weight value between the nodes of the topological map of the associated part according to the mapping value of the degree of change of the associated part; inputs the topological map sequence of key parts of the human body into a trained graph neural network model, and obtains the behavior features output by the graph neural network. By generating a topological map sequence from the positions of the human body parts recognized by the semantic segmentation neural network, and setting weights using the correlation between the moving parts of the human body, the trained graph neural network is used to accurately obtain the recognition of the behavior features, and then the prediction of the operation is realized, which can not only improve the accuracy of the behavior feature recognition, but also make predictions in advance, and realize early warning of abnormal operations.

[0069] Based on the above embodiments, the acquisition module includes:

[0070] an extraction unit, used for extracting a fixed object in each image in the image sequence;

[0071] A coordinate system setting unit, used for setting a reference coordinate system based on a fixed object in the image;

[0072] an adjusting unit, configured to adjust the human body proportions of each image in the image sequence according to the reference coordinate system;

[0073] A generating unit is used to generate a reference image proportion based on the adjusted human body proportion.

[0074] Based on the above embodiments, the setting module includes:

[0075] A setting unit, configured to set a change degree mapping value for a current position of each associated part based on a maximum motion range of each associated part;

[0076] A selection unit, configured to select feature points from the associated parts based on the mapping values;

[0077] An establishing unit, used for establishing a plurality of feature sequences based on the positions of the feature points;

[0078] The determination unit is used to perform aggregation processing on the multiple feature sequences and determine the weight values ​​between the nodes of the associated part topology map according to a preset motion space aggregation point rule.

[0079] Based on the above embodiments, the device further includes:

[0080] A generating module, used for generating a simulated image of a key part of a human body according to the front and back images of the reference image sequence when images of key parts of a human body are missing in some images of the reference image sequence;

[0081] The filling module is used to generate a simulated image based on the simulated image of the key part of the human body to replace the original image and fill it into the reference image sequence.

[0082] Based on the above embodiments, the associated part determination module includes:

[0083] The determination unit is used to determine the associated part based on the movement association and position change degree of the human body muscles.

[0084] Based on the above embodiments, the device further includes:

[0085] A motion feature set determination module, configured to determine a corresponding motion feature set according to the attributes of a tool when a tool is detected in the semantic segmentation neural network;

[0086] Accordingly, the determining unit includes:

[0087] A rule determination subunit, used to determine a preset motion space aggregation point rule according to the motion feature set;

[0088] The weight value determination unit is used to determine the weight values ​​between the nodes of the associated part topology map according to a preset motion space aggregation point rule.

[0089] The behavior feature recognition device provided in the embodiment of the present invention can execute the behavior feature recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0090] Embodiment 4

[0091] Figure 4 A schematic diagram of the structure of a server provided in Embodiment 4 of the present invention. Figure 4 A block diagram of an exemplary server 12 suitable for use in implementing embodiments of the present invention is shown. Figure 4 The server 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0092] like Figure 4 As shown, the server 12 is in the form of a general purpose computing device. The components of the server 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0093] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0094] The server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the server 12, including volatile and non-volatile media, removable and non-removable media.

[0095] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0096] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0097] The server 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the server 12, and / or may communicate with any device that enables the server 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the server 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the server 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the server 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0098] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the server file operation option recommendation method provided in the embodiment of the present invention.

[0099] Embodiment 5

[0100] Embodiment 5 of the present invention further provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to execute any of the behavior feature recognition methods provided in the above embodiments.

[0101] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0102] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0103] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0104] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or device. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0105] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A behavior feature recognition method, characterized in that: include: Acquire a reference image sequence close to the target object, input the reference image sequence into the trained semantic segmentation neural network, and obtain recognition results and corresponding positions of key parts of the human body; Creating a topological map sequence of key parts of the human body according to the recognition results and corresponding positions; Acquire the position change of each part according to the topological map sequence of the key parts of the human body, and determine the associated parts according to the degree of the position change; Setting weight values ​​between nodes in the topological graph of the associated parts according to the mapping value of the degree of change of the associated parts; The topological map sequence of key parts of the human body is input into the trained graph neural network model to obtain the behavioral characteristics of the graph neural network output.

2. The method according to claim 1, characterized in that: The step of acquiring a sequence of images close to the target object comprises: extracting a fixed object in each image in the image sequence; Setting a reference coordinate system based on fixed objects in the image; adjusting the human body proportions of each image in the image sequence according to the reference coordinate system; Generate reference image proportions based on the adjusted human proportions.

3. The method according to claim 1, characterized in that: The step of setting weight values ​​between nodes in the topological graph of the associated parts according to the mapping value of the degree of change of the associated parts includes: Setting a change degree mapping value for the current position of each associated part based on the maximum motion range of each associated part; Selecting feature points from the associated parts based on the mapping values; Establishing a plurality of feature sequences based on the positions of the feature points; Aggregation processing is performed on the multiple feature sequences, and weight values ​​between nodes of the associated part topology map are determined according to a preset motion space aggregation point rule.

4. The method according to claim 1, characterized in that: The method further comprises: When images of key parts of the human body are missing in some images in the reference image sequence, generating simulated images of key parts of the human body according to the front and back images of the reference image sequence; A simulated image is generated based on the simulated image of the key part of the human body to replace the original image and fill in the reference image sequence.

5. The method according to claim 1, characterized in that Determining the associated part according to the position change degree includes: The associated parts are determined based on the movement association and position change degree of human muscles.

6. The method according to claim 3, characterized in that The method further comprises: When a tool is detected in the semantic segmentation neural network, a corresponding motion feature set is determined according to the attributes of the tool; Correspondingly, determining the weight values ​​between the nodes of the associated part topology map according to the preset motion space aggregation point rule includes: Determining a preset motion space aggregation point rule according to the motion feature set; The weight values ​​between the nodes of the topological map of the associated parts are determined according to the preset motion space aggregation point rules.

7. A behavior feature recognition device, characterized in that: include: An acquisition module is used to acquire a reference image sequence close to the target object, and input the reference image sequence into the trained semantic segmentation neural network to obtain the recognition results and corresponding positions of key parts of the human body; A creation module, used for creating a topological map sequence of key parts of the human body according to the recognition results and corresponding positions; An associated part determination module is used to obtain the position change of each part according to the topological map sequence of the key parts of the human body, and determine the associated parts according to the degree of position change; A setting module, used for setting weight values ​​between nodes of the topological map of the associated parts according to the mapping value of the degree of change of the associated parts; The input module is used to input the topological map sequence of key parts of the human body into the trained graph neural network model to obtain the behavioral characteristics of the graph neural network output.

8. A server, characterized in that: The server comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the behavior feature recognition method as described in any one of claims 1-6.

9. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the behavior feature recognition method as described in any one of claims 1-6 when executed by a computer processor.

Images