Behavioral feature recognition methods, devices, servers, and storage media
By combining semantic segmentation neural networks and graph neural networks, a topological map of key parts of the human body is generated and weight values are set, which solves the problems of low accuracy and lag in behavioral feature recognition in existing technologies, and realizes accurate prediction and early warning of downhole operations.
Patent Information
- Application Number
- CN202411389840.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-08
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-10-08
AI Technical Summary
Existing technologies for behavioral feature recognition are not very accurate and have a certain lag, making them unable to effectively prevent and stop illegal operations.
By acquiring a sequence of reference images close to the target object, a trained semantic segmentation neural network is used to identify key parts of the human body and generate a sequence of topological graphs. Weight values of associated parts are set according to positional changes, and the graph neural network model is input to obtain behavioral features.
It improves the accuracy of behavioral feature recognition and enables early warning and prediction of abnormal operations.
Smart Images

Figure CN119229375B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, server, and storage medium for behavioral feature recognition. Background Technology
[0002] With the application of artificial intelligence technology, intelligent management in the coal mining industry has become a trend. Based on video images, AI algorithm models can be deployed to achieve intelligent identification and analysis of underground coal mine monitoring scenes, enabling real-time monitoring and early warning of abnormal events, violations, and safety hazards in the coal mine safety production process.
[0003] In existing technologies, neural network models are typically used to identify human images and determine the actions performed by the human body based on the movement relationships between various parts identified in the image, thereby determining whether the action is illegal or legitimate. However, this method only relies on some basic operational rules for judgment. Not only is the accuracy of the judgment poor, but because it only provides a judgment result after the human body has performed the action, it has a certain lag and cannot achieve the purpose of preventing and stopping illegal operations. Summary of the Invention
[0004] This invention provides a behavioral feature recognition method, device, server, and storage medium to solve the technical problems of low accuracy and certain lag in behavioral feature recognition in the prior art.
[0005] In a first aspect, embodiments of the present invention provide a behavioral feature recognition method, including:
[0006] A reference image sequence close to the target object is obtained, and the reference image sequence is input into a trained semantic segmentation neural network to obtain the recognition results and corresponding positions of key human body parts;
[0007] A sequence of topological maps of key human body parts is created based on the recognition results and corresponding locations.
[0008] The positional changes of each part are obtained based on the topological map sequence of key human body parts, and the associated parts are determined based on the degree of positional changes.
[0009] The weight values between nodes in the topology graph of the associated parts are set according to the mapping value of the degree of change of the associated parts;
[0010] The sequence of topological graphs of key human body parts is input into a trained graph neural network model to obtain the behavioral features output by the graph neural network.
[0011] Secondly, embodiments of the present invention also provide a behavior feature recognition device, comprising:
[0012] The acquisition module is used to acquire a reference image sequence close to the target object, input the reference image sequence into the trained semantic segmentation neural network, and obtain the recognition results and corresponding positions of key human body parts;
[0013] A creation module is used to create a sequence of topological maps of key human body parts based on the recognition results and corresponding locations.
[0014] The associated part determination module is used to obtain the positional change of each part according to the sequence of topological maps of key human body parts, and determine the associated part according to the degree of positional change.
[0015] The setting module is used to set the weight values between nodes in the topology map of related parts based on the mapping value of the degree of change of related parts;
[0016] The input module is used to input the sequence of topological graphs of key human body parts into the trained graph neural network model to obtain the behavioral features output by the graph neural network.
[0017] Thirdly, embodiments of the present invention provide a server, the server comprising:
[0018] One or more processors;
[0019] Storage device for storing one or more programs.
[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the behavioral feature recognition method as described in any of the above embodiments.
[0021] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the behavioral feature recognition method provided in the above embodiments.
[0022] The behavioral feature recognition method, apparatus, server, and storage medium provided in this invention acquire a sequence of reference images approaching a target object, input the sequence of reference images into a trained semantic segmentation neural network to obtain the recognition results and corresponding positions of key human body parts; create a sequence of topological maps of key human body parts based on the recognition results and corresponding positions; obtain the positional changes of each part based on the sequence of topological maps of key human body parts, and determine associated parts based on the degree of positional changes; set weight values between nodes in the topological map of associated parts based on the mapping values of the degree of changes in associated parts; input the sequence of topological maps of key human body parts into a trained graph neural network model to obtain the behavioral features output by the graph neural network. By generating a sequence of topological maps from the positions of human body parts identified by the semantic segmentation neural network, and setting weights based on the correlation between human movement parts, the trained graph neural network accurately obtains the recognition of behavioral features, thereby achieving prediction of operations. This not only improves the accuracy of behavioral feature recognition but also enables early prediction and early warning of abnormal operations. Attached Figure Description
[0023] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0024] Figure 1 This is a flowchart illustrating the behavioral feature recognition method provided in Embodiment 1 of the present invention;
[0025] Figure 2 This is a flowchart illustrating the behavioral feature recognition method provided in Embodiment 2 of the present invention;
[0026] Figure 3 This is a schematic diagram of the behavioral feature recognition device provided in Embodiment 3 of the present invention;
[0027] Figure 4 This is a schematic diagram of the server structure provided in Embodiment 4 of the present invention. Detailed Implementation
[0028] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0029] Example 1
[0030] Figure 1This is a flowchart of the behavioral feature recognition method provided in Embodiment 1 of the present invention. This embodiment is applicable to the identification of behavioral features in underground mining operations. The method can be executed by a behavioral feature recognition device and specifically includes the following steps:
[0031] Step 110: Obtain a reference image sequence close to the target object, and input the reference image sequence into the trained semantic segmentation neural network to obtain the recognition results and corresponding positions of key human body parts.
[0032] In this embodiment, the target object can be various operable or controllable equipment underground, or a working face, etc. Since the movement or other actions of personnel underground are unrelated to the operation, there is no possibility of unauthorized operation. Therefore, only images of proximity to the target object are considered. These images can be acquired by an image acquisition device installed underground and sorted according to the acquisition time to generate a reference image sequence.
[0033] Furthermore, since multiple cameras may be installed underground, positioned at different locations and angles, in this embodiment, acquiring an image sequence close to the target object includes: extracting a fixed object from each image in the image sequence; establishing a reference coordinate system based on the fixed object in the image; adjusting the human body proportions in each image in the image sequence according to the reference coordinate system; and generating a reference image proportion based on the adjusted human body proportions. For example, the fixed object can be the target object, which has fixed length, width, and height parameters. Based on this characteristic, the center of the fixed object can be used as the origin of the three-dimensional coordinate system to establish a reference coordinate system. Adjusting the human body proportions in the image according to the reference coordinate system ensures that the figures in images acquired by different image acquisition devices are consistent.
[0034] In this embodiment, the main features of the semantic segmentation neural network include: the core of the semantic segmentation neural network lies in assigning a semantic category to each pixel in the image, achieving pixel-level classification. Using the semantic segmentation neural network, each moving part of the human body and its corresponding positional relationship can be accurately identified.
[0035] Step 120: Create a sequence of topological maps of key human body parts based on the recognition results and corresponding locations.
[0036] In this embodiment, key parts can be extracted from the original image based on the recognition results and corresponding locations, and then sorted by time to form a sequence of topological maps of key human body parts. The topological map simplifies the parts into nodes, and mathematical methods are used to express motion features.
[0037] Step 130: Obtain the positional changes of each part according to the topological map sequence of key human body parts, and determine the associated parts according to the degree of positional changes.
[0038] The sequence of topological maps of key human body parts obtained through the above steps can determine the changes in each part, and the degree of change can be used to identify the active parts. Furthermore, based on the characteristics of human movement, related parts moving in the same direction can be identified from the images based on the key parts as associated parts.
[0039] Step 140: Set the weight values between nodes in the topology graph of the associated parts according to the mapping value of the degree of change of the associated parts.
[0040] For example, since key parts may have multiple related parts, and the degree of movement of these related parts varies when performing a specific action, it is necessary to define the related parts based on their degree of change. Optionally, one approach is to define a basic motion vector for each related part based on various possible operations, including: motion amplitude and minimum angle change vector. The basic motion vector is subtracted from the currently acquired vector of the related part, and the ratio of the difference to the maximum amplitude vector is calculated to obtain the degree of change. This degree of change is then used as the weight value between nodes in the related part's topology graph according to a preset mapping value.
[0041] Step 150: Input the sequence of topological graphs of key human body parts into the trained graph neural network model to obtain the behavioral features output by the graph neural network.
[0042] The sequence of topological maps of key human body parts and their corresponding weight values are input into a trained graph neural network model. Leveraging the characteristics of graph neural networks, different ranges of the topological map are determined based on the motion relationships of different body parts during various operations. Interacting weights at different layers reveals the correlation between instantaneous actions, locations, and body parts. Furthermore, features are interacted to varying degrees based on the relationships between the corresponding body part activities to better extract the feature changes in instantaneous body part motion and the corresponding operations, thereby further improving the accuracy of action recognition and prediction.
[0043] This embodiment acquires a sequence of reference images close to the target object, inputs the sequence into a trained semantic segmentation neural network, and obtains the recognition results and corresponding positions of key human body parts. Based on the recognition results and corresponding positions, a sequence of topological maps of key human body parts is created. The positional changes of each part are obtained from the topological map sequence, and associated parts are determined based on the degree of positional change. Weight values between nodes in the topological map of associated parts are set based on the mapping values of the degree of change of associated parts. The sequence of topological maps of key human body parts is input into a trained graph neural network model to obtain the behavioral features output by the graph neural network. By generating a sequence of topological maps from the positions of human body parts identified by the semantic segmentation neural network, and setting weights based on the correlation between moving parts of the human body, the trained graph neural network accurately identifies behavioral features, thereby enabling prediction of operations. This not only improves the accuracy of behavioral feature recognition but also allows for early prediction, providing early warning of abnormal operations.
[0044] In a preferred embodiment of this example, the method may further include the following steps: when some images in the reference image sequence are missing images of key human body parts, simulated images of key human body parts are generated based on the preceding and following images in the reference image sequence; simulated images are generated based on the simulated images of key human body parts to replace the original images and fill them into the reference image sequence. Due to limitations imposed by the position and angle of the image acquisition device, some acquired images may contain missing images of key human body parts. When this is the case, simulated images can be generated by calculating the spatial position of images acquired by cameras at other angles. If images acquired by other cameras still contain missing images of key human body parts, simulated images of key human body parts can be estimated by using the preceding and following images in the sequence that are closest in time to the original images and contain images of key human body parts, based on their positions through a compromise method. This facilitates the output of accurate behavioral features by the graph neural network model.
[0045] Example 2
[0046] Figure 2 This is a flowchart illustrating the behavioral feature recognition method provided in Embodiment 2 of the present invention. This embodiment is based on the above embodiment and is optimized by setting the weight values between nodes in the topology map of the associated parts according to the mapping value of the degree of change of the associated parts. Specifically, the optimization is as follows: set the degree of change mapping value for the current position of each associated part based on the maximum movement range of each associated part; select feature points from the associated parts based on the mapping value; establish multiple feature sequences based on the position of the feature points; and determine the weight values between nodes in the topology map of the associated parts according to the preset motion space aggregation point rules.
[0047] See Figure 2 The behavioral feature recognition method includes:
[0048] Step 210: Obtain a reference image sequence close to the target object, and input the reference image sequence into the trained semantic segmentation neural network to obtain the recognition results and corresponding positions of key human body parts.
[0049] Step 220: Create a sequence of topological maps of key human body parts based on the recognition results and corresponding locations.
[0050] Step 230: Obtain the positional changes of each part according to the topological map sequence of key human body parts, and determine the associated parts according to the degree of positional changes.
[0051] Step 240: Based on the maximum range of motion of each associated part, set a change degree mapping value for the current position of each associated part.
[0052] In this embodiment, the mapping value can be determined by the normalized value of the movement change of the corresponding part, that is, the maximum mapping value is based on the maximum movement range, and the corresponding mapping value is determined based on the range from 0 to the maximum movement range.
[0053] Step 250: Select feature points from the associated parts based on the mapping values, and establish multiple feature sequences based on the positions of the feature points.
[0054] Based on the mapping values, corresponding feature points can be determined. These feature points can be representative points in the image that reflect the motion characteristics of that region. A feature sequence for the image is generated based on the position of each feature point in the image; for multiple images, multiple feature sequences are generated.
[0055] Step 260: Aggregate the multiple feature sequences and determine the weight values between nodes in the topology map of the associated parts according to the preset motion space aggregation point rules.
[0056] Using the above method, feature points are mapped to a multi-dimensional space, and the corresponding weight values are determined based on the distance between the corresponding positions in the multi-dimensional space and the positions of the preset aggregation point rules. This method fully considers the subtle features of part movement and sets the weight values accordingly.
[0057] Step 270: Input the sequence of topological graphs of key human body parts into the trained graph neural network model to obtain the behavioral features output by the graph neural network.
[0058] For example, two thresholds can be set separately to correspond to the sequence and the mean squared error, respectively. When both are less than the two thresholds, it can be determined that the reconstruction requirements are met. Alternatively, different weight values can be set for the correlation between the two, and the correlation between the two can be added together using the weight values, and a comprehensive correlation threshold can be set to determine whether the reconstruction requirements are met.
[0059] This embodiment optimizes the setting of weight values between nodes in the topology map of associated parts based on the mapping value of the degree of change of the associated parts. Specifically, it optimizes this process by: setting a change degree mapping value for the current position of each associated part based on its maximum movement range; selecting feature points from the associated parts based on the mapping value; establishing multiple feature sequences based on the positions of the feature points; and determining the weight values between nodes in the topology map of associated parts according to a preset motion space aggregation point rule. This approach can further extract the features of the moving parts and set the weight values more scientifically and accurately.
[0060] In a preferred embodiment of this example, the method may include the following steps: when a tool is detected in the semantic segmentation neural network, a corresponding motion feature set is determined based on the attributes of the tool; correspondingly, the step of determining the weight values between nodes in the topology graph of associated parts according to preset motion space aggregation point rules is specifically optimized as follows: a preset motion space aggregation point rule is determined based on the motion feature set; the weight values between nodes in the topology graph of associated parts are determined based on the preset motion space aggregation point rule. By utilizing the tool identified in the image, the scope of recognition and prediction of the corresponding behavioral features can be narrowed, reducing the computational load without affecting the accuracy of the set weight values, thus improving computational efficiency.
[0061] Example 3
[0062] Figure 3 This is a schematic diagram of the behavioral feature recognition device provided in Embodiment 3 of the present invention. See also... Figure 3 The behavioral feature recognition device includes:
[0063] The acquisition module 310 is used to acquire a reference image sequence close to the target object, input the reference image sequence into the trained semantic segmentation neural network, and obtain the recognition results and corresponding positions of key human body parts;
[0064] A creation module 320 is used to create a sequence of topological maps of key human body parts based on the recognition results and corresponding locations.
[0065] The associated part determination module 330 is used to obtain the positional change of each part according to the sequence of topological maps of key human body parts, and determine the associated part according to the degree of positional change.
[0066] The setting module 340 is used to set the weight values between nodes in the topology map of the associated parts according to the mapping value of the degree of change of the associated parts;
[0067] The input module 350 is used to input the sequence of topological maps of key human body parts into the trained graph neural network model to obtain the behavioral features output by the graph neural network.
[0068] The behavioral feature recognition device provided in this embodiment acquires a reference image sequence approaching a target object, inputs the reference image sequence into a trained semantic segmentation neural network, and obtains the recognition results and corresponding positions of key human body parts. Based on the recognition results and corresponding positions, a topological map sequence of key human body parts is created. The positional changes of each part are obtained from the topological map sequence, and associated parts are determined based on the degree of positional change. Weight values between nodes in the topological map of associated parts are set based on the mapping value of the degree of change of associated parts. The topological map sequence of key human body parts is input into a trained graph neural network model to obtain the behavioral features output by the graph neural network. By generating a topological map sequence from the positions of human body parts identified by the semantic segmentation neural network, and setting weights based on the correlation between human movement parts, the trained graph neural network accurately identifies behavioral features, thereby enabling prediction of operations. This not only improves the accuracy of behavioral feature recognition but also allows for early prediction, providing early warning of abnormal operations.
[0069] Based on the above embodiments, the acquisition module includes:
[0070] Extraction unit, used to extract fixed objects from each image in the image sequence;
[0071] The coordinate system setting unit is used to set a reference coordinate system based on fixed objects in the image;
[0072] An adjustment unit is used to adjust the human body proportions of each image in the image sequence according to the reference coordinate system;
[0073] The generation unit is used to generate a reference image scale based on the adjusted human body proportions.
[0074] Based on the above embodiments, the setting module includes:
[0075] The setting unit is used to set a change degree mapping value for the current position of each associated part based on the maximum range of motion of each associated part.
[0076] A selection unit is used to select feature points from the associated region based on the mapping value;
[0077] A building unit is used to build multiple feature sequences based on the positions of the feature points;
[0078] The determining unit is used to aggregate the multiple feature sequences and determine the weight values between nodes in the topology map of the associated parts according to the preset motion space aggregation point rules.
[0079] Based on the above embodiments, the device further includes:
[0080] The generation module is used to generate simulated images of key human body parts based on the preceding and following images of the reference image sequence when some images in the reference image sequence are missing images of key human body parts.
[0081] The filling module is used to generate simulated images based on the simulated images of the key parts of the human body and fill them into the reference image sequence to replace the original images.
[0082] Based on the above embodiments, the associated part determination module includes:
[0083] The unit is used to determine the associated parts based on the movement correlation and the degree of positional change of human muscles.
[0084] Based on the above embodiments, the device further includes:
[0085] A motion feature set determination module is used to determine the corresponding motion feature set based on the attributes of the tool when the tool is detected in the semantic segmentation neural network.
[0086] Accordingly, the determining unit includes:
[0087] The rule determination subunit is used to determine a preset motion space aggregation point rule based on the motion feature set.
[0088] The weight value determination unit is used to determine the weight values between nodes in the topology map of related parts according to the preset rules of motion space aggregation points.
[0089] The behavioral feature recognition device provided in the embodiments of the present invention can execute the behavioral feature recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.
[0090] Example 4
[0091] Figure 4 This is a schematic diagram of the structure of a server provided in Embodiment 4 of the present invention. Figure 4 A block diagram of an exemplary server 12 suitable for implementing embodiments of the present invention is shown. Figure 4 The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0092] like Figure 4 As shown, server 12 is presented as a general-purpose computing device. The components of server 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).
[0093] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0094] Server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by server 12, including volatile and non-volatile media, removable and non-removable media.
[0095] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. Server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.
[0096] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.
[0097] Server 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable users to interact with server 12, and / or with any device that enables server 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, server 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of server 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0098] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the server file operation option recommendation method provided in the embodiments of the present invention.
[0099] Example 5
[0100] Embodiment 5 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform any of the behavioral feature recognition methods provided in the above embodiments.
[0101] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0102] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.
[0103] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0104] Computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or device. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0105] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.
Claims
1. A behavioral feature recognition method, characterized in that, include: A reference image sequence close to the target object is obtained, and the reference image sequence is input into a trained semantic segmentation neural network to obtain the recognition results and corresponding positions of key human body parts; A sequence of topological maps of key human body parts is created based on the recognition results and corresponding locations. The positional changes of each part are obtained based on the topological map sequence of key human body parts, and the associated parts are determined based on the degree of positional changes. The weight values between nodes in the topology graph of the associated parts are set according to the mapping value of the degree of change of the associated parts; The sequence of topological graphs of key human body parts is input into a trained graph neural network model to obtain the behavioral features output by the graph neural network model. The step of setting weight values between nodes in the topology map of related parts based on the mapping value of the degree of change of related parts includes: Based on the maximum range of motion of each associated part, a change degree mapping value is set for the current position of each associated part; Feature points are selected from the associated regions based on the mapping values; Multiple feature sequences are established based on the positions of the feature points; The multiple feature sequences are aggregated, and the weight values between nodes in the topology map of the associated parts are determined according to the preset motion space aggregation point rules.
2. The method according to claim 1, characterized in that, The acquisition of a reference image sequence close to the target object includes: Extract the fixed objects from each image in the reference image sequence; A reference coordinate system is established based on the fixed objects in the image; The proportions of the human body in each image of the reference image sequence are adjusted according to the reference coordinate system. A reference image scale is generated based on the adjusted human body proportions.
3. The method according to claim 1, characterized in that, The method further includes: When some images in the reference image sequence are missing images of key human body parts, simulated images of key human body parts are generated based on the preceding and following images in the reference image sequence. Simulated images are generated based on the simulated images of the key parts of the human body and are used to replace the original images in the reference image sequence.
4. The method according to claim 1, characterized in that, The step of determining the associated part based on the degree of positional change includes: The associated sites are determined based on the movement correlation and the degree of positional change of human muscles.
5. The method according to claim 1, characterized in that, The method further includes: When a tool is detected in the semantic segmentation neural network, the corresponding motion feature set is determined based on the attributes of the tool; Accordingly, determining the weight values between nodes in the topology map of associated parts according to the preset motion space aggregation point rules includes: Determine the preset motion space aggregation point rules based on the motion feature set; The weight values between nodes in the topology of related parts are determined according to the preset rules for aggregation points in the motion space.
6. A behavioral feature recognition device, characterized in that, include: The acquisition module is used to acquire a reference image sequence close to the target object, input the reference image sequence into the trained semantic segmentation neural network, and obtain the recognition results and corresponding positions of key human body parts; A creation module is used to create a sequence of topological maps of key human body parts based on the recognition results and corresponding locations. The associated part determination module is used to obtain the positional change of each part according to the sequence of topological maps of key human body parts, and determine the associated part according to the degree of positional change. The setting module is used to set the weight values between nodes in the topology map of related parts based on the mapping value of the degree of change of related parts; The input module is used to input the sequence of topological maps of key human body parts into the trained graph neural network model to obtain the behavioral features output by the graph neural network model. The setting module includes: The setting unit is used to set a change degree mapping value for the current position of each associated part based on the maximum range of motion of each associated part. A selection unit is used to select feature points from the associated region based on the mapping value; A building unit is used to build multiple feature sequences based on the positions of the feature points; The determining unit is used to aggregate the multiple feature sequences and determine the weight values between nodes in the topology map of the associated parts according to the preset motion space aggregation point rules.
7. A server, characterized in that, The server includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the behavior feature recognition method as described in any one of claims 1-5.
8. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the behavioral feature recognition method as described in any one of claims 1-5.
Citation Information
Patent Citations
Human body behavior recognition method and system based on graph convolution network
CN110796110A
