Information processing method and apparatus

Abstract

The application discloses an information processing method and device. The method comprises the following steps: obtaining a target video to be analyzed, wherein the target video comprises a plurality of image frames, and at least one image frame comprises information of a human object; identifying pixel coordinates of human key points of the human object in the at least one image frame; determining a directed graph sequence according to the pixel coordinates of the human key points; inputting the directed graph sequence into a U-shaped conditional directed graph convolution network for processing, and outputting posture information of the human object in the at least one image frame. The application solves the technical problem that the accuracy of human posture information recognized by the human posture recognition method in the related art is low.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and more specifically, to an information processing method and apparatus. Background Technology

[0002] In computer animation tasks such as virtual character-driven animation, it is necessary to capture realistic human postures to transfer realistic motion sequences to virtual characters. However, due to depth ambiguity, estimating the coordinates of keypoints in 3D physical space based on their pixel coordinates in 2D images is an ill-posed problem. Therefore, prior information about the human skeleton is often needed to assist this task. A common method is to represent the human skeletal structure as an undirected graph and apply a graph convolutional network. However, this method cannot accurately represent the hierarchical characteristics of the human skeletal structure, resulting in low accuracy in recognizing human postures.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides an information processing method and apparatus to at least address the technical problem of low accuracy in human posture recognition methods in related technologies.

[0005] According to one aspect of the present invention, an information processing method is provided, comprising: acquiring a target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; identifying the pixel coordinates of key human points of the human object in the at least one image frame; determining a directed graph sequence based on the pixel coordinates of the key human points; inputting the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and outputting the pose information of the human object in the at least one image frame.

[0006] Furthermore, determining the directed graph sequence based on the pixel coordinates of human body key points includes: determining the skeleton graph topology of the human body object based on the pixel coordinates of human body key points in at least one image frame; constructing a directed graph structure based on the skeleton graph topology of the human body object in at least one image frame; and connecting the directed graph structures of each frame to form a directed graph sequence.

[0007] Furthermore, the directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, where each edge is a vector composed of directed connections between nodes. The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing, outputting the pose information of the human object in at least one image frame. This includes: performing convolution processing on multiple nodes and edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolutionally processed nodes and edges; performing convolution processing on the convolutionally processed nodes and edges using the spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; and determining the three-dimensional pose information of the human object in at least one image frame using the three-dimensional coordinates of each node.

[0008] Furthermore, after performing convolution processing on multiple nodes and edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges, the method further includes: downsampling the convolution-processed nodes and edges using a preset downsampling factor to obtain downsampling result one; downsampling result one using a preset downsampling factor to obtain downsampling result two; and downsampling result two using a preset downsampling factor to obtain downsampling result three.

[0009] Furthermore, before processing the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the 3D coordinates of each node, the method further includes: processing the downsampling result three through the spatial-temporal directed graph convolution module to obtain the processed downsampling result three; fusing the processed downsampling result three with the downsampling result three processed by the spatial-temporal directed graph convolution module to obtain fused result one; upsampling the fused result one to obtain the upsampled fused result one; fusing the upsampled fused result one with the downsampling result two processed by the spatial-temporal directed graph convolution module to obtain fused result two; upsampling the fused result two to obtain the upsampled fused result two; fusing the upsampled fused result two with the downsampling result one processed by the spatial-temporal directed graph convolution module to obtain fused result three; fusing the fused result three with the convolutionally processed nodes and edges to obtain fused result four; and using fused result four as the input to the spatial-temporal conditional directed graph convolution module.

[0010] Furthermore, before convolving the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node, the method further includes: determining a preset number of weight values ​​through a routing function; determining the adjacency matrix basis function based on the directed connection relationships of nodes in the directed graph sequence, wherein the number of data in the adjacency matrix basis function is a preset number; and applying the preset number of weight values ​​to the adjacency matrix basis function in a linear combination to obtain the conditional adjacency matrix.

[0011] Furthermore, before convolving the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the 3D coordinates of each node, the method further includes: updating the 2D coordinates of each node in the directed graph sequence using adjacency matrix basis functions; updating the coordinates of the updated nodes again using a conditional adjacency matrix to obtain the target coordinates of the nodes; and updating the edges in the directed graph sequence using adjacency matrix basis functions to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input to the spatial-temporal conditional directed graph convolution module.

[0012] According to another aspect of the present invention, an information processing method is also provided, comprising: a cloud server receiving a target video to be analyzed from a client, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; identifying the pixel coordinates of human key points of the human object in the at least one image frame in the cloud server; determining a directed graph sequence based on the pixel coordinates of the human key points; inputting the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of the human object in the at least one image frame; and outputting the pose information of the human object in the at least one image frame to the client.

[0013] According to another aspect of the present invention, an information processing method is also provided, comprising: receiving an image processing instruction for a recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object; responding to the image processing instruction, processing at least one image frame in the target video to obtain a recognition result, wherein the pixel coordinates of human key points of the human object in the at least one image frame are identified, a directed graph sequence is determined based on the pixel coordinates of the human key points, the directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain pose information of the human object in the at least one image frame, and the pose information of the human object in the at least one image frame is used as the recognition result; and determining a strategy for performing a disease detection task on the human object based on the recognition result.

[0014] According to another aspect of the present invention, an information processing apparatus is also provided, comprising: a first acquisition unit, configured to acquire a target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; a first recognition unit, configured to recognize the pixel coordinates of key human points of the human object in at least one image frame; a first determination unit, configured to determine a directed graph sequence based on the pixel coordinates of the key human points; and a first processing unit, configured to input the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and output the pose information of the human object in at least one image frame.

[0015] Furthermore, the first determining unit includes: a first determining module, used to determine the skeleton graph topology of the human object based on the pixel coordinates of human key points in at least one image frame; a first constructing module, used to construct a directed graph structure based on the skeleton graph topology of the human object in at least one image frame; and a first acquiring module, used to connect the directed graph structures of each frame to form a directed graph sequence.

[0016] Furthermore, the directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes. The first processing unit includes: a first processing module, used to perform convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges; a second processing module, used to perform convolution processing on the convolution-processed nodes and edges using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; and a second determination module, used to determine the three-dimensional human pose information of the human object in at least one image frame using the three-dimensional coordinates of each node.

[0017] Furthermore, the device also includes: a second processing unit, used to perform convolution processing on multiple nodes and multiple edges in the directed graph sequence using a space-time directed graph convolution module to obtain convolution-processed nodes and edges, and then downsample the convolution-processed nodes and edges using a preset downsampling factor to obtain downsampling result one; a second acquisition unit, used to downsample downsampling result one using a preset downsampling factor to obtain downsampling result two; and a third acquisition unit, used to downsample downsampling result two using a preset downsampling factor to obtain downsampling result three.

[0018] Furthermore, the device also includes: a fourth acquisition unit, used to process the downsampled result three through a spatial-temporal directed graph convolution module to obtain the processed downsampled result three; a fifth acquisition unit, used to fuse the processed downsampled result three with the downsampled result three processed by the spatial-temporal directed graph convolution module to obtain a fused result one; a sixth acquisition unit, used to upsample the fused result one to obtain an upsampled fused result one; a seventh acquisition unit, used to fuse the upsampled fused result one with the downsampled result two processed by the spatial-temporal directed graph convolution module to obtain a fused result two; an eighth acquisition unit, used to upsample the fused result two to obtain an upsampled fused result two; a ninth acquisition unit, used to fuse the upsampled fused result two with the downsampled result one processed by the spatial-temporal directed graph convolution module to obtain a fused result three; a tenth acquisition unit, used to fuse the fused result three with the nodes and edges processed by convolution to obtain a fused result four; and an input unit, used to use the fused result four as the input to the spatial-temporal conditional directed graph convolution module.

[0019] Furthermore, the device also includes: a second determining unit, used to determine a preset number of weight values ​​through a routing function before performing convolution processing on the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; a third determining unit, used to determine the adjacency matrix basis function based on the directed connection relationship of nodes in the directed graph sequence, wherein the number of data in the adjacency matrix basis function is a preset number; and a ninth obtaining unit, used to apply the preset number of weight values ​​to the adjacency matrix basis function for linear combination to obtain the conditional adjacency matrix.

[0020] Furthermore, the second processing module includes: a first update submodule, used to update the two-dimensional coordinates of each node in the directed graph sequence using the adjacency matrix basis function; a second update submodule, used to update the coordinates of the updated nodes again using the conditional adjacency matrix to obtain the target coordinates of the nodes; and a third update submodule, used to update the edges in the directed graph sequence using the adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input to the spatial-temporal conditional directed graph convolution module.

[0021] According to another aspect of the present invention, an information processing apparatus is also provided, comprising: a first receiving unit, configured to receive a target video to be analyzed from a client via a cloud server, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; a fourth determining unit, configured to identify the pixel coordinates of human key points of the human object in the at least one image frame in the cloud server, determine a directed graph sequence based on the pixel coordinates of the human key points, input the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and obtain the pose information of the human object in the at least one image frame; and a first output unit, configured to output the pose information of the human object in the at least one image frame to the client.

[0022] According to another aspect of the present invention, an information processing apparatus is also provided, comprising: a second receiving unit, configured to receive an image processing instruction for a recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object; a first responding unit, configured to respond to the image processing instruction and process at least one image frame in the target video to obtain a recognition result, wherein the pixel coordinates of human key points of the human object in the at least one image frame are identified, a directed graph sequence is determined based on the pixel coordinates of the human key points, the directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain pose information of the human object in the at least one image frame, and the pose information of the human object in the at least one image frame is used as the recognition result; and a fifth determining unit, configured to determine a strategy for performing a disease detection task on the human object based on the recognition result.

[0023] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program executes any of the methods described above during runtime.

[0024] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program / instructions are stored, which, when executed by a processor, perform any of the methods described above.

[0025] In this embodiment of the invention, a U-shaped conditional directed graph convolutional network is used to estimate human pose. This involves acquiring a target video to be analyzed, which includes multiple image frames, at least one of which contains information about a human object. The pixel coordinates of key human points in at least one image frame are identified. Based on these pixel coordinates, a directed graph sequence is determined. This sequence is then input into the U-shaped conditional directed graph convolutional network for processing, outputting the pose information of the human object in at least one image frame. This achieves the goal of using conditional directed graph convolution technology to allow different types of actions to use appropriate topological connections, better utilizing the non-local dependencies between key human points. This improves the accuracy of human pose recognition and solves the problem of low accuracy in human pose recognition methods in related technologies. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0027] Figure 1 This is a hardware structure block diagram of a computer terminal according to an embodiment of the present invention;

[0028] Figure 2 This is a flowchart of the information processing method provided in Embodiment 1 of the present invention;

[0029] Figure 3 This is a schematic diagram of the spatial-temporal directed graph convolution module in the U-shaped conditional directed graph convolutional network provided in Embodiment 1 of the present invention;

[0030] Figure 4 This is a schematic diagram of the spatial-temporal conditional directed graph convolution module in the U-shaped conditional directed graph convolution network provided in Embodiment 1 of the present invention;

[0031] Figure 5 This is a schematic diagram of an optional information processing method provided according to Embodiment 1 of the present invention;

[0032] Figure 6 This is a schematic diagram of nodes and edges in the information processing method provided according to Embodiment 1 of the present invention;

[0033] Figure 7 This is a schematic diagram of node updating in the information processing method provided according to Embodiment 1 of the present invention;

[0034] Figure 8 This is a schematic diagram of a node updated using a conditional adjacency matrix in the information processing method provided according to Embodiment 1 of the present invention;

[0035] Figure 9 This is a schematic diagram of edge updating in the information processing method provided according to Embodiment 1 of the present invention;

[0036] Figure 10 This is the flow chart of the information processing method provided in Embodiment 2 of the present invention. Figure 1 ;

[0037] Figure 11 This is the flow chart of the information processing method provided in Embodiment 3 of the present invention. Figure 2 ;

[0038] Figure 12 This is a schematic diagram of an information processing apparatus provided according to Embodiment 4 of the present invention; and

[0039] Figure 13 This is a structural block diagram of an optional computer terminal according to an embodiment of the present invention. Detailed Implementation

[0040] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0042] First, some nouns or terms that appear in the description of the embodiments of the present invention shall be interpreted as follows:

[0043] 2D human pose: The position of key points of the human body in a 2D image plane.

[0044] 3D human pose: The position of key points of the human body in 3D physical space.

[0045] A directed graph sequence is a set of nodes and directed edges. Unlike undirected graphs, the edges in a directed graph are directional, and their direction determines the parent-child connection relationship between nodes.

[0046] Directed graph convolution: A generalization of the regular convolution operation defined in Euclidean space to directed graphs in non-Euclidean space.

[0047] Conditional directed graph convolution: Defined as a convolution operation on the topology of a directed graph that is adaptively generated based on the input.

[0048] Virtual character driver: Obtain source data of human body key points from RGB video stream and use it to drive any virtual character.

[0049] Example 1

[0050] According to an embodiment of the present invention, an embodiment of an information processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0051] The method embodiment provided in Embodiment 1 of the present invention can be executed in a mobile terminal, computer terminal or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing an information processing method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0052] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of the present invention, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0053] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the information processing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the information processing method of the application described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0054] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0055] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0056] Under the above operating environment, the present invention provides, as follows: Figure 2 The information processing method shown. Figure 2 This is a flowchart of an information processing method according to Embodiment 1 of the present invention.

[0057] Step S201: Obtain the target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object.

[0058] Step S202: Identify the pixel coordinates of key human body points of a human body object in at least one image frame.

[0059] A pose estimator is used to estimate the pixel positions of human key points in the image space of each frame, thus obtaining the two-dimensional pixel coordinates of each human key point, which can also be understood as 2D human pose.

[0060] Step S203: Determine the directed graph sequence based on the pixel coordinates of key points on the human body.

[0061] Optionally, the skeleton graph topology of the human object is determined based on the pixel coordinates of key points of the human body in at least one image frame; a directed graph structure is constructed based on the skeleton graph topology of the human object in at least one image frame; and the directed graph structures of each frame are connected to form a directed graph sequence.

[0062] For example, the number of keypoints on the human body is J, and the number of image frames in the target video is T. The resulting directed graph sequence is represented as: nodes + edges + adjacency matrix. Batch training can be used during training, with a batch size of B. Therefore, the vectors of the directed graph sequence are: nodes: Bx2xJxT (where 2 is the two-dimensional coordinates of the node, x, y); edges: Bx2xExT (where 2 is the two-dimensional vector of the edge, x, y, and E is the number of edges); and the adjacency matrix determined by the nodes and edges.

[0063] Step S204: Input the directed graph sequence into the U-shaped conditional directed graph convolutional network for processing, and output the pose information of the human object in at least one image frame.

[0064] A directed graph sequence (e.g., nodes: B×2×J×T and edges: B×2×E×T) is input into a U-shaped conditional directed graph convolutional network for processing. The output of the U-shaped conditional directed graph convolutional network is B×3×J×T, where 3 represents the three-dimensional coordinates of the nodes. Based on the three-dimensional coordinates of each human key point in at least one image frame, which can also be understood as the 3D human pose, the pose information of the human object is determined.

[0065] It should be noted that, in the processing flow, the directed graph sequence mentioned above can be displayed on the interface, and a schematic diagram of the directed graph sequence can be shown later. Figure 5 As shown in the diagram, this allows users to intuitively see the directed graph sequence converted from video frame images throughout the entire processing, as well as the 3D pose information of the human object output subsequently.

[0066] Through steps S201 to S204, conditional directed graph convolution is used to allow different types of actions to utilize appropriate topological connections, thereby better leveraging the non-local dependencies between key points on the human body. This improves the accuracy of human pose recognition and solves the problem of low accuracy in human pose recognition methods in related technologies. Only then can the accurately recognized human poses be transferred to virtual character avatars to perform computer animation tasks such as virtual character-driven animation.

[0067] Optionally, in the information processing method provided in this application embodiment, the directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes. Inputting the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing and outputting the pose information of a human object in at least one image frame includes: performing convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges; performing convolution processing on the convolution-processed nodes and edges using the spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; and determining the three-dimensional human pose information of the human object in at least one image frame using the three-dimensional coordinates of each node.

[0068] It should be noted that the U-shaped conditional directed graph convolutional network consists of the following five basic modules:

[0069] (1) ST-DGConv (Spatial-Temporal Directed Graph Convolution): A spatial-temporal directed graph convolution module, as follows: Figure 3 As shown, it includes directed graph convolution operations in the spatial domain and ordinary convolution operations in the temporal domain. For example... Figure 3 As shown, PREV LAYER OUTPUT is the input of the previous layer. For example, after passing through DGConv, the adjacent shapes of nodes B×C×J×T and edges B×C×E×T remain B×C×J×T and B×C×E×T. TemporalConv is a regular one-dimensional convolution and polynomial multiplication. After passing through it, the shapes become B×C'×J×T' and B×C'×E×T'.

[0070] (2) ST-CondDGconv (Spatial-Temporal Conditional Directed Graph Convolution): A spatial-temporal conditional directed graph convolution module used to adaptively select different graph topologies for different types of actions. See below. Figure 4 As shown, it includes a routing function to predict the weights of the topology basis from the output of the previous layer, and then, based on these weights and the topology basis functions, derives the topology suitable for the action (corresponding to...) through a linear combination. Figure 4 The predicted topology is used as the input for the conditional graph convolution CondDGConv operation. The output of CondDGConv is passed through TemporalConv and has the shape B×C'×J×T' and B×C'×E×T'.

[0071] (3) Temporal downsampling module, temporal upsampling module and fully connected layer. These three modules are common operations in convolutional neural networks and will not be elaborated here.

[0072] The above scheme outputs the three-dimensional coordinates of each node after processing by modules such as the spatial-temporal directed graph convolution module and the spatial-temporal conditional directed graph convolution module; the three-dimensional human pose information of the human object in at least one image frame can be accurately determined from the three-dimensional coordinates of each node.

[0073] Optionally, in the information processing method provided in the embodiments of this application, after performing convolution processing on multiple nodes and multiple edges in a directed graph sequence using a space-time directed graph convolution module to obtain convolution-processed nodes and edges, the method further includes: downsampling the convolution-processed nodes and edges using a preset downsampling factor to obtain downsampling result one; downsampling downsampling result one using a preset downsampling factor to obtain downsampling result two; and downsampling downsampling result two using a preset downsampling factor to obtain downsampling result three.

[0074] like Figure 5 As shown, after performing convolution processing on multiple nodes and edges in a directed graph sequence using a space-time directed graph convolution module, the output is B×16×J×T and B×16×E×T. For example, if the preset downsampling factor is 1 / 2, then downsampling the nodes and edges after convolution using a 1 / 2 sampling factor yields downsampling result one: B×32×J×T / 2 and B×32×E×T / 2; downsampling sample result one (B×32×J×T / 2 and B×32×E×T / 2) using a 1 / 2 sampling factor yields downsampling result two: B×64×J×T / 4 and B×64×E×T / 4; downsampling sample result two (B×64×J×T / 4 and B×64×E×T / 4) using a 1 / 2 sampling factor yields downsampling result three: B×128×J×T / 8 and B×128×E×T / 8.

[0075] Optionally, in the information processing method provided in this application embodiment, before performing convolution processing on the nodes and edges after convolution processing through the spatial-temporal conditional directed graph convolution module and outputting the three-dimensional coordinates of each node, the method further includes: processing the downsampling result three through the spatial-temporal directed graph convolution module to obtain the processed downsampling result three; fusing the processed downsampling result three with the downsampling result three processed by the spatial-temporal directed graph convolution module to obtain the fusion result one; upsampling the fusion result one to obtain the upsampled fusion result one; fusing the upsampled fusion result one with the downsampling result two processed by the spatial-temporal directed graph convolution module to obtain the fusion result two; upsampling the fusion result two to obtain the upsampled fusion result two; fusing the upsampled fusion result two with the downsampling result one processed by the spatial-temporal directed graph convolution module to obtain the fusion result three; fusing the fusion result three with the nodes and edges after convolution processing to obtain the fusion result four; and using the fusion result four as the input of the spatial-temporal conditional directed graph convolution module.

[0076] like Figure 5As shown, for example, the downsampling result three (B×128×J×T / 8 and B×128×E×T / 8) is processed by the spatial-temporal directed graph convolution module to obtain the processed downsampling result three; the processed downsampling result three is fused with the downsampling result three (B×128×J×T / 8 and B×128×E×T / 8) processed by the spatial-temporal directed graph convolution module to obtain the fused result one; the fused result one is upsampled to obtain the upsampled fused result one (B×64×J×T / 4 and B×64×E×T / 4). The upsampled fusion result 1 is fused with the downsampled result 2 (B×64×J×T / 4 and B×64×E×T / 4) processed by the spatial-temporal directed graph convolution module to obtain fusion result 2. Fusion result 2 is then upsampled to obtain upsampled fusion result 2 (B×32×J×T / 2 and B×32×E×T / 2). Fusion result 2 is then fused with the downsampled result 1 processed by the spatial-temporal directed graph convolution module to obtain fusion result 3 (B×16×J×T and B×16×E×T). Fusion result 3 is then fused with the nodes and edges processed by convolution (B×16×J×T and B×16×E×T) to obtain fusion result 4 (B×16×J×T and B×16×E×T). Fusion result 4 is then used as the input to the spatial-temporal conditional directed graph convolution module.

[0077] Optionally, in the information processing method provided in this application embodiment, before performing convolution processing on the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node, the method further includes: determining a preset number of weight values ​​through a routing function; determining an adjacency matrix basis function based on the directed connection relationship of nodes in the directed graph sequence, wherein the number of data in the adjacency matrix basis function is a preset number; and applying the preset number of weight values ​​to the adjacency matrix basis function in a linear combination to obtain a conditional adjacency matrix.

[0078] like Figure 4 As shown, PREV LAYER OUTPUT is the input of the previous layer, with input nodes: B×C×J×T and edges: B×C×E×T. Figure 4In this context, ROUTE FN is the routing function, outputting m weights. Its network structure can be: global average pooling + fully connected layer + sigmoid activation; {E1, E2, E3, ..., Em} are the adjacency matrix basis functions, numbered m, each adjacency matrix having a shape of JxJ, representing the directed connections between nodes in the directed graph, with positive and negative signs indicating parent-child nodes. These adjacency matrix basis functions are trainable, using sparse initialization and updated during training. Then, the m weights output by the routing function are linearly combined on the m adjacency matrix basis functions to obtain the conditional adjacency matrix (corresponding to...). Figure 4 (CondE in the middle).

[0079] In addition, the nodes (B×C×J×T), edges (B×C×E×T), and conditional adjacency matrix CondE input to PREV LAYER OUTPUT are used as input to CondDGConv. After processing, the shapes of the nodes and edges remain B×C×J×T and B×C×E×T, respectively.

[0080] Optionally, in the information processing method provided in this application embodiment, before the nodes and edges after convolution processing are convolved through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node, the method further includes: updating the two-dimensional coordinates of each node in the directed graph sequence using adjacency matrix basis functions; updating the coordinates of the updated nodes again using a conditional adjacency matrix to obtain the target coordinates of the nodes; and updating the edges in the directed graph sequence using adjacency matrix basis functions to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input to the spatial-temporal conditional directed graph convolution module.

[0081] For example, the input directed graph sequence is as follows: Figure 6 As shown, n i Represents a node, e i Represents the edges in a predefined adjacency matrix, Conde i This represents the edges in the conditional adjacency matrix obtained above.

[0082] First, node updates are performed using a predefined adjacency matrix. For example... Figure 7 As shown, taking node n4 as an example, node n4 is connected to input edge e3 and output edge (e4, e5). This step is defined as the convolution of the input edge, the node itself, and the output edge. The number of output edges may not be 1, so averaging can be used to obtain the aggregated features of the output edges. After the update, node n will be updated to n'.

[0083] Then, as Figure 8As shown, nodes are updated using the conditional adjacency matrix. Taking n'4 as an example, based on conditional connections, its parent node set is n'2 and its child node set is n'3. Since conditional connections are trainable, we can also aggregate the features of the parent node set and the child node set by averaging, and then calculate the convolution of the parent node set, the current node set, and the child node set. After this step, node n' will be updated to n.

[0084] Finally, as Figure 9 As shown, edges are updated using a predefined adjacency matrix (e.g., adjacency matrix basis functions). Taking edge e3 as an example, its source node is n”1, and its target node is n”4, both with a count of 1. Therefore, it can be defined as a convolution of the source node, the edge itself, and the target node. After this update, edge e will be updated to e'. After the above three steps, the CondDGConv operation is completed, thereby obtaining the topological structure of the human body in each image frame. The topological structure serves as the input to the spatial-temporal conditional directed graph convolution module. Through the above operations, different types of actions can use appropriate topological structures to optimally utilize the nonlocal dependency information between different nodes for 3D pose estimation.

[0085] Additionally, it should be noted that DGConv operates as follows: it updates nodes using a predefined adjacency matrix and updates edges using a predefined adjacency matrix (e.g., adjacency matrix basis functions). The specific process is the same as described above, and will not be repeated here.

[0086] In summary, the information processing method provided in this application acquires a target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame contains information about a human object; identifies the pixel coordinates of key human points of the human object in at least one image frame; determines a directed graph sequence based on the pixel coordinates of the key human points; inputs the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and outputs the pose information of the human object in at least one image frame. This achieves the goal of using conditional directed graph convolution technology to allow different types of actions to use appropriate topological connections, and better utilizes the non-local dependencies between key human points, thereby improving the accuracy of human pose recognition and solving the technical problem of low accuracy in human pose recognition methods in related technologies.

[0087] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0089] Example 2

[0090] The present invention also provides, for example Figure 10 The information processing method shown. Figure 10 This is a flowchart of an information processing method according to Embodiment 1 of the present invention.

[0091] Step S1001: The cloud server receives the target video to be analyzed from the client. The target video includes multiple image frames, and at least one image frame includes information about a human object.

[0092] Step S1002: Identify the pixel coordinates of human key points of human objects in at least one image frame in the cloud server, determine a directed graph sequence based on the pixel coordinates of human key points, and input the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of human objects in at least one image frame.

[0093] Step S1003: Output the pose information of the human object in at least one image frame to the client.

[0094] Through the above steps, conditional directed graph convolution technology can be used quickly and accurately in cloud servers to enable different types of actions to use appropriate topological connections, thereby making better use of the non-local dependencies between human body key points. This achieves the technical effect of improving the accuracy of human posture recognition, and solves the technical problem of low accuracy of human posture recognition methods in related technologies.

[0095] The specific methods and steps executed on the server are the same as those in Embodiment 1 above, and will not be repeated here.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0097] Example 3

[0098] This invention provides, for example Figure 11 The information processing method shown. Figure 11 This is a flowchart of an information processing method according to Embodiment 1 of the present invention.

[0099] Step S1101: Receive an image processing instruction for the recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object.

[0100] Step S1102: In response to the image processing instruction, the recognition result is obtained by processing at least one image frame in the target video. Specifically, the pixel coordinates of human key points of the human object in at least one image frame are identified. Based on the pixel coordinates of the human key points, a directed graph sequence is determined. The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of the human object in at least one image frame. The pose information of the human object in at least one image frame is used as the recognition result.

[0101] Step S1103: Based on the recognition results, determine the strategy for performing disease detection tasks on human subjects.

[0102] Through the steps described above, conditional directed graph convolution can be used to assign appropriate topological connections to different types of actions, better utilizing the non-local dependencies between key points on the human body. This improves the accuracy of human posture recognition and allows for analysis based on posture information to determine whether disease detection is necessary. For example, if the posture of the left leg in a human subject's walking posture is not normal, subsequent detection might be performed to check for numbness in the left leg, lumbar disc herniation, etc., automatically providing feedback for user reference and enabling users to quickly obtain relevant information.

[0103] It should be noted that the specific method steps performed in the recognition result obtained by processing at least one image frame in the target video are the same as those in the method in Embodiment 1 above, and will not be repeated here.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0105] Example 4

[0106] According to embodiments of the present invention, an apparatus for implementing the above-described information processing method is also provided, such as... Figure 12 As shown, the device includes: a first acquisition unit 1201, a first identification unit 1202, a first determination unit 1203, and a first processing unit 1204.

[0107] Specifically, the first acquisition unit 1201 is used to acquire the target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object.

[0108] The first recognition unit 1202 is used to recognize the pixel coordinates of key points of human body objects in at least one image frame.

[0109] The first determining unit 1203 is used to determine a directed graph sequence based on the pixel coordinates of key points of the human body;

[0110] The first processing unit 1204 is used to input the directed graph sequence into the U-shaped conditional directed graph convolutional network for processing and output the pose information of the human object in at least one image frame.

[0111] The information processing apparatus provided in Embodiment 4 of this application uses conditional directed graph convolution technology to enable different types of actions to use appropriate topological connection relationships, thereby making better use of the non-local dependencies between key points of the human body. This achieves the technical effect of improving the accuracy of human posture recognition, and solves the technical problem of low accuracy of human posture recognition methods in related technologies.

[0112] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, the first determining unit 1203 includes: a first determining module, configured to determine the skeleton graph topology of a human object based on the pixel coordinates of key points of the human body in at least one image frame; a first constructing module, configured to construct a directed graph structure based on the skeleton graph topology of the human object in at least one image frame; and a first acquiring module, configured to connect the directed graph structures of each frame to form a directed graph sequence.

[0113] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, the directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes, and the first processing unit includes: a first processing module, used to perform convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges; a second processing module, used to perform convolution processing on the convolution-processed nodes and edges using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; and a second determining module, used to determine the three-dimensional human pose information of a human object in at least one image frame using the three-dimensional coordinates of each node.

[0114] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, the apparatus further includes: a second processing unit, configured to perform convolution processing on multiple nodes and multiple edges in a directed graph sequence using a space-time directed graph convolution module to obtain convolution-processed nodes and edges, and then downsample the convolution-processed nodes and edges using a preset downsampling factor to obtain downsampling result one; a second acquisition unit, configured to downsample downsampling result one using a preset downsampling factor to obtain downsampling result two; and a third acquisition unit, configured to downsample downsampling result two using a preset downsampling factor to obtain downsampling result three.

[0115] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, the apparatus further includes: a fourth acquisition unit, used to process the downsampling result 3 through a spatial-temporal directed graph convolution module to obtain a processed downsampling result 3; a fifth acquisition unit, used to fuse the processed downsampling result 3 with the downsampling result 3 processed by the spatial-temporal directed graph convolution module to obtain a fusion result 1; a sixth acquisition unit, used to upsample the fusion result 1 to obtain an upsampled fusion result 1; a seventh acquisition unit, used to fuse the upsampled fusion result 1 with the downsampling result 2 processed by the spatial-temporal directed graph convolution module to obtain a fusion result 2; an eighth acquisition unit, used to upsample the fusion result 2 to obtain an upsampled fusion result 2; a ninth acquisition unit, used to fuse the upsampled fusion result 2 with the downsampling result 1 processed by the spatial-temporal directed graph convolution module to obtain a fusion result 3; a tenth acquisition unit, used to fuse the fusion result 3 with the nodes and edges processed by convolution to obtain a fusion result 4; and an input unit, used to use the fusion result 4 as the input to the spatial-temporal conditional directed graph convolution module.

[0116] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, the apparatus further includes: a second determining unit, configured to determine a preset number of weight values ​​through a routing function before performing convolution processing on the nodes and edges after convolution processing through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; a third determining unit, configured to determine the adjacency matrix basis function based on the directed connection relationship of nodes in the directed graph sequence, wherein the number of data in the adjacency matrix basis function is a preset number; and a ninth obtaining unit, configured to apply the preset number of weight values ​​to the adjacency matrix basis function for linear combination to obtain a conditional adjacency matrix.

[0117] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, the second processing module includes: a first update submodule, used to update the two-dimensional coordinates of each node in the directed graph sequence using an adjacency matrix basis function; a second update submodule, used to update the coordinates of the updated nodes again using a conditional adjacency matrix to obtain the target coordinates of the nodes; and a third update submodule, used to update the edges in the directed graph sequence using an adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input to the spatial-temporal conditional directed graph convolution module.

[0118] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, an apparatus for implementing the above-mentioned information processing is also provided, comprising: a first receiving unit, configured to receive a target video to be analyzed from a client via a cloud server, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; a fourth determining unit, configured to identify the pixel coordinates of human key points of the human object in the at least one image frame in the cloud server, determine a directed graph sequence based on the pixel coordinates of the human key points, input the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and obtain the pose information of the human object in the at least one image frame; and a first output unit, configured to output the pose information of the human object in the at least one image frame to the client.

[0119] Optionally, in the information processing apparatus provided in Embodiment 4 of the present invention, an apparatus for implementing the above-mentioned information processing is also provided, comprising: a second receiving unit, configured to receive an image processing instruction for a recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object; a first responding unit, configured to respond to the image processing instruction and process at least one image frame in the target video to obtain a recognition result, wherein the pixel coordinates of human key points of the human object in the at least one image frame are identified, a directed graph sequence is determined based on the pixel coordinates of the human key points, the directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of the human object in the at least one image frame, and the pose information of the human object in the at least one image frame is used as the recognition result; and a fifth determining unit, configured to determine a strategy for performing a disease detection task on the human object based on the recognition result.

[0120] It should be noted that the aforementioned acquisition unit 1201, first identification unit 1202, first determination unit 1203, and first processing unit 1204 correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by the two modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0121] Example 5

[0122] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0123] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0124] In this embodiment, the computer terminal described above can execute the program code for the following steps in the information processing method of the application: acquiring the target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; identifying the pixel coordinates of key human points of the human object in at least one image frame; determining a directed graph sequence based on the pixel coordinates of the key human points; inputting the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and outputting the pose information of the human object in at least one image frame.

[0125] The aforementioned computer terminal can also execute program code for the following steps in the information processing method of the application: determining the skeleton graph topology of the human object based on the pixel coordinates of key points of the human body in at least one image frame; constructing a directed graph structure based on the skeleton graph topology of the human object in at least one image frame; and connecting the directed graph structures of each frame to form a directed graph sequence.

[0126] The aforementioned computer terminal can also execute program code for the following steps in the information processing method of the application: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes; a spatial-temporal directed graph convolution module is used to convolve multiple nodes and multiple edges in the directed graph sequence to obtain convolved nodes and edges; the convolved nodes and edges are then convolved again using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; the three-dimensional coordinates of each node are used to determine the three-dimensional human pose information of the human object in at least one image frame.

[0127] The aforementioned computer terminal can also execute program code for the following steps in the information processing method of the application: wherein, after performing convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges, the convolution-processed nodes and edges are downsampled using a preset downsampling factor to obtain downsampling result one; downsampling result one is downsampled using a preset downsampling factor to obtain downsampling result two; downsampling result two is downsampled using a preset downsampling factor to obtain downsampling result three.

[0128] The aforementioned computer terminal can also execute the program code for the following steps in the information processing method of the application: processing downsampling result three through a spatial-temporal directed graph convolution module to obtain processed downsampling result three; fusing processed downsampling result three with downsampling result three processed by the spatial-temporal directed graph convolution module to obtain fused result one; upsampling fused result one to obtain upsampled fused result one; fusing upsampled fused result one with downsampling result two processed by the spatial-temporal directed graph convolution module to obtain fused result two; upsampling fused result two to obtain upsampled fused result two; fusing upsampled fused result two with downsampling result one processed by the spatial-temporal directed graph convolution module to obtain fused result three; fusing fused result three with nodes and edges processed by convolution to obtain fused result four; and using fused result four as input to the spatial-temporal conditional directed graph convolution module.

[0129] The aforementioned computer terminal can also execute program code for the following steps in the information processing method of the application: Before convolving the convolved nodes and edges through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node, a preset number of weight values ​​are determined through a routing function; based on the directed connection relationships of nodes in the directed graph sequence, the adjacency matrix basis function is determined, wherein the number of data in the adjacency matrix basis function is a preset number; the preset number of weight values ​​are applied to the adjacency matrix basis function for linear combination to obtain the conditional adjacency matrix.

[0130] The aforementioned computer terminal can also execute program code for the following steps in the information processing method of the application: wherein, the two-dimensional coordinates of each node in the directed graph sequence are updated using the adjacency matrix basis function; the updated coordinates of the nodes are updated again using the conditional adjacency matrix to obtain the target coordinates of the nodes; the edges in the directed graph sequence are updated using the adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input to the spatial-temporal conditional directed graph convolution module.

[0131] The aforementioned computer terminal can also execute program code for the following steps in the information processing method of the application: the cloud server receives a target video to be analyzed from the client, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; the cloud server identifies the pixel coordinates of key human points of the human object in at least one image frame, determines a directed graph sequence based on the pixel coordinates of the key human points, inputs the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and obtains the pose information of the human object in at least one image frame; and outputs the pose information of the human object in at least one image frame to the client.

[0132] The aforementioned computer terminal can also execute program code for the following steps in the information processing method of the application: receiving an image processing instruction for a recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object; responding to the image processing instruction, processing at least one image frame in the target video to obtain a recognition result, wherein the pixel coordinates of human key points of the human object in the at least one image frame are identified, a directed graph sequence is determined based on the pixel coordinates of the human key points, the directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of the human object in the at least one image frame, and the pose information of the human object in the at least one image frame is used as the recognition result; and based on the recognition result, determining a strategy for performing a disease detection task on the human object.

[0133] Optionally, Figure 13 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 13 As shown, the computer terminal may include: one or more ( Figure 13 (Only one is shown in the image) processor and memory.

[0134] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the information processing method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned information processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0135] The processor can access the information and application programs stored in the memory via the transmission device to perform the following steps: acquiring the target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; identifying the pixel coordinates of key human points of the human object in at least one image frame; determining a directed graph sequence based on the pixel coordinates of the key human points; inputting the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and outputting the pose information of the human object in at least one image frame.

[0136] Optionally, the processor may also execute program code that performs the following steps: determining the skeleton graph topology of the human object based on the pixel coordinates of key points of the human body in at least one image frame; constructing a directed graph structure based on the skeleton graph topology of the human object in at least one image frame; and connecting the directed graph structures of each frame to form a directed graph sequence.

[0137] Optionally, the processor may also execute program code for the following steps: 2D coordinates of multiple nodes, multiple edges, and an adjacency matrix, where each edge is a vector composed of directed connections between nodes; convolution processing of multiple nodes and edges in a directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges; convolution processing of the convolution-processed nodes and edges using a spatial-temporal conditional directed graph convolution module to output the 3D coordinates of each node; determining the 3D human pose information of the human object in at least one image frame using the 3D coordinates of each node.

[0138] Optionally, the processor may also execute program code with the following steps: after performing convolution processing on multiple nodes and edges in the directed graph sequence using a space-time directed graph convolution module to obtain convolution-processed nodes and edges, downsampling the convolution-processed nodes and edges using a preset downsampling factor to obtain downsampling result one; downsampling result one using a preset downsampling factor to obtain downsampling result two; downsampling result two using a preset downsampling factor to obtain downsampling result three.

[0139] Optionally, the processor may also execute program code with the following steps: before convolving the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the 3D coordinates of each node, the downsampling result three is processed by the spatial-temporal directed graph convolution module to obtain the processed downsampling result three; the processed downsampling result three is fused with the downsampling result three processed by the spatial-temporal directed graph convolution module to obtain fused result one; fused result one is upsampled to obtain upsampled fused result one; upsampled fused result one is fused with the downsampling result two processed by the spatial-temporal directed graph convolution module to obtain fused result two; fused result two is upsampled to obtain upsampled fused result two; upsampled fused result two is fused with the downsampling result one processed by the spatial-temporal directed graph convolution module to obtain fused result three; fused result three is fused with the convolutionally processed nodes and edges to obtain fused result four; fused result four is used as the input to the spatial-temporal conditional directed graph convolution module.

[0140] Optionally, the processor may also execute program code with the following steps: before convolving the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node, a preset number of weight values ​​are determined through a routing function; based on the directed connection relationships of nodes in the directed graph sequence, the adjacency matrix basis function is determined, wherein the number of data in the adjacency matrix basis function is a preset number; the preset number of weight values ​​are applied to the adjacency matrix basis function for linear combination to obtain the conditional adjacency matrix.

[0141] Optionally, the processor may also execute program code with the following steps: updating the two-dimensional coordinates of each node in the directed graph sequence using adjacency matrix basis functions; updating the coordinates of the updated nodes again using conditional adjacency matrices to obtain the target coordinates of the nodes; updating the edges in the directed graph sequence using adjacency matrix basis functions to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input to the spatial-temporal conditional directed graph convolution module.

[0142] Optionally, the processor may also execute program code for the following steps: the cloud server receives a target video to be analyzed from the client, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; the cloud server identifies the pixel coordinates of key human points of the human object in at least one image frame, determines a directed graph sequence based on the pixel coordinates of the key human points, inputs the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and obtains the pose information of the human object in at least one image frame; and outputs the pose information of the human object in at least one image frame to the client.

[0143] Optionally, the processor may also execute program code for the following steps: receiving an image processing instruction for a recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object; responding to the image processing instruction, processing at least one image frame in the target video to obtain a recognition result, wherein the pixel coordinates of human key points of the human object in the at least one image frame are identified, a directed graph sequence is determined based on the pixel coordinates of the human key points, the directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of the human object in the at least one image frame, and the pose information of the human object in the at least one image frame is used as the recognition result; and based on the recognition result, determining a strategy for performing a disease detection task on the human object.

[0144] This invention provides an information processing method. By employing a U-shaped conditional directed graph convolutional network to estimate human pose, the method involves acquiring a target video to be analyzed, wherein the target video includes multiple image frames, at least one of which contains information about a human object; identifying the pixel coordinates of key human points in at least one image frame; determining a directed graph sequence based on the pixel coordinates of the key human points; inputting the directed graph sequence into the U-shaped conditional directed graph convolutional network for processing; and outputting the pose information of the human object in at least one image frame. This method achieves the goal of using conditional directed graph convolution technology to allow different types of actions to use appropriate topological connections, better utilizing the non-local dependencies between key human points, thereby improving the accuracy of human pose recognition and solving the technical problem of low accuracy in human pose recognition methods in related technologies.

[0145] Those skilled in the art will understand that Figure 13 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 13 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include components that are more advanced than those described above. Figure 13 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 13 The different configurations shown.

[0146] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0147] Example 6

[0148] Embodiments of the present invention also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the information processing method provided in Embodiment 1.

[0149] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0150] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; identifying the pixel coordinates of key human points of the human object in at least one image frame; determining a directed graph sequence based on the pixel coordinates of the key human points; inputting the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and outputting the pose information of the human object in at least one image frame.

[0151] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: determining the skeleton graph topology of the human object based on the pixel coordinates of key points of the human body in at least one image frame; constructing a directed graph structure based on the skeleton graph topology of the human object in at least one image frame; and connecting the directed graph structures of each frame to form a directed graph sequence.

[0152] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes; a spatial-temporal directed graph convolution module is used to convolve multiple nodes and multiple edges in a directed graph sequence to obtain convolved nodes and edges; the convolved nodes and edges are then convolved using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; and the three-dimensional coordinates of each node are used to determine the three-dimensional human pose information of a human object in at least one image frame.

[0153] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: wherein, after performing convolution processing on multiple nodes and multiple edges in the directed graph sequence using a space-time directed graph convolution module to obtain convolution-processed nodes and edges, the convolution-processed nodes and edges are downsampled using a preset downsampling factor to obtain downsampling result one; downsampling result one is downsampled using a preset downsampling factor to obtain downsampling result two; downsampling result two is downsampled using a preset downsampling factor to obtain downsampling result three.

[0154] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: Before convolving the convolutionally processed nodes and edges through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node, the downsampling result three is processed by the spatial-temporal directed graph convolution module to obtain the processed downsampling result three; the processed downsampling result three is fused with the downsampling result three processed by the spatial-temporal directed graph convolution module to obtain fused result one; fused result one is upsampled to obtain upsampled fused result one; upsampled fused result one is fused with the downsampling result two processed by the spatial-temporal directed graph convolution module to obtain fused result two; fused result two is upsampled to obtain upsampled fused result two; upsampled fused result two is fused with the downsampling result one processed by the spatial-temporal directed graph convolution module to obtain fused result three; fused result three is fused with the convolutionally processed nodes and edges to obtain fused result four; fused result four is used as the input to the spatial-temporal conditional directed graph convolution module.

[0155] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: wherein, before convolving the convolved nodes and edges through a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node, a preset number of weight values ​​are determined through a routing function; based on the directed connection relationships of nodes in the directed graph sequence, an adjacency matrix basis function is determined, wherein the number of data in the adjacency matrix basis function is a preset number; and the preset number of weight values ​​are applied to the adjacency matrix basis function for linear combination to obtain a conditional adjacency matrix.

[0156] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: wherein the two-dimensional coordinates of each node in the directed graph sequence are updated using the adjacency matrix basis function; the updated coordinates of the nodes are updated again using the conditional adjacency matrix to obtain the target coordinates of the nodes; the edges in the directed graph sequence are updated using the adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input to the spatial-temporal conditional directed graph convolution module.

[0157] Optionally, the processor may also execute program code for the following steps: the cloud server receives a target video to be analyzed from the client, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; the cloud server identifies the pixel coordinates of key human points of the human object in at least one image frame, determines a directed graph sequence based on the pixel coordinates of the key human points, inputs the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing, and obtains the pose information of the human object in at least one image frame; and outputs the pose information of the human object in at least one image frame to the client.

[0158] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: receiving an image processing instruction for a recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object; responding to the image processing instruction, processing at least one image frame in the target video to obtain a recognition result, wherein the pixel coordinates of human key points of the human object in the at least one image frame are identified, a directed graph sequence is determined based on the pixel coordinates of the human key points, the directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of the human object in the at least one image frame, and the pose information of the human object in the at least one image frame is used as the recognition result; and based on the recognition result, determining a strategy for performing a disease detection task on the human object.

[0159] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0160] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0161] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0165] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An information processing method, characterized in that, include: Acquire the target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object; Identify the pixel coordinates of key human points of a human object in at least one image frame; Based on the pixel coordinates of the key points of the human body, a directed graph sequence is determined; The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing, and the pose information of the human object in at least one image frame is output. The U-shaped conditional directed graph convolutional network includes a spatial-temporal directed graph convolutional module, a temporal downsampling module, a temporal upsampling module, a spatial-temporal conditional directed graph convolutional module, and a fully connected layer connected in sequence. The directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes. The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing, and the pose information of the human object in at least one image frame is output by: performing convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges. A preset number of weight values ​​are determined through a routing function; based on the directed connection relationships of nodes in the directed graph sequence, an adjacency matrix basis function is determined, wherein the number of data in the adjacency matrix basis function is the preset number; the preset number of weight values ​​are applied to the adjacency matrix basis function for linear combination to obtain a conditional adjacency matrix; The two-dimensional coordinates of each node in the directed graph sequence are updated using the adjacency matrix basis function; the updated coordinates of the nodes are updated again using the conditional adjacency matrix to obtain the target coordinates of the nodes; the edges in the directed graph sequence are updated using the adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input of the spatial-temporal conditional directed graph convolution module; The nodes and edges after convolution are then convolved using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; the three-dimensional coordinates of each node are used to determine the three-dimensional human pose information of the human object in at least one image frame.

2. The method according to claim 1, characterized in that, Based on the pixel coordinates of the key human body points, the directed graph sequence is determined as follows: The skeleton topology of the human body object is determined based on the pixel coordinates of the human body key points in at least one image frame. A directed graph structure is constructed based on the skeleton graph topology of the human object in at least one image frame; The directed graph structures of each frame are connected to form the directed graph sequence.

3. The method according to claim 1, characterized in that, in, After performing convolution processing on multiple nodes and edges in the directed graph sequence using a space-time directed graph convolution module to obtain the convolutionally processed nodes and edges, the method further includes: The nodes and edges after convolution are downsampled using a preset downsampling factor to obtain downsampling result one. The downsampling result one is downsampled using the preset downsampling factor to obtain downsampling result two; The downsampling result two is downsampled using the preset downsampling factor to obtain downsampling result three.

4. The method according to claim 3, characterized in that, in, Before performing convolution processing on the convolutionally processed nodes and edges through a space-time conditional directed graph convolution module to output the three-dimensional coordinates of each node, the method further includes: The downsampling result three is processed by the spatial-temporal directed graph convolution module to obtain the processed downsampling result three. The processed downsampling result three is fused with the downsampling result three processed by the spatial-temporal directed graph convolution module to obtain fusion result one. The fusion result 1 is upsampled to obtain the upsampled fusion result 1. The upsampled fusion result one is fused with the downsampled result two processed by the spatial-temporal directed graph convolution module to obtain fusion result two. The second fusion result is upsampled to obtain the second upsampled fusion result. The upsampled fusion result 2 is fused with the downsampled result 1 processed by the spatial-temporal directed graph convolution module to obtain fusion result 3. The third fusion result is then fused with the nodes and edges after convolution to obtain the fourth fusion result. The fusion result 4 is used as the input to the spatial-temporal conditional directed graph convolution module.

5. An information processing method, characterized in that, include: The cloud server receives a target video to be analyzed from the client, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object. In the cloud server, the pixel coordinates of human key points of human objects in at least one image frame are identified. Based on the pixel coordinates of the human key points, a directed graph sequence is determined. The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing to obtain the pose information of human objects in at least one image frame. The pose information of the human object in at least one image frame is output to the client. The U-shaped conditional directed graph convolutional network includes a spatial-temporal directed graph convolutional module, a temporal downsampling module, a temporal upsampling module, a spatial-temporal conditional directed graph convolutional module, and a fully connected layer connected in sequence. The directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes. The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing, and the pose information of the human object in at least one image frame is output by: performing convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges. A preset number of weight values ​​are determined through a routing function; based on the directed connection relationships of nodes in the directed graph sequence, an adjacency matrix basis function is determined, wherein the number of data in the adjacency matrix basis function is the preset number; the preset number of weight values ​​are applied to the adjacency matrix basis function for linear combination to obtain a conditional adjacency matrix; The two-dimensional coordinates of each node in the directed graph sequence are updated using the adjacency matrix basis function; the updated coordinates of the nodes are updated again using the conditional adjacency matrix to obtain the target coordinates of the nodes; the edges in the directed graph sequence are updated using the adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input of the spatial-temporal conditional directed graph convolution module; The nodes and edges after convolution are then convolved using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; the three-dimensional coordinates of each node are used to determine the three-dimensional human pose information of the human object in at least one image frame.

6. An information processing method, characterized in that, include: Receive an image processing instruction for a recognition task, wherein the image processing instruction includes a target video, the target video includes multiple image frames, and at least one image frame includes information about a human object; In response to the image processing instruction, a recognition result is obtained by processing at least one image frame in the target video, wherein the pixel coordinates of human key points of a human object in at least one image frame are identified, a directed graph sequence is determined based on the pixel coordinates of the human key points, the directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing, the pose information of the human object in at least one image frame is obtained, and the pose information of the human object in at least one image frame is used as the recognition result; Based on the identification results, a strategy for performing disease detection tasks on the human body is determined; The U-shaped conditional directed graph convolutional network includes a spatial-temporal directed graph convolutional module, a temporal downsampling module, a temporal upsampling module, a spatial-temporal conditional directed graph convolutional module, and a fully connected layer connected in sequence. The directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes. The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing, and the pose information of the human object in at least one image frame is output by: performing convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges. A preset number of weight values ​​are determined through a routing function; based on the directed connection relationships of nodes in the directed graph sequence, an adjacency matrix basis function is determined, wherein the number of data in the adjacency matrix basis function is the preset number; the preset number of weight values ​​are applied to the adjacency matrix basis function for linear combination to obtain a conditional adjacency matrix; The two-dimensional coordinates of each node in the directed graph sequence are updated using the adjacency matrix basis function; the updated coordinates of the nodes are updated again using the conditional adjacency matrix to obtain the target coordinates of the nodes; the edges in the directed graph sequence are updated using the adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input of the spatial-temporal conditional directed graph convolution module; The nodes and edges after convolution are then convolved using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; the three-dimensional coordinates of each node are used to determine the three-dimensional human pose information of the human object in at least one image frame.

7. An information processing device, characterized in that, include: The first acquisition unit is used to acquire the target video to be analyzed, wherein the target video includes multiple image frames, and at least one image frame includes information about a human object. The first recognition unit is used to recognize the pixel coordinates of key points of a human body object in at least one image frame. The first determining unit is used to determine a directed graph sequence based on the pixel coordinates of the key points of the human body; The first processing unit is used to input the directed graph sequence into a U-shaped conditional directed graph convolutional network for processing and output the pose information of the human object in at least one image frame. The U-shaped conditional directed graph convolutional network includes a spatial-temporal directed graph convolutional module, a temporal downsampling module, a temporal upsampling module, a spatial-temporal conditional directed graph convolutional module, and a fully connected layer connected in sequence. The directed graph sequence includes: two-dimensional coordinates of multiple nodes, multiple edges, and an adjacency matrix, wherein the edges are vectors composed of directed connections between nodes. The directed graph sequence is input into a U-shaped conditional directed graph convolutional network for processing, and the pose information of the human object in at least one image frame is output by: performing convolution processing on multiple nodes and multiple edges in the directed graph sequence using a spatial-temporal directed graph convolution module to obtain convolution-processed nodes and edges. A preset number of weight values ​​are determined through a routing function; based on the directed connection relationships of nodes in the directed graph sequence, an adjacency matrix basis function is determined, wherein the number of data in the adjacency matrix basis function is the preset number; the preset number of weight values ​​are applied to the adjacency matrix basis function for linear combination to obtain a conditional adjacency matrix; The two-dimensional coordinates of each node in the directed graph sequence are updated using the adjacency matrix basis function; the updated coordinates of the nodes are updated again using the conditional adjacency matrix to obtain the target coordinates of the nodes; the edges in the directed graph sequence are updated using the adjacency matrix basis function to obtain the topological structure of the human body in each image frame, wherein the topological structure serves as the input of the spatial-temporal conditional directed graph convolution module; The nodes and edges after convolution are then convolved using a spatial-temporal conditional directed graph convolution module to output the three-dimensional coordinates of each node; the three-dimensional coordinates of each node are used to determine the three-dimensional human pose information of the human object in at least one image frame.

Images