A method and device for skeleton matching

Abstract

The present application provides a skeleton matching method and apparatus, which are applied to the field of action recognition. The skeleton matching method includes: classifying a set of to-be-classified 3D points corresponding to the to-be-classified action to obtain a first classification result; sequentially matching the to-be-classified 3D points in each category in the first classification result with an initial skeleton to obtain a standard skeleton corresponding to the to-be-classified action. In the above solution, before matching the 3D points and the initial skeleton, the 3D points can be classified first, and then the 3D points in each category are matched separately. Since the 3D points in each category are matched separately, the global matching of each 3D point is avoided. Therefore, the matching efficiency can be improved; at the same time, during the process of matching the 3D points in each category, there will be no situation of matching errors. Therefore, the matching accuracy can be improved.

Description

Technical Field

[0001] This application relates to the technical field of action recognition. Specifically, it relates to a skeleton matching method and device. Background Art

[0002] With the development of 3D technology, 3D point matching is required in more and more scenarios. For example, in the scenario of action recognition, it is necessary to match the 3D points corresponding to a person's action with an initial skeleton. In the prior art, when performing skeleton matching of 3D points, it is usually achieved by calculating the distance to match the 3D points with the initial skeleton. However, the efficiency of this matching method is low and it is prone to matching errors. For example, in a photo with a tilted head, the shoulder point is very close to the head point, so the shoulder point will be matched to the head. Therefore, the matching efficiency and accuracy of 3D points and the initial skeleton in the prior art are both low. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a skeleton matching method and device to solve the technical problems that the matching efficiency and accuracy of 3D points and the initial skeleton in the prior art are both low.

[0004] In a first aspect, the embodiments of this application provide a skeleton matching method, including: classifying a group of to-be-classified 3D points corresponding to a to-be-classified action to obtain a first classification result; sequentially matching the to-be-classified 3D points in each category in the first classification result with the initial skeleton to obtain a standard skeleton corresponding to the to-be-classified action. In the above solution, before matching the 3D points and the initial skeleton, the 3D points can be classified first, and then the 3D points in each category are respectively matched. Since the 3D points in each category are respectively matched, the global matching of each 3D point is avoided. Therefore, the matching efficiency can be improved; at the same time, during the process of matching the 3D points in each category, there will be no matching error situation. Therefore, the matching accuracy can be improved.

[0005] In an optional implementation manner, the classifying the group of to-be-classified 3D points corresponding to the to-be-classified action includes: classifying the to-be-classified 3D points according to a second classification result and a standard distance corresponding to a group of standard 3D points corresponding to a standard action, and in a preset order. In the above solution, the to-be-classified 3D points can be classified in a preset order based on the second classification result and the standard distance corresponding to the standard 3D points. Since the to-be-classified 3D points in each category are classified in a preset order, the global classification of each to-be-classified 3D point is avoided. Therefore, the classification efficiency can be improved.

[0006] In an alternative embodiment, the standard distance includes all straight-line distances and one-way closed straight-line distances; before classifying the 3D points to be classified according to the second classification result corresponding to a set of standard 3D points corresponding to a standard action and the standard distance, the method further includes: classifying the standard 3D points by using a classification algorithm to obtain the second classification result; for multiple standard 3D points in each category in the second classification result, calculating multiple all straight-line distances between the multiple standard 3D points, and determining multiple one-way closed straight-line distances between the multiple standard 3D points from the multiple all straight-line distances. In the above solution, before classifying the 3D points to be classified, the standard 3D points corresponding to the standard action can be classified first and the standard distance between the standard 3D points can be calculated, so that the 3D points to be classified can be classified based on the second classification result and the standard distance.

[0007] In an alternative embodiment, classifying the 3D points to be classified according to the second classification result corresponding to a set of standard 3D points corresponding to a standard action and the standard distance, and in a preset order, includes: for a current 3D point among the 3D points to be classified, searching for at least one set of first 3D points in the current 3D point according to multiple one-way closed straight-line distances; wherein each set of first 3D points satisfies multiple one-way closed straight-line distances corresponding to a first category, and the first category is a classification category in the second classification result; screening a set of second 3D points from multiple sets of first 3D points according to multiple all straight-line distances, and classifying the second 3D points as the first category; wherein the second 3D points satisfy multiple all straight-line distances corresponding to the first category; deleting the second 3D points in the current 3D point to obtain a new current 3D point, and updating the first category in a preset order. In the above solution, some first 3D points that may belong to the first category can be screened out from multiple current 3D points based on the one-way closed straight-line distance first, and the second 3D points belonging to the first category can be determined based on the all straight-line distance, so as to classify the 3D points to be classified. Since the 3D points to be classified in each category are classified in a preset order, the global classification of each 3D point to be classified is avoided, and thus the classification efficiency can be improved.

[0008] In an alternative embodiment, screening a set of second 3D points from multiple sets of first 3D points according to multiple all straight-line distances includes: screening multiple sets of third 3D points from multiple sets of first 3D points according to multiple all straight-line distances; screening the second 3D points from multiple sets of third 3D points based on the nearest principle. In the above solution, when the second 3D points belonging to the first category cannot be determined based on the all straight-line distance, the second 3D points belonging to the first category can be determined based on the nearest principle. Since the nearest principle is introduced, the classification accuracy can be improved.

[0009] In an alternative embodiment, the preset order is as follows: head, chest, waist, left shoulder, left upper arm, left forearm, left hand, right shoulder, right upper arm, right forearm, right hand, left thigh, left calf, left foot, right thigh, right calf, right foot. In the above solution, classification starts with the categories having a larger number of 3D points and then proceeds to the categories with a smaller number of 3D points. Since the number of current 3D points has decreased by a certain amount when classifying the categories with a smaller number of 3D points, the classification efficiency can be improved. At the same time, since the larger area is determined first and then the smaller area, the classification accuracy can be enhanced.

[0010] In a second aspect, an embodiment of the present application provides a skeleton matching device, including: a first classification module configured to classify a set of 3D points to be classified corresponding to an action to be classified, obtaining a first classification result; and a matching module configured to sequentially match the 3D points to be classified in each category in the first classification result with the initial skeleton, obtaining a standard skeleton corresponding to the action to be classified. In the above solution, before matching the 3D points and the initial skeleton, the 3D points can be classified first, and then the 3D points in each category are matched separately. Since the 3D points in each category are matched separately, global matching of each 3D point is avoided, so the matching efficiency can be improved. At the same time, during the process of matching the 3D points in each category, there will be no situation of incorrect matching, thus the matching accuracy can be enhanced.

[0011] In an alternative embodiment, the first classification module is specifically configured to: classify the 3D points to be classified according to a second classification result and a standard distance corresponding to a set of standard 3D points corresponding to a standard action, and in a preset order. In the above solution, the 3D points to be classified can be classified in a preset order based on the second classification result and the standard distance corresponding to the standard 3D points. Since the 3D points to be classified in each category are classified in a preset order, global classification of each 3D point to be classified is avoided, so the classification efficiency can be improved.

[0012] In an alternative embodiment, the standard distances include all straight-line distances and one-way closed straight-line distances; the skeleton matching device further includes: a second classification module, configured to classify the standard 3D points by using a classification algorithm to obtain the second classification result; a calculation module, configured to calculate, for multiple standard 3D points in each category in the second classification result, multiple all straight-line distances between the multiple standard 3D points, and determine multiple one-way closed straight-line distances between the multiple standard 3D points from the multiple all straight-line distances. In the above solution, before classifying the 3D points to be classified, the standard 3D points corresponding to the standard actions can be classified first and the standard distances between the standard 3D points can be calculated, so that the 3D points to be classified can be classified based on the second classification result and the standard distances.

[0013] In an alternative embodiment, the first classification module is specifically configured to: for the current 3D point in the 3D points to be classified, search for at least one set of first 3D points in the current 3D point according to multiple one-way closed straight-line distances; wherein each set of first 3D points satisfies multiple one-way closed straight-line distances corresponding to the first category, and the first category is a classification category in the second classification result; screen a set of second 3D points from multiple sets of first 3D points according to multiple all straight-line distances, and classify the second 3D points into the first category; wherein the second 3D points satisfy multiple all straight-line distances corresponding to the first category; delete the second 3D points in the current 3D point to obtain a new current 3D point, and update the first category in a preset order. In the above solution, some first 3D points that may belong to the first category can be screened out from multiple current 3D points based on the one-way closed straight-line distances, and the second 3D points belonging to the first category can be determined based on the all straight-line distances, so as to classify the 3D points to be classified. Since the 3D points to be classified in each category are classified in a preset order, the global classification of each 3D point to be classified is avoided, and thus the classification efficiency can be improved.

[0014] In an alternative embodiment, the first classification module is specifically configured to: screen multiple sets of third 3D points from multiple sets of first 3D points according to multiple all straight-line distances; screen the second 3D points from multiple sets of third 3D points based on the nearest principle. In the above solution, when the second 3D points belonging to the first category cannot be determined based on the all straight-line distances, the second 3D points belonging to the first category can be determined based on the nearest principle. Since the nearest principle is introduced, the classification accuracy can be improved.

[0015] In an alternative embodiment, the preset order is as follows: head, chest, waist, left shoulder, left upper arm, left lower arm, left hand, right shoulder, right upper arm, right lower arm, right hand, left thigh, left lower leg, left foot, right thigh, right lower leg, right foot. In the above solution, classification starts with the category having a larger number of 3D points and then proceeds to the category having a smaller number of 3D points. Since the number of current 3D points has decreased by a certain amount when classifying the category with a smaller number of 3D points, the classification efficiency can be improved. At the same time, since the larger area is determined first and then the smaller area is determined, the classification accuracy can be improved.

[0016] In a third aspect, an embodiment of the present application provides a computer program product, including computer program instructions which, when read and run by a processor, execute the skeleton matching method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus; the processor and the memory communicate with each other through the bus; the memory stores computer program instructions executable by the processor, and the processor can execute the skeleton matching method as described in the first aspect by invoking the computer program instructions.

[0018] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer program instructions which, when run by a computer, cause the computer to execute the skeleton matching method as described in the first aspect.

[0019] To make the above objects, features, and advantages of the present application more obvious and understandable, specific embodiments of the present application are hereinafter given, and in conjunction with the accompanying drawings, the following detailed description is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of a skeleton matching method provided by an embodiment of the present application;

[0022] Figure 2 It is a structural block diagram of a skeleton matching device provided by an embodiment of the present application;

[0023] Figure 3A structural block diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0025] Before introducing the skeleton matching method provided by the embodiments of the present application, a brief introduction to the application scenario of the skeleton matching method will be given first.

[0026] The skeleton matching method provided by the embodiments of the present application is mainly applied to action recognition. In one application scenario, an actor can wear a motion capture suit and attach multiple reflective points (i.e., mark points) on the motion capture suit, and then use multiple infrared cameras to collect images of the actor from multiple different angles. During the image collection process, the actor can perform multiple actions. For each action, multiple action images captured by multiple infrared cameras can be obtained. Each action image includes multiple 2D points. Reconstructing the 2D points in multiple action images can obtain a set of 3D points corresponding to the action. Among them, in the skeleton matching method provided by the embodiments of the present application, the above-mentioned set of 3D points needs to be classified.

[0027] It can be understood that due to the occlusion problem during image collection, the number of 2D points included in each action image is less than the total number of reflective points, and the number of a set of 3D points reconstructed based on the above 2D points is less than or equal to the total number of reflective points. For example, assume that the total number of reflective points is 53, the number of 2D points in multiple action images is distributed between 40 and 50, and the number of reconstructed 3D points is 52.

[0028] Among them, during the collection process, as an implementation manner, any one of the actions can be determined as the standard action, and other actions can be determined as the actions to be classified; as another implementation manner, any one of the actions can also be first determined as the standard action, and during the process of image collection and classification of 3D points, the standard action can be updated.

[0029] It can be understood that since the actions to be classified need to be classified based on the standard action for 3D points, the number of standard 3D points corresponding to the standard action should be equal to the total number of reflective points, otherwise the standard 3D points need to be re-determined.

[0030] Regarding the number of to-be-classified 3D points corresponding to the to-be-classified action, as an implementation, the number of to-be-classified 3D points may not be limited, that is, regardless of the number of to-be-classified 3D points, this group of to-be-classified 3D points will be classified; as another implementation, the number of to-be-classified 3D points can be limited, that is, when the number of to-be-classified 3D points is greater than a preset value, this group of to-be-classified 3D points will be classified, otherwise this group of to-be-classified 3D points will not be classified.

[0031] Taking the total number of reflective points as 53 as an example, the number of standard 3D points should be 53, and the number of to-be-classified 3D points should be greater than 43.

[0032] Please refer to Figure 1 , Figure 1 which is a flowchart of a skeleton matching method provided by an embodiment of the present application. The skeleton matching method may include the following contents:

[0033] Step S101: Classify a group of to-be-classified 3D points corresponding to the to-be-classified action to obtain a first classification result.

[0034] Step S102: Sequentially match the to-be-classified 3D points in each category in the first classification result with the initial skeleton to obtain a standard skeleton corresponding to the to-be-classified action.

[0035] Specifically, before matching a group of to-be-classified 3D points corresponding to the to-be-classified action with the initial skeleton, the to-be-classified 3D points can be classified first to obtain a first classification result; then, the to-be-classified 3D points in each category in the first classification result are sequentially matched with the initial skeleton to obtain a standard skeleton. Among them, the completed standard skeleton should have an action consistent with the to-be-classified action.

[0036] For example, assume that in step S101, the to-be-classified 3D points are divided into two categories: torso and limbs. Then, in step S102, the to-be-classified 3D points in the torso category can be first matched with the initial skeleton, and after the matching is completed, the to-be-classified 3D points in the limbs category are matched with the initial skeleton, and finally a completed standard skeleton is obtained.

[0037] It can be understood that the above classification categories for classifying the to-be-classified 3D points are only an example provided by the embodiment of the present application, and those skilled in the art can adjust the specific classification method according to the actual situation. For example: the to-be-classified 3D points can be divided into four categories: head, chest, waist, and limbs, etc., and the embodiment of the present application does not make specific limitations on this.

[0038] It should be noted that the specific classification method in step S101 of the embodiments of the present application is not specifically limited either. Those skilled in the art can adopt a suitable classification method according to the actual situation. For example, a classification model can be used to classify the 3D points to be classified; or, the 3D points to be classified can be classified by means of manual annotation, etc.

[0039] Before performing the above step S101, the skeleton matching method provided by the embodiments of the present application may further include the step of obtaining the 3D points to be classified, and there are various implementation manners for the step of obtaining the 3D points to be classified. As an implementation manner, the 3D points to be classified stored in the cloud or locally can be read; as another implementation manner, the 3D points to be classified sent by other devices can be received; as still another implementation manner, the 3D points to be classified can be reconstructed based on the action images collected by the infrared camera.

[0040] In the above solution, before matching the 3D points and the initial skeleton, the 3D points can be classified first, and then the 3D points in each category are matched separately. Since the 3D points in each category are matched separately, the global matching of each 3D point is avoided. Therefore, the matching efficiency can be improved; at the same time, during the process of matching the 3D points in each category, there will be no situation of matching errors. Therefore, the matching accuracy can be improved.

[0041] Further, a specific implementation manner of classifying the 3D points to be classified provided by the embodiments of the present application will be introduced in detail below. In this implementation manner, the above step S101 may specifically include the following content:

[0042] Classify the 3D points to be classified according to the second classification result and the standard distance corresponding to a set of standard 3D points corresponding to the standard action, and in a preset order.

[0043] Specifically, the 3D points to be classified can be classified based on the standard 3D points in the standard action. Among them, in the specific implementation process of classification, the 3D points to be classified can be classified based on the second classification result and the standard distance of the standard 3D points.

[0044] The second classification result is the result of classifying the standard 3D points. There are also multiple ways of classification here. For example, a classification model can be used to classify the standard 3D points; or, the standard 3D points can be classified by manual annotation, etc. The standard distance is the distance between multiple standard 3D points. Depending on the application scenario, there are also multiple implementation methods. For example, the standard distance can include all straight-line distances, that is, all straight-line distances between every two of the multiple standard 3D points; or, the standard distance can include one-way closed straight-line distances, that is, when the multiple standard 3D points form a closed figure, the straight-line distances of all sides of the figure, etc.

[0045] On the basis of classifying the 3D points to be classified based on the second classification result of the standard 3D points and the standard distance, the 3D points to be classified can also be classified in a preset order. Among them, the embodiments of the present application do not specifically limit the specific implementation method of the preset order. For one classification category, there can be multiple implementation methods for the preset order. For example: when the classification category includes the torso and limbs, the preset order can be the torso, limbs or the limbs, torso; when the classification category includes the head, chest, waist and limbs, the preset order can be the head, chest, waist, limbs or the head, limbs, chest, waist, etc.

[0046] Taking the classification category including the head, chest, waist and limbs and the preset order being the head, chest, waist, limbs as an example, the specific classification steps are as follows: separating the 3D points of the head category from the 3D points to be classified according to the second classification result and the standard distance; separating the 3D points of the chest category from the 3D points to be classified except those of the head category according to the second classification result and the standard distance; separating the 3D points of the waist category from the 3D points to be classified except those of the head category and the chest category according to the second classification result and the standard distance; separating the 3D points of the limb category from the 3D points to be classified except those of the head category, chest category and waist category according to the second classification result and the standard distance; completing the classification of all 3D points to be classified, and the 3D points to be classified are divided into four categories: head, chest, waist, and limbs.

[0047] It should be noted that the specific implementation method of classifying the 3D points to be classified according to the second classification result and the standard distance will be described in detail in the subsequent embodiments and will not be introduced here for the time being.

[0048] Before performing the above steps, the skeleton matching method provided by the embodiments of the present application can also include the step of obtaining standard 3D points. Among them, similar to the method of obtaining the 3D points to be classified, there are also multiple implementation methods for the step of obtaining standard 3D points. As an implementation method, the standard 3D points stored can be read from the cloud or locally; as another implementation method, the standard 3D points sent by other devices can be received; as yet another implementation method, the standard 3D points can be reconstructed based on the action images collected by the infrared camera.

[0049] In addition, during the process of executing the skeleton matching method multiple times, as an implementation, the standard 3D points can be constant. For example, the first action is used as the standard action, and subsequent actions are classified based on this for the 3D points to be classified.

[0050] As another implementation, the standard action can be updated in real time. For example, first, the first action is used as the standard action, and based on the standard action, the 3D points to be classified for the second and third actions are classified. Then, the third action is used as the standard action, and based on the standard action, the 3D points to be classified for the fourth and fifth actions are classified, and so on.

[0051] For another example, first, the first action is used as the standard action, and based on the standard action, the 3D points to be classified for the second action are classified. Then, it is determined whether the second action meets the preset rule; if it meets, the second action is used as the standard action, and based on the standard action, the 3D points to be classified for the third action are classified; if it does not meet, the first action is still used as the standard action, and based on the standard action, the 3D points to be classified for the third action are classified, and so on.

[0052] Among them, the embodiments of the present application do not specifically limit the specific implementation manner of the above preset rule. For example, the preset rule can be that the number of 3D points to be classified corresponding to the third action is greater than a preset value; or, the preset rule can be that the number of 3D points to be classified in the torso category corresponding to the third action is greater than another preset value, etc.

[0053] In the above solution, the 3D points to be classified can be classified in a preset order based on the second classification result and the standard distance corresponding to the standard 3D points. Since the 3D points to be classified in each category are classified in a preset order, the global classification of each 3D point to be classified is avoided, and thus, the classification efficiency can be improved.

[0054] Further, before classifying the 3D points to be classified according to the second classification result and the standard distance corresponding to a set of standard 3D points corresponding to the standard action, the skeleton matching method provided by the embodiments of the present application may further include the following content:

[0055] Step 1), classifying the standard 3D points using a classification algorithm to obtain a second classification result.

[0056] Step 2), for multiple standard 3D points in each category in the second classification result, calculate multiple all straight-line distances between the multiple standard 3D points, and determine multiple one-way closed straight-line distances between the multiple standard 3D points from the multiple all straight-line distances.

[0057] Specifically, in the embodiments of the present application, a classification algorithm is used to classify standard 3D points. Among them, the embodiments of the present application do not specifically limit the above classification algorithm. For example, the classification algorithm can adopt the Self-Organizing Migrating Algorithm (SOMA).

[0058] In addition, in the embodiments of the present application, the standard distance includes all straight-line distances and one-way closed straight-line distances. Among them, the above all straight-line distances and one-way closed straight-line distances are both for the standard 3D points in one category.

[0059] Taking the classification categories including the torso and limbs and the number of standard 3D points being 53 as an example, assuming the second classification result is: among them, 20 standard 3D points belong to the torso category and 33 standard points belong to the limbs category. Then, in the above step 2), calculate all the straight-line distances between the 20 standard 3D points belonging to the torso category, that is, a total of 190 all straight-line distances, and then determine 20 one-way closed straight-line distances from the above 190 all straight-line distances that can make the above 20 standard 3D points form a closed figure; at the same time, calculate all the straight-line distances between the 33 standard 3D points belonging to the limbs category, that is, a total of 528 all straight-line distances, and then determine 33 one-way closed straight-line distances from the above 528 all straight-line distances that can make the above 33 standard 3D points form a closed figure.

[0060] In the above solution, before classifying the 3D points to be classified, the standard 3D points corresponding to the standard actions can be classified first and the standard distances between the standard 3D points can be calculated, so that the 3D points to be classified can be classified based on the second classification result and the standard distance.

[0061] Further, the following details the specific implementation manner of classifying the 3D points to be classified according to the second classification result and the standard distance. In this implementation manner, the steps of classifying the 3D points to be classified according to the second classification result and the standard distance corresponding to a group of standard 3D points corresponding to the standard action and in a preset order may specifically include the following contents:

[0062] Step 1), for the current 3D point among the 3D points to be classified, search for at least one group of first 3D points in the current 3D point according to multiple one-way closed straight-line distances.

[0063] Step 2), screen a group of second 3D points from multiple groups of first 3D points according to multiple all straight-line distances and classify the second 3D points into the first category.

[0064] Step 3), delete the second 3D points in the current 3D point to obtain a new current 3D point and update the first category in a preset order.

[0065] Specifically, the above steps 1)-3) are a cyclic process. First, all 3D points to be classified are the current 3D points. At this time, a set of second 3D points is screened according to multiple one-way closed straight-line distances and multiple total straight-line distances, and the second 3D points are classified into the first category in the preset order; then, the above second 3D points are deleted from the 3D points to be classified, and the remaining 3D points to be classified are the current 3D points. At this time, a set of second 3D points is screened according to multiple one-way closed straight-line distances and multiple total straight-line distances, and the second 3D points are classified into the second category in the preset order; and so on until all 3D points to be classified are classified.

[0066] Taking the cyclic process of classifying the second 3D points into the first category in the preset order as an example, the specific implementation manners of the above steps 1) and 2) are introduced in detail.

[0067] In the above step 1), multiple one-way closed straight-line distances between the standard 3D points corresponding to the first category in the second classification result are required. Based on the above multiple one-way closed straight-line distances, at least one set of first 3D points that satisfy the above multiple one-way closed straight-line distances can be searched from the current 3D points. Among them, each set of the searched first 3D points satisfies the multiple one-way closed straight-line distances corresponding to the first category.

[0068] Suppose the number of standard 3D points corresponding to the first category in the second classification result is 5, and there are 5 one-way closed straight-line distances between these 5 standard 3D points; based on these 5 one-way closed straight-line distances, 3 sets of first 3D points can be screened from the current 3D points, and each set of first 3D points includes 5 standard 3D points. Each side length of the closed figure formed by these 5 standard 3D points corresponds to and satisfies one of the 5 one-way closed straight-line distances.

[0069] It should be noted that a side length satisfying a one-way closed straight-line distance can mean that the length of the side is equal to the one-way closed straight-line distance; or it can also mean that the difference between the length of the side and the one-way closed straight-line distance is less than a preset threshold (for example: 1.5 cm, 2 cm, etc.).

[0070] Among them, in the above step 1), if more attention is paid to the search accuracy, the breadth-first algorithm can be used for searching; if more attention is paid to the search speed, the depth-first algorithm can be used for searching.

[0071] In the above step 2), multiple all straight-line distances between the standard 3D points corresponding to the first category in the second classification result are required, as well as at least one group of first 3D points selected in the above step 1). For one group of 3D points, the straight-line distances between every two of the 3D points in this group are compared with the multiple all straight-line distances between the standard 3D points corresponding to the first category to determine whether this group of 3D points meets the above multiple all straight-line distances. If it meets, this group of 3D points is determined as a group of second 3D points, and the category of this group of 3D points is the first category. Among them, the determined second 3D points meet the multiple all straight-line distances corresponding to the first category.

[0072] Suppose the number of standard 3D points corresponding to the first category in the second classification result is 5, and there are 10 all straight-line distances between these 5 standard 3D points. In step 1), 3 groups of first 3D points are selected from the current 3D points; based on these 10 all straight-line distances, the straight-line distances between every two of each group of first 3D points can be compared. If the straight-line distances between every two of this group of first 3D points all correspond to and meet one of the 10 all straight-line distances, then this group of first 3D points is determined as the first category.

[0073] It should be noted that the straight-line distance between every two meeting one all straight-line distance can mean that this straight-line distance is equal to this all straight-line distance; or, it can also mean that the difference between this straight-line distance and this all straight-line distance is less than a preset threshold (for example: 1.5 cm, 2 cm, etc.).

[0074] In this way, after the above step 1) and step 2), the to-be-classified 3D points belonging to the first category in the preset order among the current 3D points are screened out; then the current 3D points and the first category are updated, and the above steps 1)-2) are repeatedly executed to screen out the to-be-classified 3D points belonging to the second category in the preset order among the current 3D points; and so on, the to-be-classified 3D points are divided into all categories in the preset order.

[0075] In the above solution, some first 3D points that may belong to the first category can be screened out from multiple current 3D points based on the one-way closed straight-line distance, and the second 3D points belonging to the first category are determined based on the all straight-line distances, so as to realize the classification of the to-be-classified 3D points. Since the to-be-classified 3D points of each category are classified according to the preset order, therefore, the global classification of each to-be-classified 3D point is avoided, and thus, the classification efficiency can be improved.

[0076] Further, in step 2) of the above embodiment, there may be a situation where multiple sets of third 3D points are selected according to multiple total straight-line distances. Therefore, the following describes how to select a set of second 3D points from multiple sets of third 3D points. In this embodiment, the steps of selecting a set of second 3D points from multiple sets of first 3D points according to multiple total straight-line distances may specifically include the following content:

[0077] Step 1), select multiple sets of third 3D points from multiple sets of first 3D points according to multiple total straight-line distances.

[0078] Step 2), select the second 3D points from multiple sets of third 3D points based on the nearest principle.

[0079] Specifically, when multiple sets of third 3D points are selected according to multiple total straight-line distances, the second 3D points can be selected from multiple sets of third 3D points based on the nearest principle. Among them, the nearest principle means that for a certain category, the 3D points to be classified therein should be closer to the 3D points in a classified category. Therefore, the sum of the distances between each set of third 3D points and a 3D point in this category can be calculated, and the set of third 3D points with the smallest sum of the above distances is determined as the second 3D points.

[0080] For example, for the category of the left thigh, the 3D points to be classified therein should be closer to the 3D points in the category of the waist. Suppose that during the classification of the left thigh, two sets of third 3D points are selected. The sum of the distances between one set of third 3D points and a 3D point in the waist category is 100 cm, and the sum of the distances between the other set of third 3D points and this 3D point in the waist category is 90 cm. Therefore, the second set of third 3D points can be determined as the second 3D points.

[0081] It should be noted that the above nearest principle is only an example provided by the embodiments of the present application. According to different classification categories, there should be multiple implementation manners of the nearest principle, and the nearest principle may include multiple principles. The embodiments of the present application do not make specific limitations thereto.

[0082] In addition, in some application scenarios, after determining the category of the second 3D points, the EK algorithm can also be used to perform angular constraints on the spatial position information of the classification result of the second 3D points. For example: The 3D points in the thigh category cannot be close to the 3D points in the head category, etc.

[0083] In the above solution, when the second 3D points belonging to the first category cannot be determined based on the total straight-line distances, the second 3D points belonging to the first category can be determined based on the nearest principle. Since the nearest principle is introduced, the classification accuracy can be improved.

[0084] Further, in the embodiments of the present application, the above preset order may be: head, chest, waist, left shoulder, left upper arm, left forearm, left hand, right shoulder, right upper arm, right forearm, right hand, left thigh, left calf, left foot, right thigh, right calf, and right foot.

[0085] Specifically, under this preset order, the classification categories include: head, chest, waist, left shoulder, left upper arm, left forearm, left hand, right shoulder, right upper arm, right forearm, right hand, left thigh, left calf, left foot, right thigh, right calf, and right foot; the nearest principle includes: the left and right thighs follow the principle of being closest to the waist, the left and right calves follow the principle of being closest to the left and right thighs, the left and right feet follow the principle of being closest to the left and right calves, the left and right shoulders follow the principle of being closest to the chest, the left and right upper arms follow the principle of being closest to the shoulders, the left and right forearms follow the principle of being closest to the upper arms, and the left and right hands follow the principle of being closest to the left and right forearms.

[0086] In the above solution, classification starts with the categories having a larger number of 3D points and then proceeds to the categories with a smaller number of 3D points. Since when classifying the categories with a smaller number of 3D points, the number of current 3D points has already decreased by a certain amount, the classification efficiency can be improved; at the same time, since the larger area is determined first and then the smaller area, the classification accuracy can be improved.

[0087] Further, after the above step S101, the skeleton matching method provided by the embodiments of the present application may further include a step of testing the classification result.

[0088] Among them, during the process of testing the classification result, it is not necessary to test in the preset order. It is only necessary to test whether the 3D points in each category meet the 3D points in the same category of the corresponding standard action. If they meet, step S102 can be executed; if not, the 3D points to be classified can be re-classified. For example, when classifying the 3D points to be classified according to the second classification result and the standard distance, the preset threshold can be adjusted to re-classify the points to be classified.

[0089] Please refer to Figure 2 , Figure 2 which is a structural block diagram of a skeleton matching device provided by the embodiments of the present application. The skeleton matching device 200 includes: a first classification module 201 for classifying a set of 3D points to be classified corresponding to the action to be classified to obtain a first classification result; a matching module 202 for sequentially matching the 3D points to be classified in each category in the first classification result with the initial skeleton to obtain a standard skeleton corresponding to the action to be classified.

[0090] In the embodiment of the present application, before matching the 3D points and the initial skeleton, the 3D points can be classified first, and then the 3D points in each category are matched separately. Since the 3D points in each category are matched separately, the global matching of each 3D point is avoided. Therefore, the matching efficiency can be improved; at the same time, during the process of matching the 3D points in each category, there will be no situation of matching errors. Therefore, the matching accuracy can be improved.

[0091] Further, the first classification module 201 is specifically configured to: classify the 3D points to be classified according to the second classification result and the standard distance corresponding to a set of standard 3D points corresponding to the standard action, and in a preset order.

[0092] In the embodiment of the present application, the 3D points to be classified can be classified based on the second classification result corresponding to the standard 3D points and the standard distance, and in a preset order. Since the 3D points to be classified in each category are classified in a preset order, the global classification of each 3D point to be classified is avoided. Therefore, the classification efficiency can be improved.

[0093] Further, the standard distance includes all straight-line distances and one-way closed straight-line distances; the skeleton matching device 200 further includes: a second classification module, configured to classify the standard 3D points by using a classification algorithm to obtain the second classification result; a calculation module, configured to calculate multiple all straight-line distances between multiple standard 3D points in each category in the second classification result, and determine multiple one-way closed straight-line distances between the multiple standard 3D points from the multiple all straight-line distances.

[0094] In the embodiment of the present application, before classifying the 3D points to be classified, the standard 3D points corresponding to the standard action can be classified first and the standard distance between the standard 3D points can be calculated, so that the 3D points to be classified can be classified based on the second classification result and the standard distance.

[0095] Further, the first classification module 201 is specifically configured to: for the current 3D point in the 3D points to be classified, search for at least one set of first 3D points in the current 3D point according to multiple one-way closed straight-line distances; wherein each set of first 3D points satisfies multiple one-way closed straight-line distances corresponding to the first category, and the first category is a classification category in the second classification result; screen a set of second 3D points from multiple sets of first 3D points according to multiple all straight-line distances, and classify the second 3D points into the first category; wherein the second 3D point satisfies multiple all straight-line distances corresponding to the first category; delete the second 3D point in the current 3D point to obtain a new current 3D point, and update the first category in a preset order.

[0096] In an embodiment of the present application, first, based on the one-way closed straight-line distance, some first 3D points that may belong to the first category can be screened out from multiple current 3D points, and second 3D points belonging to the first category can be determined based on all the straight-line distances, so as to classify the 3D points to be classified. Since the 3D points to be classified for each category are classified in a preset order, therefore, the global classification of each 3D point to be classified is avoided, and thus, the classification efficiency can be improved.

[0097] Further, the first classification module 201 is specifically configured to: screen multiple groups of third 3D points from multiple groups of first 3D points according to multiple all straight-line distances; screen the second 3D points from multiple groups of third 3D points based on the nearest principle.

[0098] In an embodiment of the present application, when the second 3D points belonging to the first category still cannot be determined based on all the straight-line distances, the second 3D points belonging to the first category can be determined based on the nearest principle. Since the nearest principle is introduced, therefore, the classification accuracy can be improved.

[0099] Further, the preset order is: head, chest, waist, left shoulder, left upper arm, left lower arm, left hand, right shoulder, right upper arm, right lower arm, right hand, left thigh, left lower leg, left foot, right thigh, right lower leg, right foot.

[0100] In an embodiment of the present application, classification starts from the category with a larger number of 3D points first, and then classification starts for the category with a smaller number of 3D points. Since when classifying the category with a smaller number of 3D points, the number of current 3D points has decreased by a part, therefore, the classification efficiency can be improved; at the same time, since the area with a larger area is determined first and then the area with a smaller area is determined, therefore, the classification accuracy can be improved.

[0101] Please refer to Figure 3 , Figure 3 , which is a structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device 300 includes: at least one processor 301, at least one communication interface 302, at least one memory 303, and at least one communication bus 304. Among them, the communication bus 304 is used to realize the direct connection communication between these components, the communication interface 302 is used to communicate with other node devices for signaling or data, and the memory 303 stores machine-readable instructions executable by the processor 301. When the electronic device 300 runs, the processor 301 communicates with the memory 303 through the communication bus 304, and when the machine-readable instructions are called by the processor 301, the above-mentioned skeleton matching method is executed.

[0102] For example, the processor 301 in the embodiment of the present application can read a computer program from the memory 303 through the communication bus 304 and execute the computer program to implement the following method: classify a group of 3D points to be classified corresponding to the action to be classified to obtain a first classification result; sequentially match the 3D points to be classified in each category in the first classification result with the initial skeleton to obtain a standard skeleton corresponding to the action to be classified.

[0103] Among them, the processor 301 includes one or more, which can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 301 can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a microcontroller unit (Micro Controller Unit, MCU for short), a network processor (Network Processor, NP for short), or other conventional processors; it can also be a dedicated processor, including a neural network processor (Neural-network Processing Unit, NPU for short), a graphics processing unit (Graphics Processing Unit, GPU for short), a digital signal processor (Digital Signal Processor, DSP for short), an application specific integrated circuit (Application Specific Integrated Circuits, ASIC for short), a field programmable gate array (Field Programmable Gate Array, FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. And when there are multiple processor 301s, a part of them can be general-purpose processors and another part can be dedicated processors.

[0104] The memory 303 includes one or more, which can be, but is not limited to, a random access memory (Random Access Memory, RAM for short), a read only memory (Read Only Memory, ROM for short), a programmable read-only memory (Programmable Read-Only Memory, PROM for short), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable programmable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM for short), etc.

[0105] It can be understood that Figure 3The structure shown is only illustrative, and the electronic device 300 may also include more or fewer components than those Figure 3 shown in, or have a configuration different from that Figure 3 shown. Figure 3 Each component shown in can be implemented by hardware, software, or a combination thereof. In the embodiments of the present application, the electronic device 300 may be, but is not limited to, physical devices such as desktop computers, laptop computers, smartphones, smart wearable devices, vehicle-mounted devices, etc., and may also be virtual devices such as virtual machines. In addition, the electronic device 300 is not necessarily a single device, and may also be a combination of multiple devices, such as a server cluster, and so on.

[0106] The embodiments of the present application also provide a computer program product, including a computer program stored on a computer-readable storage medium. The computer program includes computer program instructions. When the computer program instructions are executed by a computer, the computer can execute the steps of the skeleton matching method in the above embodiments, for example, including: Step S101: Classify a group of to-be-classified 3D points corresponding to the to-be-classified action to obtain a first classification result. Step S102: Sequentially match the to-be-classified 3D points in each category in the first classification result with the initial skeleton to obtain a standard skeleton corresponding to the to-be-classified action.

[0107] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, and the indirect coupling or communication connection of the devices or units may be in an electrical, mechanical, or other form.

[0108] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] Furthermore, in each embodiment of the present application, the various functional modules may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0110] It should be noted that if a function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0111] In this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0112] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A skeleton matching method, characterized in that, it includes: classifying a group of to-be-classified 3D points corresponding to the action to be classified to obtain a first classification result; successively matching the to-be-classified 3D points in each category in the first classification result with an initial skeleton to obtain a standard skeleton corresponding to the action to be classified; The classifying the group of to-be-classified 3D points corresponding to the action to be classified includes: classifying the to-be-classified 3D points according to a second classification result and a standard distance corresponding to a group of standard 3D points corresponding to a standard action, and in a preset order; The standard distance includes all straight-line distances and one-way closed straight-line distances; Before classifying the to-be-classified 3D points according to the second classification result and the standard distance corresponding to a group of standard 3D points corresponding to a standard action, the method further includes: using a classification algorithm to classify the standard 3D points to obtain the second classification result; for multiple standard 3D points in each category in the second classification result, calculating multiple all straight-line distances between the multiple standard 3D points, and determining multiple one-way closed straight-line distances between the multiple standard 3D points from the multiple all straight-line distances; The classifying the to-be-classified 3D points according to the second classification result and the standard distance corresponding to a group of standard 3D points corresponding to a standard action, and in a preset order, includes: for the current 3D point among the to-be-classified 3D points, searching for at least one group of first 3D points in the current 3D point according to multiple one-way closed straight-line distances; wherein, each group of first 3D points satisfies multiple one-way closed straight-line distances corresponding to a first category, and the first category is a classification category in the second classification result; screening a group of second 3D points from multiple groups of first 3D points according to multiple all straight-line distances, and classifying the second 3D points as the first category; wherein, the second 3D point satisfies multiple all straight-line distances corresponding to the first category; deleting the second 3D point in the current 3D point to obtain a new current 3D point, and updating the first category in a preset order.

2. The skeleton matching method according to claim 1, characterized in that, The screening a group of second 3D points from multiple groups of first 3D points according to multiple all straight-line distances includes: screening multiple groups of third 3D points from multiple groups of first 3D points according to multiple all straight-line distances; screening the second 3D point from multiple groups of third 3D points based on the nearest principle.

3. The skeleton matching method according to claim 1 or 2, characterized in that, The preset order is: head, chest, waist, left shoulder, left upper arm, left lower arm, left hand, right shoulder, right upper arm, right lower arm, right hand, left thigh, left lower leg, left foot, right thigh, right lower leg, right foot.

4. A skeleton matching device, characterized in that, it includes: a first classification module for classifying a group of to-be-classified 3D points corresponding to the action to be classified to obtain a first classification result; a matching module for successively matching the to-be-classified 3D points in each category in the first classification result with an initial skeleton to obtain a standard skeleton corresponding to the action to be classified; The first classification module is specifically configured to: Classify the 3D points to be classified according to the second classification result corresponding to a set of standard 3D points corresponding to the standard action and the standard distance, and in a preset order; The standard distance includes all straight-line distances and one-way closed straight-line distances; The skeleton matching device further includes: A second classification module, configured to classify the standard 3D points by using a classification algorithm to obtain the second classification result; A calculation module, configured to calculate multiple all straight-line distances between the multiple standard 3D points in each category in the second classification result, and determine multiple one-way closed straight-line distances between the multiple standard 3D points from the multiple all straight-line distances; The first classification module is specifically configured to: For the current 3D point in the 3D points to be classified, search for at least one set of first 3D points in the current 3D point according to the multiple one-way closed straight-line distances; wherein, each set of first 3D points satisfies the multiple one-way closed straight-line distances corresponding to the first category, and the first category is a classification category in the second classification result; Screen a set of second 3D points from the multiple sets of first 3D points according to the multiple all straight-line distances, and classify the second 3D points as the first category; wherein, the second 3D points satisfy the multiple all straight-line distances corresponding to the first category; Delete the second 3D points in the current 3D point to obtain a new current 3D point, and update the first category in a preset order.

5. A computer program product, Characterized in that It includes computer program instructions, and when the computer program instructions are read and run by a processor, the method described in any one of claims 1-3 is executed.

6. An electronic device, Characterized in that It includes: A processor, a memory and a bus; The processor and the memory complete communication with each other through the bus; The memory stores computer program instructions executable by the processor, and the processor can execute the method described in any one of claims 1-3 by calling the computer program instructions.

7. A computer-readable storage medium, Characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are run by a computer, the computer is caused to execute the method described in any one of claims 1-3.

Images