Limb movement action equipment recognition method and system based on artificial intelligence

Abstract

The invention provides a limb movement equipment recognition method and system based on artificial intelligence, and relates to the technical field of data processing, and the method comprises the steps: obtaining a to-be-recognized angle group sequence JX and a to-be-recognized reference movement identification sequence DX corresponding to a to-be-recognized target in a target time window; obtaining a first feature list and a second feature list according to the JX, the DX, a preset reference angle group sequence JB, a preset reference action identification sequence DB and a hidden Markov model; obtaining a first matching degree according to the first feature list and the second feature list; obtaining a second matching degree according to the JX and the JB; obtaining a limb movement recognition result corresponding to the to-be-recognized target according to the first matching degree and the second matching degree; wherein the limb movement recognition result is used for describing the execution accuracy of the to-be-recognized target on the target limb movement set. According to the invention, an accurate and objective identification result can be obtained.

Description

Background Art

[0002] With the continuous improvement of people's attention to health and fitness, exercise training methods based on standard movements such as dance or gymnastics are becoming increasingly popular. In such exercise scenarios, the accurate recognition and analysis of movements play a crucial role in training effect evaluation, correction of incorrect movements, and improvement of sports skills.

[0003] Currently, when carrying out exercises according to dance or gymnastics standard movements, image analysis technology is mainly used to identify and analyze the movements of exercisers. This technology uses a camera to collect the limb movement images of exercisers, and then uses image processing algorithms to extract and analyze information such as human body contours and joint points in the images, and then compares them with pre-stored standard movement images or models to judge the accuracy and standardization of the movements.

[0004] However, this traditional image analysis technology has exposed many serious defects in practical applications. First of all, the image itself has limitations. It presents two-dimensional plane information and cannot obtain depth information, which makes it difficult to accurately know the true position and movement trajectory of the limbs in three-dimensional space when judging movements. For example, in some dance movements, the amplitude of the body leaning forward or backward and the stretching degree of the limbs in the front-back direction are difficult to accurately evaluate only relying on two-dimensional images, which is likely to lead to misjudgment of the movement accuracy.

[0005] Secondly, due to significant differences in physiological characteristics such as body shape and height among different individuals, image analysis is difficult to adapt to this diversity, resulting in inaccurate movement recognition. For example, when a tall person and a short person complete the same dance movement, although the standardization of the movement is the same, their visual performances in the image will be different due to different body proportions, which may cause the image analysis system to misjudge the visual difference caused by body shape differences as an unstandard movement.

[0006] In summary, the existing image analysis-based movement recognition and analysis technology can no longer meet the requirements of accurate movement recognition and comprehensive analysis in dance or gymnastics exercise training. There is an urgent need for a more comprehensive, accurate and reliable movement recognition and analysis method and system to improve the scientificity and effectiveness of exercise training and promote the technological development and progress of sports fields such as dance and gymnastics. Summary of the Invention

[0007] To solve the above technical problems, the present application provides an artificial intelligence-based method for recognizing limb movement actions, which at least partially solves the problems existing in the prior art.

[0008] In the first aspect of the present application, there is provided an artificial intelligence-based method for recognizing limb movement actions, and the method includes: Obtain the sequence JX of angle groups to be recognized and the sequence DX of reference action identifiers corresponding to the target to be recognized within the target time window; wherein, the start time of the target time window is the time when the target limb movement set starts to be executed; the end time of the target time window is the time when the execution of the target limb movement set ends; According to the sequence JX of angle groups to be recognized, the sequence DX of reference action identifiers to be recognized, the preset sequence JB of reference angle groups, the preset sequence DB of reference action identifiers, and the hidden Markov model, obtain a first feature list and a second feature list; wherein the first feature list is used to describe the motion features obtained by JX and DX according to the hidden Markov model; the second feature list is used to describe the motion features obtained by JB and DB according to the hidden Markov model; Obtain a first matching degree according to the first feature list and the second feature list; Obtain a second matching degree according to the sequence JX of angle groups to be recognized and the preset sequence JB of reference angle groups; Obtain the limb movement recognition result corresponding to the target to be recognized according to the above first matching degree and the above second matching degree; wherein, the above limb movement recognition result is used to describe the execution accuracy of the target to be recognized for the target limb movement set.

[0009] In the second aspect of the present application, there is provided an artificial intelligence-based limb movement action recognition system, and the above system includes: A first acquisition unit, configured to acquire the sequence JX of angle groups to be recognized and the sequence DX of reference action identifiers corresponding to the target to be recognized within the target time window; wherein, the start time of the target time window is the time when the target limb movement set starts to be executed; the end time of the target time window is the time when the execution of the target limb movement set ends; A second acquisition unit, configured to obtain a first feature list and a second feature list according to the sequence JX of angle groups to be recognized, the sequence DX of reference action identifiers to be recognized, the preset sequence JB of reference angle groups, the preset sequence DB of reference action identifiers, and the hidden Markov model; wherein the first feature list is used to describe the motion features obtained by JX and DX according to the hidden Markov model; the second feature list is used to describe the motion features obtained by JB and DB according to the hidden Markov model; A first matching unit, configured to obtain a first matching degree according to the first feature list and the second feature list; A second matching unit, configured to obtain a second matching degree according to the sequence JX of angle groups to be recognized and the preset sequence JB of reference angle groups; A recognition unit, configured to obtain the limb movement recognition result corresponding to the target to be recognized according to the above first matching degree and the above second matching degree; wherein, the above limb movement recognition result is used to describe the execution accuracy of the target to be recognized for the target limb movement set.

[0010] This application has at least the following beneficial effects: The method for identifying limb movement actions of a device based on artificial intelligence provided in this application determines the limb movement recognition result according to the first matching degree and the second matching degree corresponding to the target to be recognized. Among them, the first matching degree considers the matching degree between the sequence of action identification marks to be recognized corresponding to the target to be recognized and the corresponding sequence of reference action identification marks, and the second matching degree considers the matching degree between the movement amplitude of each limb movement corresponding to the target to be recognized and the movement amplitude of the reference limb movement. Here, the matching degree between the sequence of action identification marks to be recognized corresponding to the target to be recognized and the corresponding sequence of reference action identification marks is an evaluation of whether a certain action has been performed on the target to be recognized. That is, in this embodiment, the final recognition result is obtained by considering both the movement amplitude itself and whether the overall action has been executed. For the target to be recognized who is relatively proficient but whose actions are not standard enough, and for the target to be recognized who is relatively unfamiliar with the actions and has some actions not followed and not performed, this recognition result can obtain a relatively accurate and objective recognition result. Description of the Drawings

[0011] In order to more clearly illustrate the technical solutions in the embodiments of this application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a flowchart of the method for identifying limb movement actions of a device based on artificial intelligence provided in the embodiments of this application; Figure 2 It is a structural block diagram of the system for identifying limb movement actions of a device based on artificial intelligence provided in the embodiments of this application. Detailed Embodiments

[0013] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0014] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0015] It should be noted that the following description relates to various aspects of embodiments within the scope of the appended claims. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement a device and / or practice a method. Additionally, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.

[0016] Please refer to Figure 1 As shown, an embodiment of this application provides a method for identifying limb movement actions of a device based on artificial intelligence. The above method includes the following steps: Step S100, obtaining a sequence JX of angles to be recognized and a sequence DX of reference action identifiers to be recognized corresponding to the target to be recognized within a target time window; wherein, the start time of the target time window is the time when the target limb movement set starts to be executed; the end time of the target time window is the time when the target limb movement set ends to be executed.

[0017] Specifically, the target limb movement set can be a dance or a broadcast gymnastics, etc. with standard decomposed actions. The target to be recognized is the user who executes the target limb movement set. The target time window is the time period corresponding to the target to be recognized executing the target limb movement set.

[0018] Step S200: Obtain a first feature list and a second feature list based on the sequence of angles to be recognized JX, the sequence of reference action identifiers to be recognized DX, the preset sequence of reference angle groups JB, the preset sequence of reference action identifiers DB, and the hidden Markov model. The first feature list is used to describe the motion features obtained from JX and DX according to the hidden Markov model, and the second feature list is used to describe the motion features obtained from JB and DB according to the hidden Markov model.

[0019] Specifically, a hidden Markov model (HMM) is a statistical model used to describe a Markov process with hidden unknown parameters. It consists of a hidden Markov chain (i.e., the sequence of hidden states) and a visible sequence related to the non-visible states. The sequence of hidden states is a sequence of states that cannot be directly observed, and there are transition probabilities between these states. The generation of the visible sequence is related to the sequence of hidden states, and each non-visible state in the sequence of hidden states has a probability distribution for generating visible values.

[0020] Step S300: Obtain a first matching degree based on the first feature list and the second feature list.

[0021] Specifically, the first matching degree is used to describe the matching degree between the motion features corresponding to the target to be recognized and the reference motion features corresponding to the target limb motion set.

[0022] Step S400: Obtain a second matching degree based on the sequence of angles to be recognized JX and the preset sequence of reference angle groups JB.

[0023] Specifically, the second matching degree is used to describe the matching degree between the limb action angles corresponding to the target to be recognized and the reference limb action angles.

[0024] Step S500: Obtain the recognition result of the limb motion corresponding to the target to be recognized based on the above first matching degree and the above second matching degree. The recognition result of the limb motion is used to describe the execution accuracy of the target to be recognized for the target limb motion set.

[0025] Specifically, in this embodiment, the limb movement recognition result is determined based on the first matching degree and the second matching degree corresponding to the target to be recognized. The first matching degree takes into account the matching degree between the movement amplitude of each limb movement corresponding to the target to be recognized and the movement amplitude of the reference limb movement. The second matching degree takes into account the matching degree between the sequence of to-be-recognized reference action identifiers corresponding to the target to be recognized and the corresponding sequence of reference action identifiers. Here, the matching degree between the sequence of to-be-recognized reference action identifiers corresponding to the target to be recognized and the corresponding sequence of reference action identifiers is an evaluation of whether the target to be recognized has performed a certain action. That is, in this embodiment, the final recognition result is obtained by considering both the movement amplitude itself and whether the overall movement has been performed. This recognition result can obtain a relatively accurate and objective recognition result for the target to be recognized who is relatively proficient but has non-standard movements, as well as for the target to be recognized who is relatively unfamiliar with the movements and has missed some movements due to being unable to keep up.

[0026] In an exemplary embodiment of the present application, step S100 includes: Step S110, obtaining a sequence of to-be-recognized angle groups JX = (JX1, JX2,..., JX i ,..., JX n ) corresponding to the target to be recognized within the target time window; i = 1, 2,..., n; where n is the number of reference sub-limb movements obtained by decomposing the target limb movement set; JX i is the angle group corresponding to the target to be recognized when performing the i-th reference sub-limb movement; JX i =(JX i，1 , JX i，2 ,..., JX i，a ,..., JX i,f(i) ); a = 1, 2,..., f(i); f(i) is the number of key limb angles included in the i-th reference sub-limb movement; JX i，a is the a-th key limb angle corresponding to the target to be recognized when performing the i-th reference sub-limb movement; the above key limb angles are used to describe the execution accuracy of the target to be recognized for the reference sub-limb movement.

[0027] Specifically, the target limb movement set can be divided into multiple reference sub-limb movements. Each reference sub-action has corresponding multiple key limb angles. Here, the key limb angles are used to describe the execution accuracy of the target to be recognized for the reference sub-limb movement.

[0028] JX i is determined according to the following steps: Step S111: Obtain the three-dimensional space coordinate system corresponding to the target to be recognized. Among them, the relationship between the origin in the three-dimensional space coordinate system corresponding to the target to be recognized and the target to be recognized is the same as the relationship between the origin in the reference three-dimensional space coordinate system corresponding to the target limb movement set and the reference target.

[0029] Specifically, the reference target can be a real person. In this case, the target limb movement set is obtained by recording the corresponding reference target. The reference target can also be a human body model, which is used to dynamically demonstrate each reference sub-limb movement.

[0030] As an example: When the origin in the reference three-dimensional space coordinate system corresponding to the target limb movement set is the center of the feet of the reference target, the origin in the three-dimensional space coordinate system corresponding to the target to be recognized is the center of the feet of the target to be recognized.

[0031] Step S112: According to the three-dimensional space coordinate system corresponding to the target to be recognized, obtain the list set GJ=(GJ1, GJ2,..., GJ i , …, GJ n ) of the key joint point coordinates corresponding to the target to be recognized within the target time window; where GJ i is the list of key joint point coordinates corresponding to the target to be recognized when performing the i-th reference sub-limb movement within the target time window; GJ i =(GJ i，1 , GJ i，2 , …, GJ i，b , …, GJ i，h(i) ); b = 1, 2, …, h(i); h(i) is the number of key joint points of the i-th reference sub-limb movement of the target limb movement set; GJ i，b is the coordinate of the b-th key joint point of the target to be recognized when performing the i-th reference sub-limb movement within the target time window.

[0032] Specifically, each reference sub-limb movement has a corresponding multiple key joint points. As an example: The key joint points can be the ankle, knee, hip joint, shoulder, elbow, wrist, etc. In this application, when recording the image of the target to be recognized while performing the target limb movement set, a binocular camera capable of obtaining the depth information of the image can be used. To obtain the three-dimensional coordinates corresponding to each key node when the target to be recognized performs each reference sub-limb movement.

[0033] Step S113: According to the three-dimensional space coordinate system corresponding to the target to be recognized and GJ, obtain (JX i，1 , JX i，2 , …, JX i，a , …, JX i,f(i) ); where each key limb angle is the included angle between the line connecting the corresponding two key joint points and the corresponding key coordinate axis.

[0034] Specifically, as an example, if the reference sub - limb movement is a side - kick, the corresponding reference key limb angles include: the angle between the line connecting the hip joint and the knee joint and the vertical axis is 45 degrees, and the angle between the line connecting the knee joint and the ankle joint and the vertical axis is 135 degrees. In this embodiment, when obtaining the target to be recognized during a side - kick, the angle between the line connecting the hip joint and the knee joint and the vertical axis, and the angle between the line connecting the knee joint and the ankle joint and the vertical axis are acquired.

[0035] Step S120: According to JX, obtain the sequence of reference action identifiers DX=(DX1, DX2, …, DX i , …, DX n ); where DX i is the reference action identifier determined according to JX i . The above - mentioned reference action identifier is the action identifier corresponding to the reference sub - limb movement obtained by decomposing the target limb movement set.

[0036] Specifically, each reference sub - limb movement has a corresponding reference execution time. After the expiration of this reference execution time, a reminder is sent to the target to be recognized. In this embodiment, each reference sub - limb movement executed by the target to be recognized within each reference execution time is obtained. During the reference execution time when the first reference sub - limb movement of the target limb movement set should be executed, the action video of the target to be recognized is collected, and the corresponding reference sub - limb movement is obtained according to this action video and a preset image analysis model.

[0037] In an exemplary embodiment of the present application, the above - mentioned step S200 includes: Step S210: According to the sequence of angles to be recognized JX, the sequence of reference action identifiers DX to be recognized, and the hidden Markov model, obtain the first feature list.

[0038] Specifically, using the sequence of angles to be recognized JX as the visible sequence of the hidden Markov model, and using the sequence of reference action identifiers DX to be recognized as the hidden state sequence of the hidden Markov model, the first feature list YT=(YTZ, YTG, YTC) is obtained according to the hidden Markov model; where YTZ is the state transition probability matrix corresponding to the target to be recognized within the target time window; YTG is the visible probability matrix corresponding to the target to be recognized within the target time window; YTC is the initial state probability vector corresponding to the target to be recognized within the target time window.

[0039] Step S220: According to the preset sequence of reference angle groups JB, the preset sequence of reference action identifiers DB, and the hidden Markov model, obtain the second feature list.

[0040] Specifically, taking the preset reference angle group sequence JB as the visible sequence of the hidden Markov model and the preset reference action identification sequence DB as the hidden state sequence of the hidden Markov model, a second feature list ET = (ETZ, ETG, ETC) is obtained according to the hidden Markov model; where ETZ is the reference state transition probability matrix corresponding to the target limb motion set; YTG is the reference visible probability matrix corresponding to the target limb motion set; YTC is the reference initial state probability vector corresponding to the target limb motion set; wherein, the preset reference angle group sequence JB and the preset reference action identification sequence DB are obtained according to each reference sub-limb action.

[0041] It should be noted that the state transition probability matrix in the HMM model represents the transition probability between hidden states; the visible probability matrix represents the probability of generating various visible values in each hidden state; the initial state probability vector represents the probability that the hidden Markov model is in each hidden state at the initial moment.

[0042] In an exemplary embodiment of the present application, the first matching degree YP meets the following conditions: YP = (YT · ET) / (|YT| × |ET|).

[0043] In this embodiment, in the first feature list corresponding to the target to be recognized, since the target to be recognized may have a poor mastery of a certain reference sub-limb action, may miss a certain reference sub-limb action or do a certain reference sub-limb action wrong (obviously wrong), at this time, the obtained hidden state may be different from the reference hidden state corresponding to the reference sub-limb action. And there are also differences between the visible sequence (angle group sequence to be recognized) corresponding to each hidden state and the corresponding preset reference angle group sequence. Therefore, in this embodiment, the first feature list corresponding to the target to be recognized is matched with the second feature list corresponding to each reference sub-limb action obtained by decomposing the target limb motion set to obtain the first matching degree. Since the state transition probability matrix in the HMM model represents the transition probability between hidden states; the visible probability matrix represents the probability of generating various visible values in each hidden state; the initial state probability vector represents the probability that the hidden Markov model is in each hidden state at the initial moment. It can be seen that the first matching degree mainly describes the recognition result of the target to be recognized from the execution accuracy of the hidden state, that is, the first matching degree mainly focuses on the completion degree of the target to be recognized for all reference sub-limb actions, and does not focus on the influence of the difference between the key limb angles of the target to be recognized and the reference key limb angles on the recognition result of the target to be recognized.

[0044] In an exemplary embodiment of the present application, the second matching degree EP meets the following conditions: EP = 1 - Σ n i=1 ZQi Among them, ZQ i is the execution accuracy corresponding to the target to be recognized when performing the i-th reference sub-limb movement; ZQ i =Σ f(i) a=1 α i，a ×(|JB i，a -JX i，a | / JB i，a ); Among them, JB i，a is the a-th reference key limb angle corresponding to the i-th reference sub-limb movement of the target limb movement set; α i，a is the weight corresponding to the a-th reference key limb angle of the i-th reference sub-limb movement of the target limb movement set. If |JB i，a -JX i，a |≤β×JB i，a , then |JB i，a -JX i，a |=|JB i，a -JX i，a |; If |JB i，a -JX i，a |>β×JB i，a , then |JB i，a -JX i，a |=0; Among them, 1 / 3≤β≤0.

[0045] In this embodiment, the second matching degree describes the difference between the key limb angle of the target to be recognized and the reference key limb angle. And in this embodiment, if |JB i，a -JX i，a |≤β×JB i，a , among them, 1 / 3≤β≤0. At this time, it indicates that the execution degree of the target to be recognized for this key limb angle is low, and there is a large difference from the reference key limb angle. It may be because the target to be recognized is not familiar with this reference sub-limb movement, so no action is taken or the action is incorrect. At this time, in this embodiment, the influence of the execution degree of this key limb angle on the execution degree of the corresponding reference sub-limb movement is not considered, because the above first matching degree has determined the influence on the recognition result of the target to be recognized from the overall completion degree of the action. Therefore, in this embodiment, when the execution degree of a certain key limb angle of a certain reference sub-limb movement of the target to be recognized is within the preset range (roughly the same as the reference key limb angle, but there may be slight differences, that is, outside the range of 1 / 3≤β≤0), the corresponding second matching degree is considered.

[0046] In summary, the first matching degree in this embodiment mainly focuses on the completion degree of the target to be recognized for all reference sub-limb movements, rather than emphasizing the impact of the difference between the key limb angles of the target to be recognized and the reference key limb angles on the recognition result of the target to be recognized. The second matching degree describes the difference between the execution degree of a certain key limb angle of a certain reference sub-limb movement of the target to be recognized within a preset range (roughly the same as the reference key limb angle, but there may be slight differences, that is, outside the range of 1 / 3 ≤ β ≤ 0) and the reference key limb angle.

[0047] It should be noted that the above α i，a is the weight corresponding to the ath reference key limb angle of the ith reference sub-limb movement in the target limb movement set. That is, in each reference sub-limb movement, the importance of each reference key limb angle is different. Therefore, different weights are assigned to each reference key limb angle in each reference sub-limb movement to make the obtained result more objective and accurate, where 0 < α i，a < 1.

[0048] In an exemplary embodiment of the present application, step S500 is specifically as follows: perform weighted summation on the above first matching degree and the above second matching degree to obtain the limb movement recognition result corresponding to the target to be recognized.

[0049] Specifically, the weight settings of the first matching degree and the second matching degree can be as follows: Obtain the target type of the target to be recognized. If it is a novice, set the weight of the first matching degree to be greater than the weight of the second matching degree. At this time, focus on obtaining the comprehensive completion degree of the target to be recognized for all reference sub-limb movements and temporarily ignore the action details. If it is not a novice, set the weight of the first matching degree to be less than the weight of the second matching degree. At this time, the target to be recognized can basically complete each reference sub-limb movement, so focus on obtaining the execution degree of the action details of each reference sub-limb movement.

[0050] Please refer to Figure 2 As shown, the embodiment of the present application provides an artificial intelligence-based limb movement action device recognition system 100. For the specific description of this system, please refer to the above method embodiment and will not be elaborated here. The system includes the following units: The first acquisition unit 110 is used to acquire the sequence JX of angles to be recognized and the sequence DX of reference action identifiers corresponding to the target to be recognized within the target time window; wherein, the start time of the target time window is the time when the target limb movement set starts to be executed; the end time of the target time window is the time when the target limb movement set ends to be executed.

[0051] A second acquisition unit 120, configured to obtain a first feature list and a second feature list according to a sequence JX of angles to be recognized, a sequence DX of reference action identifiers to be recognized, a preset sequence JB of reference angle groups, a preset sequence DB of reference action identifiers, and a hidden Markov model; wherein the first feature list is used to describe the motion features obtained by JX and DX according to the hidden Markov model; the second feature list is used to describe the motion features obtained by JB and DB according to the hidden Markov model.

[0052] A first matching unit 130, configured to obtain a first matching degree according to the first feature list and the second feature list.

[0053] A second matching unit 140, configured to obtain a second matching degree according to the sequence JX of angles to be recognized and the preset sequence JB of reference angle groups.

[0054] An identification unit 150, configured to obtain a limb motion recognition result corresponding to the target to be recognized according to the first matching degree and the second matching degree; wherein, the limb motion recognition result is used to describe the execution accuracy of the target to be recognized for the target limb motion set.

[0055] In an exemplary embodiment of the present application, an electronic device capable of implementing the above method is further provided.

[0056] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, various aspects of the present application can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuitry", "module", or "system" here.

[0057] An electronic device according to this embodiment of the present application. The electronic device is only an example and should not bring any limitation to the functions and usage scopes of the embodiments of the present application.

[0058] The electronic device is presented in the form of a general computing device. The components of the electronic device may include, but are not limited to: the at least one processor described above, the at least one storage, and a bus connecting different system components (including the storage and the processor).

[0059] Wherein, the storage stores program codes, and the program codes can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present application described in the "exemplary method" part of this specification.

[0060] The storage may include a readable medium in the form of a volatile storage, such as a random access storage (RAM) and / or a cache storage, and may further include a read-only storage (ROM).

[0061] The storage may also include a program / utilities having a set (at least one) of program modules. Such program modules include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0062] The bus may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus structures.

[0063] The electronic device may also communicate with one or more external devices (such as a keyboard, a pointing device, a Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication may be through an input / output (I / O) interface. Moreover, the electronic device may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter. As shown in the figure, the network adapter communicates with other modules of the electronic device through the bus. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0064] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0065] In an exemplary embodiment of the present application, there is also provided a computer-readable storage medium, on which a program product capable of implementing the above method of this specification is stored. In some possible implementation manners, various aspects of the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0066] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0067] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0068] The program code contained on the readable medium may be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0069] The program code for performing the operations of this application may be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by connecting through the Internet service provider via the Internet).

[0070] In addition, the above drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of this application, rather than for limiting purposes. It is easy to understand that the processes shown in the above drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0071] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0072] The above are only the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for identifying limb movement actions based on artificial intelligence, characterized in that, The method includes: Obtaining a sequence of angle groups to be recognized JX and a sequence of reference action identifiers to be recognized DX corresponding to the target to be recognized within a target time window; wherein, the start time of the target time window is the time when the target limb movement set starts to be executed; the end time of the target time window is the time when the target limb movement set ends to be executed; According to the sequence of angle groups to be recognized JX, the sequence of reference action identifiers to be recognized DX, a preset sequence of reference angle groups JB, a preset sequence of reference action identifiers DB, and a hidden Markov model, obtaining a first feature list and a second feature list; wherein the first feature list is used to describe the motion features obtained by JX and DX according to the hidden Markov model; the second feature list is used to describe the motion features obtained by JB and DB according to the hidden Markov model; Obtaining a first matching degree according to the first feature list and the second feature list; Obtaining a second matching degree according to the sequence of angle groups to be recognized JX and the preset sequence of reference angle groups JB; Obtaining a limb movement recognition result corresponding to the target to be recognized according to the first matching degree and the second matching degree; wherein, the limb movement recognition result is used to describe the execution accuracy of the target to be recognized for the target limb movement set.

2. The method for identifying limb movement actions of an artificial intelligence-based device according to claim 1, characterized in that The obtaining the sequence of angle groups to be recognized JX and the sequence of reference action identifiers to be recognized DX corresponding to the target to be recognized within a target time window includes: Obtain the sequence of angle groups to be recognized JX = (JX1, JX2,..., JX i ,..., JX n ) corresponding to the target to be recognized within the target time window; i = 1, 2,..., n; where n is the number of benchmark sub-limb actions obtained by decomposing the target limb movement set; JX i is the angle group corresponding to the target to be recognized when performing the i-th benchmark sub-limb action; JX i = (JX i，1 , JX i，2 ,..., JX i，a ,..., JX i,f(i) ); a = 1, 2,..., f(i); f(i) is the number of key limb angles included in the i-th benchmark sub-limb action; JX i，a is the a-th key limb angle corresponding to the target to be recognized when performing the i-th benchmark sub-limb action; the key limb angle is used to describe the execution accuracy of the target to be recognized for the benchmark sub-limb action; According to JX, the benchmark action identification sequence DX to be recognized is obtained as DX=(DX1, DX2, …, DX i , …, DX n ); where DX i is the benchmark action identification determined according to JX i ; the benchmark action identification is the action identification corresponding to the benchmark sub-limb action obtained by decomposing the target limb movement set.

3. The method for identifying limb movement actions of an artificial intelligence-based device according to claim 2, wherein JX i Determined according to the following steps: Obtaining a three-dimensional space coordinate system corresponding to the target to be recognized; wherein, the relationship between the origin in the three-dimensional space coordinate system corresponding to the target to be recognized and the target to be recognized is the same as the relationship between the origin in the reference three-dimensional space coordinate system corresponding to the target limb movement set and the reference target; According to the three-dimensional space coordinate system corresponding to the target to be recognized, obtain the set of key joint point coordinate lists GJ=(GJ1, GJ2, …, GJ i , …, GJ n ) corresponding to the target to be recognized within the target time window; where GJ i is the list of key joint point coordinates corresponding to the target to be recognized when performing the i-th reference sub-limb movement within the target time window; GJ i =(GJ i，1 , GJ i，2 , …, GJ i，b , …, GJ i，h(i) ); b = 1, 2, …, h(i); h(i) is the number of key joint points of the i-th reference sub-limb movement of the target limb movement set; GJ i，b is the coordinate of the b-th key joint point when the target to be recognized performs the i-th reference sub-limb movement within the target time window; According to the three-dimensional space coordinate system corresponding to the target to be recognized and GJ, obtain (JX i，1 , JX i，2 , …, JX i，a , …, JX i,f(i) ); where each key limb angle is the included angle between the line connecting the corresponding two key joint points and the corresponding key coordinate axis.

4. The method for identifying limb movement actions of an artificial intelligence-based device according to claim 3, characterized in that, The obtaining the first feature list and the second feature list according to the sequence of angle groups to be recognized JX, the sequence of reference action identifiers to be recognized DX, a preset sequence of reference angle groups JB, a preset sequence of reference action identifiers DB, and a hidden Markov model includes: Obtaining a first feature list according to the sequence of angle groups to be recognized JX, the sequence of reference action identifiers to be recognized DX, and the hidden Markov model; Obtaining a second feature list according to the preset sequence of reference angle groups JB, the preset sequence of reference action identifiers DB, and the hidden Markov model.

5. The method for identifying limb movement actions of an artificial-intelligence-based device according to claim 4, characterized in that, The obtaining the first feature list according to the sequence of angle groups to be recognized JX, the sequence of reference action identifiers to be recognized DX, and the hidden Markov model includes: Using the sequence of angle groups to be recognized JX as the visible sequence of the hidden Markov model, and using the sequence of reference action identifiers to be recognized DX as the hidden state sequence of the hidden Markov model, obtaining a first feature list YT = (YTZ, YTG, YTC) according to the hidden Markov model; wherein, YTZ is the state transition probability matrix corresponding to the target to be recognized within the target time window; YTG is the visible probability matrix corresponding to the target to be recognized within the target time window; YTC is the initial state probability vector corresponding to the target to be recognized within the target time window.

6. The method for identifying limb movement actions of an artificial intelligence-based device according to claim 5, wherein, The obtaining the second feature list according to the preset sequence of reference angle groups JB, the preset sequence of reference action identifiers DB, and the hidden Markov model includes: Using the preset reference angle group sequence JB as the visible sequence of the hidden Markov model, and the preset reference action identification sequence DB as the hidden state sequence of the hidden Markov model, a second feature list ET = (ETZ, ETG, ETC) is obtained according to the hidden Markov model; where ETZ is the reference state transition probability matrix corresponding to the target limb movement set; YTG is the reference visible probability matrix corresponding to the target limb movement set; YTC is the reference initial state probability vector corresponding to the target limb movement set; among them, the preset reference angle group sequence JB and the preset reference action identification sequence DB are obtained according to each reference sub-limb action.

7. The method for identifying limb movement actions of an artificial intelligence-based device according to claim 6, wherein The first matching degree is obtained according to the first feature list and the second feature list, where the first matching degree YP meets the following conditions: YP = (YT · ET) / (|YT| × |ET|).

8. The method for identifying limb movement actions of an artificial intelligence-based device according to claim 3, wherein The second matching degree EP is obtained according to the to-be-identified angle group sequence JX and the preset reference angle group sequence JB, where the second matching degree EP meets the following conditions: EP = 1 - Σ n i=1 ZQ i ; Among them, ZQ i is the execution accuracy corresponding to the i-th reference sub-limb movement of the target to be recognized; ZQ i =Σ f(i) a=1 α i，a ×(|JB i，a -JX i，a | / JB i，a ); among them, JB i，a is the a-th reference key limb angle corresponding to the i-th reference sub-limb movement of the target limb movement set; α i，a is the weight corresponding to the a-th reference key limb angle of the i-th reference sub-limb movement of the target limb movement set.

9. The method for identifying limb movement actions of an artificial intelligence-based device according to claim 8, wherein If |JB i，a -JX i，a | ≤ β × JB i，a , then |JB i，a -JX i，a | = |JB i，a -JX i，a |; if |JB i，a -JX i，a | > β × JB i，a , then |JB i，a -JX i，a | = 0; where, 1 / 3 ≤ β ≤ 0.

10. An artificial intelligence-based limb movement action device recognition system, characterized in that, The system includes: A first acquisition unit for acquiring the to-be-identified angle group sequence JX and the to-be-identified reference action identification sequence DX corresponding to the to-be-identified target within the target time window; where the start time of the target time window is the time when the execution of the target limb movement set starts; the end time of the target time window is the time when the execution of the target limb movement set ends; A second acquisition unit for obtaining a first feature list and a second feature list according to the to-be-identified angle group sequence JX, the to-be-identified reference action identification sequence DX, the preset reference angle group sequence JB, the preset reference action identification sequence DB, and the hidden Markov model; where the first feature list is used to describe the motion features obtained by JX and DX according to the hidden Markov model; the second feature list is used to describe the motion features obtained by JB and DB according to the hidden Markov model; A first matching unit for obtaining the first matching degree according to the first feature list and the second feature list; A second matching unit for obtaining the second matching degree according to the to-be-identified angle group sequence JX and the preset reference angle group sequence JB; An identification unit for obtaining the limb movement recognition result corresponding to the to-be-identified target according to the first matching degree and the second matching degree; where the limb movement recognition result is used to describe the execution accuracy of the to-be-identified target for the target limb movement set.

Images