Action capture management method and system based on artificial intelligence

Abstract

The invention discloses a motion capture management method and system based on artificial intelligence, and belongs to the technical field of motion capture. The system comprises a data acquisition module, an action analysis module, an intelligent processing module and a visualization module. The data acquisition module is used for acquiring a historical log and acquiring a video image and an induction distance of a target object in a motion capture process; the action analysis module is used for decomposing a video image and identifying key points of a target object, screening out marked key points and establishing associated key points, and establishing a relational expression for each pair of associated key points by adopting an artificial intelligence algorithm; the intelligent processing module calculates an influence coefficient of the associated key points through the relational expression, sets reference points for the marked key points according to the influence coefficient, calculates a priority index, and sequentially sets a predicted spatial position for each marked key point according to the priority index; and the visualization module constructs a virtual mapping model and dynamically displays the virtual mapping model according to the spatial position or the predicted spatial position of each key point.

Description

Technical Field

[0001] The present invention relates to the field of motion capture technology, and in particular to an artificial intelligence-based motion capture management method and system. Background Art

[0002] In recent years, with the rapid development of new technologies such as computer vision and deep learning, the accuracy, stability and processing speed of optical motion capture systems have been continuously improved. At the same time, improvements in data processing and analysis algorithms have promoted the application of real-time capture and processing, further expanding its application scenarios.

[0003] At present, optical motion capture technology usually uses multiple cameras to synchronously collect image data of the target object, combines triangulation technology to calculate the spatial position, and reconstructs the motion trajectory of the target object. This method has certain problems. On the one hand, during the image acquisition process, some areas of the image are often blurred due to poor shooting environment or too fast movement of the target object. In the prior art, deblurring algorithms or super-resolution reconstruction and other technologies are usually used to achieve image enhancement. This method not only consumes a lot of computing resources, but also wastes time and cannot meet the requirements of virtual mapping projects with high real-time requirements. On the other hand, when the key points in the blurred area are lost and cannot be located, the key point detection and reconstruction in the prior art often only analyzes the possibility based on historical data. This method has a large selection space and cannot be combined with other existing data to predict the best position in time, resulting in inaccurate predictions. Therefore, at this stage, an efficient and intelligent motion capture management technology solution is needed to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a motion capture management method and system based on artificial intelligence to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides a motion capture management method based on artificial intelligence, comprising the following steps:

[0006] S100 , collecting historical logs of the target object. In the process of capturing the motion of the target object, a sensor is used to collect a video image and a sensing distance of the target object in real time.

[0007] S200, decomposing the video image and identifying the key points of the target object, calculating the recognition rate of each key point and marking them, thereby establishing associated key points, and using an artificial intelligence algorithm to establish a relationship expression for each pair of associated key points.

[0008] S300, calculating the influence coefficient of the associated key point through a relational expression, setting a reference point for the marked key point according to the influence coefficient and calculating a priority index, constructing an activity space and a prediction space for each marked key point in descending order of the priority index, and setting the prediction space position through the activity space and the prediction space.

[0009] S400, obtaining the spatial position or predicted spatial position of each key point of the target object in the detection image in real time, constructing a virtual mapping model, and dynamically displaying it through a visualization screen.

[0010] In S100, the target object refers to a human target used for motion capture. The sensor is installed around the target object, specifically including a camera and a sensing module. The camera is used to collect video images of the target object. Each sensing module contains n distance sensors, which dynamically track the spatial positions of each key point under the target object in the video image, and collect the distance between each key point and the sensing module in the real world as the sensing distance.

[0011] The historical log refers to the location record and distance record of each key point under the target object. The location record includes the spatial position of the key point at different times, and the spatial position is obtained by analyzing the video images of the same key point collected by different cameras. The distance record includes the sensing distance of the same key point collected by different sensing modules at different times.

[0012] To analyze the spatial position of the same key point using multi-view video images, it is first necessary to calibrate multiple cameras to determine their internal and external parameters. Then, the key points are extracted in each view using a feature detection algorithm, and the same key points are found between different viewpoints using a feature matching algorithm. The 3D reconstruction algorithm is then applied to calculate the position of these key points in 3D space, while the accuracy is improved through data fusion and optimization methods, ultimately achieving accurate analysis of the spatial position of the key points.

[0013] The sensing module can dynamically track the key point in real time according to its spatial position, and detect the distance between the key point and itself in the real link through the distance sensor as the sensing distance. The sensing module contains multiple distance sensors, each of which can independently track a key point. Different sensing modules have different sensing distances for the same key point.

[0014] In S200, the specific steps are as follows:

[0015] S201, obtain the video images collected by each camera, and use OpenCV technology to decompose these video images into single frame images. nowThe corresponding single-frame images are used as detection images, and the YOLOv5 human detection algorithm is used to perform target recognition on these detection images, and the recognized human targets are used as target objects.

[0016] S202, respectively set the key point position to be identified in each detection image, use the AlphaPose algorithm to identify the key points of the target object and mark them. Analyze the spatial position of the key points in each detection image, and treat the key points in the same spatial position in different detection images as the same key point.

[0017] S203, count the number of times each key point should be identified in all detection images YCS sum and the number of times it has been identified JCS sum , JCS sum Divide by YCS sum Get the recognition rate of the corresponding key point, and mark the key point whose recognition rate is less than 1. a Associate them with other key points in pairs in turn to establish associated key points.

[0018] S204, set u time nodes and associate them with key points M a M b Establish a parameter set and obtain the associated key points M in the historical log a M b Parameter records, and retrieve the associated key points M at each time node a M b Analyze the three-dimensional coordinates of the spatial position and associate the key point M a M b The coordinate values ​​of the same dimension at each time node are all put into the same parameter set, and the three dimensions occupy three parameter sets in total.

[0019] Each element in a parameter set can only store two coordinate values ​​of the same dimension. Different elements correspond to different time nodes. The number of elements is the same as the number of time nodes. Different parameter sets store coordinate values ​​of different dimensions, and each dimension corresponds to a parameter set. The time nodes are set in advance by the staff.

[0020] S205, marking can simultaneously collect associated key points M a M b The sensing module of the sensing distance, counts the number of marked sensing modules f, and obtains the key points M associated with the collection of the marked sensing modules a M b The distance record is the associated key point M collected by the same tag sensing module a M bAll sensing distances at each time node are placed in the corresponding parameter set, occupying a total of f parameter sets. A training set is established for each parameter set, and two coordinate values ​​or two sensing distances corresponding to each element in the parameter set are obtained as independent variables and dependent variables and packaged as samples. These u samples are placed in the training set corresponding to the parameter set to which they belong.

[0021] Each marker sensing module corresponds to a parameter set, and the marker sensing module must be able to simultaneously collect the associated key points M a and M b Different elements in each parameter set correspond to different time nodes, and different parameter sets store the sensing distances collected by different tag sensing modules.

[0022] S206, establish a linear regression model, substitute the independent variable of each sample in the training set as the input value into the linear regression model expression, calculate the difference between the output result of each sample and the dependent variable, sum the differences of all samples in the training set as the gap coefficient, and obtain the minimum gap coefficient PL by adjusting the intercept Q and the regression coefficient I in the expression min , thus obtaining the relational expression corresponding to each training set:

[0023] P=Q+IK

[0024] In the formula, P is the dependent variable and K is the independent variable.

[0025] In S300, the specific steps are as follows:

[0026] S301, obtain the associated key point M a M b Corresponding f+3 expressions, calculate the standard deviation F of the intercepts in these f+3 expressions sd , standard deviation of regression coefficient I sd and the average value of the minimum gap coefficient PL ave , substitute into the formula to calculate the associated key point M a M b Influence coefficient:

[0027]

[0028] In the formula, OX ab is the influence coefficient. Calculate the marking key points M respectively a The corresponding influence coefficients of all associated key points.

[0029] S302: Setting coefficient threshold OX th , the marker influence coefficient is greater than OX th The associated key points will mark the key points M in the associated key points b As the key point Ma Reference points; according to the above method, set reference points for each marked key point. Analyze whether each reference point is marked, substitute the influence coefficient corresponding to each reference point under the marked key point into the formula, and calculate the priority index YX of each marked key point:

[0030]

[0031] Where α and β are constants, e is the number of all unmarked reference points, and OX r is the influence coefficient of the rth reference point. w is the number of all marked reference points, OX k is the influence coefficient of the kth reference point.

[0032] Marked key points can also be used as reference points for other marked key points at the same time. Different marked key points are distinguished by priority index, and marked key points with more unmarked reference points and larger influence coefficients are given priority.

[0033] S303, sort all the marked key points in descending order according to the priority index, and obtain the marked key point M with the highest ranking c . Mark the key point M c All unmarked reference points are used as reference points. The spatial position of each reference point is analyzed by detecting the image. The three-dimensional coordinates of the spatial position are respectively substituted into the corresponding relationship expression for calculation. The estimated spatial position is obtained by combining the calculation results of the three coordinate values. The estimated spatial position of all reference points is combined as the marking key point M. c activity space.

[0034] S304, obtaining the current sensing distance of each reference point collected by the marker sensing module, and substituting them into the corresponding relationship expression to calculate the estimated sensing distance. A spherical area is established with the spatial position of the marker sensing module as the center and the estimated sensing distance as the radius, and the spherical area of ​​all marker sensing modules is used as the prediction space.

[0035] S305, analyzing the center point of the intersection area between the activity space and the prediction space, and using the three-dimensional coordinates of the center point as the marking key point M c The predicted spatial position of the key point M is cancelled c Continue to enter step S302 to calculate the priority index of each marked key point and set the predicted spatial position until all marked key points are unmarked.

[0036] In S400, the detection images are updated in real time, and a virtual mapping model is constructed based on the spatial positions or predicted spatial positions of the key points of the target object in each detection image. The model is dynamically displayed on a visualization screen, highlighting the key points corresponding to the predicted spatial positions for reference and adjustment by managers.

[0037] A motion capture management system based on artificial intelligence includes a data acquisition module, a motion analysis module, an intelligent processing module and a visualization module.

[0038] The data acquisition module is used to collect historical logs and collect the video images and sensing distance of the target object during the motion capture process. The motion analysis module is used to decompose the video image and identify the key points of the target object, filter out the marked key points and establish associated key points, and use artificial intelligence algorithms to establish relationship expressions for each pair of associated key points. The intelligent processing module calculates the influence coefficient of the associated key points through the relationship expression, sets the reference point for the marked key points according to the influence coefficient and calculates the priority index, and sets the predicted spatial position for each marked key point in turn according to the priority index. The visualization module constructs a virtual mapping model based on the spatial position or predicted spatial position of each key point and displays it dynamically.

[0039] The data acquisition module includes a history log acquisition unit, an image data acquisition unit and a sensing parameter acquisition unit.

[0040] The historical log collection unit is used to collect the position records and distance records of each key point under the target object. The position record includes the spatial position of the key point at different times, and the distance record includes the sensing distance of the same key point collected by different sensing modules at different times.

[0041] The image data acquisition unit collects video images of the target object through different cameras. The sensing parameter acquisition unit collects the distance between each key point in the real world and the sensing module as the sensing distance through the sensing module, and each sensing module contains n distance sensors.

[0042] The action analysis module includes an anomaly detection unit and a relationship analysis unit.

[0043] The anomaly detection unit is used to filter and mark key points.

[0044] First, the video images collected by each camera are decomposed into single frame images using OpenCV technology, and the current time T in each video image source is converted into now The corresponding single-frame images are used as detection images, and the YOLOv5 human detection algorithm is used to identify human targets in these detection images as target objects.

[0045] Secondly, the key points to be identified are set in each detection image, and the AlphaPose algorithm is used to identify and mark the key points of the target object; the spatial positions of the key points in each detection image are analyzed, and the key points in the same spatial positions in different detection images are regarded as the same key point.

[0046] Finally, count the number of times each key point should be identified in all detection images YCS sum and the number of times it has been identified JCS sum , JCS sum Divide by YCS sum Get the recognition rate of the corresponding key point, mark the key point with a recognition rate less than 1; mark the marked key point M a Associate them with other key points in pairs in turn to establish associated key points.

[0047] The relationship analysis unit is used to establish a relationship expression for each pair of associated key points.

[0048] First, set u time nodes and associate them with key points M a M b Establish a parameter set and obtain the associated key points M in the historical log a M b Parameter records, and retrieve the associated key points M at each time node a M b spatial location.

[0049] Secondly, analyze the three-dimensional coordinates of the spatial position and associate the key point M a M b The coordinate values ​​of the same dimension at each time node are all put into the same parameter set, and the three dimensions occupy three parameter sets in total.

[0050] Then, the marker can simultaneously collect the associated key points M a M b The sensing module of the sensing distance, counts the number of marked sensing modules f, and obtains the key points M associated with the collection of the marked sensing modules a M b The distance record is the associated key point M collected by the same tag sensing module a M b The sensing distances at each time node are all put into the corresponding parameter set, occupying a total of f parameter sets.

[0051] Finally, a training set is established for each parameter set, and the two coordinate values ​​or two sensing distances corresponding to each element in the parameter set are obtained as independent variables and dependent variables respectively and packaged as samples. These u samples are placed in the training set corresponding to the parameter set to which they belong.

[0052] Establish a linear regression model, substitute the independent variable of each sample in the training set as the input value into the linear regression model expression, calculate the difference between the output result of each sample and the dependent variable, sum the differences of all samples in the training set as the gap coefficient, and obtain the minimum gap coefficient PL by adjusting the intercept Q and regression coefficient I in the expression min, thus obtaining the relational expression P=Q+IK corresponding to each training set, where P is the dependent variable and K is the independent variable.

[0053] The intelligent processing module includes an impact positioning unit and a position reconstruction unit.

[0054] The influence positioning unit is used to calculate the influence coefficient of the associated key points and the priority index of the marked key points.

[0055] First, obtain the associated key point M a M b Corresponding f+3 expressions, calculate the standard deviation F of the intercepts in these f+3 expressions sd , standard deviation of regression coefficient I sd and the average value of the minimum gap coefficient PL ave .

[0056] Secondly, according to the formula Calculate the associated key point M a M b The influence coefficient of the marked key points M is calculated respectively a The corresponding influence coefficient of all associated key points. Set the coefficient threshold OX th , the marker influence coefficient is greater than OX th The associated key points will mark the key points M in the associated key points b As the key point M a Set the reference point for each marked keypoint as described above.

[0057] Finally, according to the formula Calculate the priority index of each marked key point. Where α and β are constants, e is the number of all unmarked reference points, OX r is the influence coefficient of the rth reference point. w is the number of all marked reference points, OX k is the influence coefficient of the kth reference point.

[0058] The position reconstruction unit is used to set the predicted spatial positions for the marked key points.

[0059] First, sort all the marked key points in descending order according to the priority index, and obtain the top-ranked marked key point M c . Mark the key point M c All unmarked reference points in the image are used as reference points. The three-dimensional coordinates of each reference point in the detected image are substituted into the corresponding relationship expression for calculation. The estimated spatial position is obtained by combining the calculation results of the three coordinate values. The estimated spatial position of all reference points is combined as the marking key point M. c activity space.

[0060] Secondly, the current sensing distance of each reference point collected by the marker sensing module is obtained, and the estimated sensing distance is calculated by substituting them into the corresponding relationship expression. A spherical area is established with the spatial position of the marker sensing module as the center and the estimated sensing distance as the radius, and the spherical area of ​​all marker sensing modules is used as the prediction space.

[0061] Finally, the center point of the intersection area between the activity space and the prediction space is analyzed, and the three-dimensional coordinates of the center point are used as the marking key point M c The predicted spatial position of the key point M is cancelled c Continue to enter step S302 to calculate the priority index of each marked key point and set the predicted spatial position until all marked key points are unmarked.

[0062] The visualization module is used to update the detection images in real time, and to build a virtual mapping model based on the spatial position or predicted spatial position of each key point of the target object in each detection image, and to display it dynamically on the visualization screen.

[0063] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0064] In this application, the video images captured by each camera are first deframed, and the detection image is defined according to the current time. The key points that should be identified and the key points that have been identified under each detection image are analyzed, so as to calculate the recognition rate of each key point and filter out the marked key points.

[0065] Secondly, the marked key points are matched with other key points in pairs to obtain several pairs of associated key points, and an artificial intelligence algorithm is used to establish a relational expression for each pair of associated key points. The relational expression is analyzed to calculate the influence coefficient of each pair of associated key points, and a reference point is set for each marked key point according to the influence coefficient.

[0066] Finally, the priority index of each marked key point is calculated by the influence coefficient of the reference point, and the predicted spatial position is set for each marked key point in descending order according to the priority index to achieve key point reconstruction. Compared with the existing technology of using deblurring algorithm to clarify the image or only judging the possibility based on historical data, it is more efficient and intelligent.

[0067] In summary, compared with traditional technologies, the present invention has the advantages of efficient key positioning and intelligent reconstruction, and can improve the efficiency of motion capture data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0069] Figure 1 It is a flow chart of a motion capture management method based on artificial intelligence of the present invention;

[0070] Figure 2 It is a structural schematic diagram of an artificial intelligence-based motion capture management system of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] See also Figure 1 The present invention provides a motion capture management method based on artificial intelligence, comprising the following steps:

[0073] S100 , collecting historical logs of the target object. In the process of capturing the motion of the target object, a sensor is used to collect a video image and a sensing distance of the target object in real time.

[0074] S200, decomposing the video image and identifying the key points of the target object, calculating the recognition rate of each key point and marking them, thereby establishing associated key points, and using an artificial intelligence algorithm to establish a relationship expression for each pair of associated key points.

[0075] S300, calculating the influence coefficient of the associated key point through a relational expression, setting a reference point for the marked key point according to the influence coefficient and calculating a priority index, constructing an activity space and a prediction space for each marked key point in descending order of the priority index, and setting the prediction space position through the activity space and the prediction space.

[0076] S400, obtaining the spatial position or predicted spatial position of each key point of the target object in the detection image in real time, constructing a virtual mapping model, and dynamically displaying it through a visualization screen.

[0077] In S100, the target object refers to a human target used for motion capture. The sensor is installed around the target object, specifically including a camera and a sensing module. The camera is used to collect video images of the target object. Each sensing module contains n distance sensors, which dynamically track the spatial positions of each key point under the target object in the video image, and collect the distance between each key point and the sensing module in the real world as the sensing distance.

[0078] The historical log refers to the location record and distance record of each key point under the target object. The location record includes the spatial position of the key point at different times, and the spatial position is obtained by analyzing the video images of the same key point collected by different cameras. The distance record includes the sensing distance of the same key point collected by different sensing modules at different times.

[0079] To analyze the spatial position of the same key point using multi-view video images, it is first necessary to calibrate multiple cameras to determine their internal and external parameters. Then, the key points are extracted in each view using a feature detection algorithm, and the same key points are found between different viewpoints using a feature matching algorithm. The 3D reconstruction algorithm is then applied to calculate the position of these key points in 3D space, while the accuracy is improved through data fusion and optimization methods, ultimately achieving accurate analysis of the spatial position of the key points.

[0080] The sensing module can dynamically track the key point in real time according to its spatial position, and detect the distance between the key point and itself in the real link through the distance sensor as the sensing distance. The sensing module contains multiple distance sensors, each of which can independently track a key point. Different sensing modules have different sensing distances for the same key point.

[0081] In S200, the specific steps are as follows:

[0082] S201, obtain the video images collected by each camera, and use OpenCV technology to decompose these video images into single frame images. now The corresponding single-frame images are used as detection images, and the YOLOv5 human detection algorithm is used to perform target recognition on these detection images, and the recognized human targets are used as target objects.

[0083] S202, respectively set the key point position to be identified in each detection image, use the AlphaPose algorithm to identify the key points of the target object and mark them. Analyze the spatial position of the key points in each detection image, and treat the key points in the same spatial position in different detection images as the same key point.

[0084] S203, count the number of times each key point should be identified in all detection images YCS sum and the number of times it has been identified JCS sum , JCS sum Divide by YCS sum Get the recognition rate of the corresponding key point, and mark the key point whose recognition rate is less than 1. a Associate them with other key points in pairs in turn to establish associated key points.

[0085] S204, set u time nodes and associate them with key points M a M b Establish a parameter set and obtain the associated key points M in the historical log a M b Parameter records, and retrieve the associated key points M at each time node a M b Analyze the three-dimensional coordinates of the spatial position and associate the key point M a M b The coordinate values ​​of the same dimension at each time node are all put into the same parameter set, and the three dimensions occupy three parameter sets in total.

[0086] Each element in a parameter set can only store two coordinate values ​​of the same dimension. Different elements correspond to different time nodes. The number of elements is the same as the number of time nodes. Different parameter sets store coordinate values ​​of different dimensions, and each dimension corresponds to a parameter set. The time nodes are set in advance by the staff.

[0087] S205, marking can simultaneously collect associated key points M a M b The sensing module of the sensing distance, counts the number of marked sensing modules f, and obtains the key points M associated with the collection of the marked sensing modules a M b The distance record is the associated key point M collected by the same tag sensing module a M b All sensing distances at each time node are placed in the corresponding parameter set, occupying a total of f parameter sets. A training set is established for each parameter set, and two coordinate values ​​or two sensing distances corresponding to each element in the parameter set are obtained as independent variables and dependent variables and packaged as samples. These u samples are placed in the training set corresponding to the parameter set to which they belong.

[0088] Each marker sensing module corresponds to a parameter set, and the marker sensing module must be able to simultaneously collect the associated key points M a and M b Different elements in each parameter set correspond to different time nodes, and different parameter sets store the sensing distances collected by different tag sensing modules.

[0089] S206, establish a linear regression model, substitute the independent variable of each sample in the training set as the input value into the linear regression model expression, calculate the difference between the output result of each sample and the dependent variable, sum the differences of all samples in the training set as the gap coefficient, and obtain the minimum gap coefficient PL by adjusting the intercept Q and the regression coefficient I in the expression min , thus obtaining the relational expression corresponding to each training set:

[0090] P=Q+IK

[0091] In the formula, P is the dependent variable and K is the independent variable.

[0092] In S300, the specific steps are as follows:

[0093] S301, obtain the associated key point M a M b Corresponding f+3 expressions, calculate the standard deviation F of the intercepts in these f+3 expressions sd , standard deviation of regression coefficient I sd and the average value of the minimum gap coefficient PL ave , substitute into the formula to calculate the associated key point M a M b Influence coefficient:

[0094]

[0095] In the formula, OX ab is the influence coefficient. Calculate the marking key points M respectively a The corresponding influence coefficients of all associated key points.

[0096] S302: Setting coefficient threshold OX th , the marker influence coefficient is greater than OX th The associated key points will mark the key points M in the associated key points b As the key point M a Reference points; according to the above method, set reference points for each marked key point. Analyze whether each reference point is marked, substitute the influence coefficient corresponding to each reference point under the marked key point into the formula, and calculate the priority index YX of each marked key point:

[0097]

[0098] Where α and β are constants, e is the number of all unmarked reference points, and OX r is the influence coefficient of the rth reference point. w is the number of all marked reference points, OX k is the influence coefficient of the kth reference point.

[0099] Marked key points can also be used as reference points for other marked key points at the same time. Different marked key points are distinguished by priority index, and marked key points with more unmarked reference points and larger influence coefficients are given priority.

[0100] S303, sort all the marked key points in descending order according to the priority index, and obtain the marked key point M with the highest ranking c . Mark the key point Mc All unmarked reference points are used as reference points. The spatial position of each reference point is analyzed by detecting the image. The three-dimensional coordinates of the spatial position are respectively substituted into the corresponding relationship expression for calculation. The estimated spatial position is obtained by combining the calculation results of the three coordinate values. The estimated spatial position of all reference points is combined as the marking key point M. c activity space.

[0101] S304, obtaining the current sensing distance of each reference point collected by the marker sensing module, and substituting them into the corresponding relationship expression to calculate the estimated sensing distance. A spherical area is established with the spatial position of the marker sensing module as the center and the estimated sensing distance as the radius, and the spherical area of ​​all marker sensing modules is used as the prediction space.

[0102] S305, analyzing the center point of the intersection area between the activity space and the prediction space, and using the three-dimensional coordinates of the center point as the marking key point M c The predicted spatial position of the key point M is cancelled c Continue to enter step S302 to calculate the priority index of each marked key point and set the predicted spatial position until all marked key points are unmarked.

[0103] In S400, the detection images are updated in real time, and a virtual mapping model is constructed based on the spatial positions or predicted spatial positions of the key points of the target object in each detection image. The model is dynamically displayed on a visualization screen, highlighting the key points corresponding to the predicted spatial positions for reference and adjustment by managers.

[0104] See also Figure 2 The present invention provides a motion capture management system based on artificial intelligence, including a data acquisition module, a motion analysis module, an intelligent processing module and a visualization module.

[0105] The data acquisition module is used to collect historical logs and collect the video images and sensing distance of the target object during the motion capture process. The motion analysis module is used to decompose the video image and identify the key points of the target object, filter out the marked key points and establish associated key points, and use artificial intelligence algorithms to establish relationship expressions for each pair of associated key points. The intelligent processing module calculates the influence coefficient of the associated key points through the relationship expression, sets the reference point for the marked key points according to the influence coefficient and calculates the priority index, and sets the predicted spatial position for each marked key point in turn according to the priority index. The visualization module constructs a virtual mapping model based on the spatial position or predicted spatial position of each key point and displays it dynamically.

[0106] The data acquisition module includes a history log acquisition unit, an image data acquisition unit and a sensing parameter acquisition unit.

[0107] The historical log collection unit is used to collect the position records and distance records of each key point under the target object. The position record includes the spatial position of the key point at different times, and the distance record includes the sensing distance of the same key point collected by different sensing modules at different times.

[0108] The image data acquisition unit collects video images of the target object through different cameras. The sensing parameter acquisition unit collects the distance between each key point in the real world and the sensing module as the sensing distance through the sensing module, and each sensing module contains n distance sensors.

[0109] The action analysis module includes an anomaly detection unit and a relationship analysis unit.

[0110] The anomaly detection unit is used to filter and mark key points.

[0111] First, the video images collected by each camera are decomposed into single frame images using OpenCV technology, and the current time T in each video image source is converted into now The corresponding single-frame images are used as detection images, and the YOLOv5 human detection algorithm is used to identify human targets in these detection images as target objects.

[0112] Secondly, the key points to be identified are set in each detection image, and the AlphaPose algorithm is used to identify and mark the key points of the target object; the spatial positions of the key points in each detection image are analyzed, and the key points in the same spatial positions in different detection images are regarded as the same key point.

[0113] Finally, count the number of times each key point should be identified in all detection images YCS sum and the number of times it has been identified JCS sum , JCS sum Divide by YCS sum Get the recognition rate of the corresponding key point, mark the key point with a recognition rate less than 1; mark the marked key point M a Associate them with other key points in pairs in turn to establish associated key points.

[0114] The relationship analysis unit is used to establish a relationship expression for each pair of associated key points.

[0115] First, set u time nodes and associate them with key points M a M b Establish a parameter set and obtain the associated key points M in the historical log a M b Parameter records, and retrieve the associated key points M at each time node a M b spatial location.

[0116] Secondly, analyze the three-dimensional coordinates of the spatial position and associate the key point M a M b The coordinate values ​​of the same dimension at each time node are all put into the same parameter set, and the three dimensions occupy three parameter sets in total.

[0117] Then, the marker can simultaneously collect the associated key points M a M b The sensing module of the sensing distance, counts the number of marked sensing modules f, and obtains the key points M associated with the collection of the marked sensing modules a M b The distance record is the associated key point M collected by the same tag sensing module a M b The sensing distances at each time node are all put into the corresponding parameter set, occupying a total of f parameter sets.

[0118] Finally, a training set is established for each parameter set, and the two coordinate values ​​or two sensing distances corresponding to each element in the parameter set are obtained as independent variables and dependent variables respectively and packaged as samples. These u samples are placed in the training set corresponding to the parameter set to which they belong.

[0119] Establish a linear regression model, substitute the independent variable of each sample in the training set as the input value into the linear regression model expression, calculate the difference between the output result of each sample and the dependent variable, sum the differences of all samples in the training set as the gap coefficient, and obtain the minimum gap coefficient PL by adjusting the intercept Q and regression coefficient I in the expression min , thus obtaining the relational expression P=Q+IK corresponding to each training set, where P is the dependent variable and K is the independent variable.

[0120] The intelligent processing module includes an impact positioning unit and a position reconstruction unit.

[0121] The influence positioning unit is used to calculate the influence coefficient of the associated key points and the priority index of the marked key points.

[0122] First, obtain the associated key point M a M b Corresponding f+3 expressions, calculate the standard deviation F of the intercepts in these f+3 expressions sd , standard deviation of regression coefficient I sd and the average value of the minimum gap coefficient PL ave .

[0123] Secondly, according to the formula Calculate the associated key point M a M b The influence coefficient of the marked key points M is calculated respectively aThe corresponding influence coefficient of all associated key points. Set the coefficient threshold OX th , the marker influence coefficient is greater than OX th The associated key points will mark the key points M in the associated key points b As the key point M a Set the reference point for each marked keypoint as described above.

[0124] Finally, according to the formula Calculate the priority index of each marked key point. Where α and β are constants, e is the number of all unmarked reference points, OX r is the influence coefficient of the rth reference point. w is the number of all marked reference points, OX k is the influence coefficient of the kth reference point.

[0125] The position reconstruction unit is used to set the predicted spatial positions for the marked key points.

[0126] First, sort all the marked key points in descending order according to the priority index, and obtain the top-ranked marked key point M c . Mark the key point M c All unmarked reference points in the image are used as reference points. The three-dimensional coordinates of each reference point in the detected image are substituted into the corresponding relationship expression for calculation. The estimated spatial position is obtained by combining the calculation results of the three coordinate values. The estimated spatial position of all reference points is combined as the marking key point M. c activity space.

[0127] Secondly, the current sensing distance of each reference point collected by the marker sensing module is obtained, and the estimated sensing distance is calculated by substituting them into the corresponding relationship expression. A spherical area is established with the spatial position of the marker sensing module as the center and the estimated sensing distance as the radius, and the spherical area of ​​all marker sensing modules is used as the prediction space.

[0128] Finally, the center point of the intersection area between the activity space and the prediction space is analyzed, and the three-dimensional coordinates of the center point are used as the marking key point M c The predicted spatial position of the key point M is cancelled c Continue to enter step S302 to calculate the priority index of each marked key point and set the predicted spatial position until all marked key points are unmarked.

[0129] The visualization module is used to update the detection images in real time, and to build a virtual mapping model based on the spatial position or predicted spatial position of each key point of the target object in each detection image, and to display it dynamically on the visualization screen.

[0130] Embodiment 1:

[0131] Assume that a pair of associated key points M a M b The standard deviation of the intercept in all expressions is 1.2, the standard deviation of the regression coefficient is 2.1, and the average value of the minimum gap coefficient is 1.5. When the number of time nodes is 8, the formula is used to calculate the pair of associated key points M. a M b Influence coefficient:

[0132] Influence coefficient:

[0133] When the coefficient threshold is 1, M b As the key point M a reference point.

[0134] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0135] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A motion capture management method based on artificial intelligence, characterized in that: The method comprises the following steps: S100, collecting historical logs of the target object, and in the process of motion capture of the target object, collecting video images and sensing distances of the target object in real time through sensors; S200, decomposing the video image and identifying key points of the target object, calculating the recognition rate of each key point and marking them, thereby establishing associated key points, and using an artificial intelligence algorithm to establish a relationship expression for each pair of associated key points; S300, calculating the influence coefficient of the associated key point through a relational expression, setting a reference point for the marked key point according to the influence coefficient and calculating a priority index, constructing an activity space and a prediction space for each marked key point in descending order of the priority index, and setting the prediction space position through the activity space and the prediction space; S400, obtaining the spatial position or predicted spatial position of each key point of the target object in the detection image in real time, constructing a virtual mapping model, and dynamically displaying it through a visualization screen.

2. The motion capture management method based on artificial intelligence according to claim 1, characterized in that: In S100, the target object refers to a human target used for motion capture; the sensors are installed around the target object, specifically including a camera and a sensing module; the camera is used to collect video images of the target object; each sensing module contains n distance sensors, which dynamically track the spatial positions of each key point under the target object in the video image, and collect the distance between each key point and the sensing module in the real world as the sensing distance; the historical log refers to the position record and distance record of each key point under the target object; the position record includes the spatial position of the key point at different times, and the spatial position is obtained by analyzing the video images of the same key point collected by different cameras; the distance record includes the sensing distance of the same key point collected by different sensing modules at different times.

3. The motion capture management method based on artificial intelligence according to claim 2, characterized in that: In S200, the specific steps are as follows: S201, obtain the video images collected by each camera, and use OpenCV technology to decompose these video images into single frame images; now The corresponding single-frame images are used as detection images, and the YOLOv5 human detection algorithm is used to perform target recognition on these detection images, and the recognized human targets are used as target objects; S202, respectively setting the key point position to be identified in each detection image, using the AlphaPose algorithm to identify the key points of the target object and marking them; analyzing the spatial position of the key points in each detection image, and treating the key points at the same spatial position in different detection images as the same key point; S203, count the number of times each key point should be identified in all detection images YCS sum and the number of times it has been identified JCS sum , JCS sum Divide by YCS sum Get the recognition rate of the corresponding key point, mark the key point with a recognition rate less than 1; mark the key point M a Sequentially associate with other key points in pairs to establish associated key points; S204, set u time nodes and associate them with key points M a M b Establish a parameter set and obtain the associated key points M in the historical log a M b Parameter records, and retrieve the associated key points M at each time node a M b The spatial position of the key point M is associated with the three-dimensional coordinates of the spatial position. a M b The coordinate values ​​of the same dimension at each time node are all put into the same parameter set. The three dimensions occupy three parameter sets in total. S205, marking can simultaneously collect associated key points M a M b The sensing module of the sensing distance, counts the number of marked sensing modules f, and obtains the key points M associated with the collection of the marked sensing modules a M b The distance record is the associated key point M collected by the same tag sensing module a M b All sensing distances at each time node are put into the corresponding parameter set, occupying a total of f parameter sets; a training set is established for each parameter set, and two coordinate values ​​or two sensing distances corresponding to each element in the parameter set are obtained, which are used as independent variables and dependent variables respectively and packaged into samples; these u samples are put into the training set corresponding to the parameter set to which they belong; S206, establish a linear regression model, substitute the independent variable of each sample in the training set as the input value into the linear regression model expression, calculate the difference between the output result of each sample and the dependent variable, sum the differences of all samples in the training set as the gap coefficient, and obtain the minimum gap coefficient PL by adjusting the intercept Q and the regression coefficient I in the expression min , thus obtaining the relational expression corresponding to each training set: P=Q+IK In the formula, P is the dependent variable and K is the independent variable.

4. The motion capture management method based on artificial intelligence according to claim 3 is characterized in that: In S300, the specific steps are as follows: S301, obtain the associated key point M a M b Corresponding f+3 expressions, calculate the standard deviation F of the intercepts in these f+3 expressions sd , standard deviation of regression coefficient I sd and the average value of the minimum gap coefficient Pl ave , substitute into the formula to calculate the associated key point M a M b Influence coefficient: In the formula, OX ab is the influence coefficient; the key points M are calculated separately a The corresponding influence coefficients of all associated key points; S302: Setting coefficient threshold OX th , the marker influence coefficient is greater than OX th The associated key points will mark the key points M in the associated key points b As the key point M a reference point; according to the above method, set a reference point for each marked key point; analyze whether each reference point is marked, substitute the influence coefficient corresponding to each reference point under the marked key point into the formula, and calculate the priority index YX of each marked key point: Where α and β are constants, e is the number of all unmarked reference points, and OX r is the influence coefficient of the rth reference point; w is the number of all marked reference points, OX k is the influence coefficient of the kth reference point; S303, sort all the marked key points in descending order according to the priority index, and obtain the marked key point M with the highest ranking c ; Mark the key point M c All unmarked reference points are used as reference points. The spatial position of each reference point is analyzed by detecting the image. The three-dimensional coordinates of the spatial position are respectively substituted into the corresponding relationship expression for calculation. The estimated spatial position is obtained by combining the calculation results of the three coordinate values. The estimated spatial positions of all reference points are combined as the marking key point M. c activity space; S304, obtaining the current sensing distance of each reference point collected by the marker sensing module, and substituting them into the corresponding relationship expression to calculate the estimated sensing distance; establishing a spherical area with the spatial position of the marker sensing module as the center and the estimated sensing distance as the radius, and using the spherical area of ​​all marker sensing modules as the prediction space; S305, analyzing the center point of the intersection area between the activity space and the prediction space, and using the three-dimensional coordinates of the center point as the marking key point M c The predicted spatial position of the key point M c Marking; Continue to enter step S302 to calculate the priority index of each marked key point and set the predicted spatial position until all marked key points are unmarked.

5. The motion capture management method based on artificial intelligence according to claim 4, characterized in that: In S400, the detection images are updated in real time, and a virtual mapping model is constructed based on the spatial positions or predicted spatial positions of the key points of the target object in each detection image. The model is dynamically displayed on a visualization screen, highlighting the key points corresponding to the predicted spatial positions for reference and adjustment by managers.

6. A motion capture management system based on artificial intelligence, characterized by: The system includes data acquisition module, motion analysis module, intelligent processing module and visualization module; The data acquisition module is used to collect historical logs and collect video images and sensing distances of target objects during motion capture. The motion analysis module is used to decompose video images and identify key points of target objects, screen out marked key points and establish associated key points, and use artificial intelligence algorithms to establish relationship expressions for each pair of associated key points. The intelligent processing module calculates the influence coefficient of associated key points through relationship expressions, sets reference points for marked key points according to the influence coefficients and calculates priority indexes, and sets predicted spatial positions for each marked key point in turn according to the priority index. The visualization module constructs a virtual mapping model and displays it dynamically based on the spatial position or predicted spatial position of each key point.

7. The motion capture management system based on artificial intelligence according to claim 6, characterized in that: The data acquisition module includes a history log acquisition unit, an image data acquisition unit and a sensing parameter acquisition unit; The historical log collection unit is used to collect the position records and distance records of each key point under the target object; the position record includes the spatial position of the key point at different times, and the distance record includes the sensing distance of the same key point collected by different sensing modules at different times; The image data acquisition unit collects video images of the target object through different cameras; the sensing parameter acquisition unit collects the distance between each key point in the real world and the sensing module as the sensing distance through the sensing module, and each sensing module contains n distance sensors.

8. The motion capture management system based on artificial intelligence according to claim 7, characterized in that: The action analysis module includes an anomaly detection unit and a relationship analysis unit; The anomaly detection unit is used to filter and mark key points; First, the video images collected by each camera are decomposed into single frame images using OpenCV technology, and the current time T in each video image source is converted into now The corresponding single-frame images are used as detection images, and the YOLOv5 human detection algorithm is used to identify human targets in these detection images as target objects; Secondly, the key points to be identified are set in each detection image, and the AlphaPose algorithm is used to identify and mark the key points of the target object; the spatial position of the key points in each detection image is analyzed, and the key points with the same spatial position in different detection images are regarded as the same key point; Finally, count the number of times each key point should be identified in all detection images YCS sum and the number of times it has been identified JCS sum , JCS sum Divide by YCS sum Get the recognition rate of the corresponding key points, and mark the key points whose recognition rate is less than 1; Mark the key point M a Sequentially associate with other key points in pairs to establish associated key points; The relation analysis unit is used to establish a relation expression for each pair of associated key points; First, set u time nodes and associate them with key points M a M b Establish a parameter set and obtain the associated key points M in the historical log a M b Parameter records, and retrieve the associated key points M at each time node a M b spatial location; Secondly, analyze the three-dimensional coordinates of the spatial position and associate the key point M a M b The coordinate values ​​of the same dimension at each time node are all put into the same parameter set. The three dimensions occupy three parameter sets in total. Then, the marker can simultaneously collect the associated key points M a M b The sensing module of the sensing distance, counts the number of marked sensing modules f, and obtains the key points M associated with the collection of the marked sensing modules a M b The distance record is the associated key point M collected by the same tag sensing module a M b The sensing distances at each time node are all put into the corresponding parameter set, occupying a total of f parameter sets; Finally, a training set is established for each parameter set, and two coordinate values ​​or two sensing distances corresponding to each element in the parameter set are obtained as independent variables and dependent variables respectively and packaged as samples; these u samples are put into the training set corresponding to the parameter set to which they belong; Establish a linear regression model, substitute the independent variable of each sample in the training set as the input value into the linear regression model expression, calculate the difference between the output result of each sample and the dependent variable, sum the differences of all samples in the training set as the gap coefficient, and obtain the minimum gap coefficient PL by adjusting the intercept Q and regression coefficient I in the expression min , thus obtaining the relational expression P=Q+IK corresponding to each training set, where P is the dependent variable and K is the independent variable.

9. The motion capture management system based on artificial intelligence according to claim 8, characterized in that: The intelligent processing module includes an impact positioning unit and a position reconstruction unit; The influence positioning unit is used to calculate the influence coefficient of the associated key points and the priority index of the marked key points; First, obtain the associated key point M a M b Corresponding f+3 expressions, calculate the standard deviation F of the intercepts in these f+3 expressions sd , standard deviation of regression coefficient I sd and the average value of the minimum gap coefficient Pl ave ; Secondly, according to the formula Calculate the associated key point M a M b The influence coefficient of the marked key points M is calculated respectively a The corresponding influence coefficient of all associated key points; set the coefficient threshold OX th , the marker influence coefficient is greater than OX th The associated key points will mark the key points M in the associated key points b As the key point M a According to the above method, set the reference point for each marked key point; Finally, according to the formula Calculate the priority index of each marked key point; where α and β are constants, e is the number of all unmarked reference points, and OX r is the influence coefficient of the rth reference point; w is the number of all marked reference points, OX k is the influence coefficient of the kth reference point; The position reconstruction unit is used to set the predicted spatial position for the marked key points; First, sort all the marked key points in descending order according to the priority index, and obtain the top-ranked marked key point M c ; Mark the key point M c All unmarked reference points in the image are used as reference points. The three-dimensional coordinates of each reference point in the detected image are respectively substituted into the corresponding relationship expression for calculation. The estimated spatial position is obtained by combining the calculation results of the three coordinate values. The estimated spatial position of all reference points is combined as the marking key point M. c activity space; Secondly, the current sensing distance of each reference point collected by the marker sensing module is obtained, and the estimated sensing distance is calculated by substituting them into the corresponding relationship expression; a spherical area is established with the spatial position of the marker sensing module as the center and the estimated sensing distance as the radius, and the spherical area of ​​all marker sensing modules is used as the prediction space; Finally, the center point of the intersection area between the activity space and the prediction space is analyzed, and the three-dimensional coordinates of the center point are used as the marking key point M c The predicted spatial position of the key point M c Continue to enter step S302 to calculate the priority index of each marked key point and set the predicted spatial position until all marked key points are unmarked.

10. The motion capture management system based on artificial intelligence according to claim 9, characterized in that: The visualization module is used to update the detection images in real time, and to build a virtual mapping model based on the spatial position or predicted spatial position of each key point of the target object in each detection image, and to display it dynamically on the visualization screen.

Images