Joint motion back-stepping shooting intention mode recognition method

Abstract

The invention relates to the technical field of pattern classification, in particular to a joint motion backstepping shooting intention pattern recognition method, which comprises the following steps of collecting joint node tracks, extracting collaborative path features, fitting a reference path line, and recognizing sample time sequence changes to judge a shooting intention. According to the method, the space structure of the forearm vector is constructed in the continuous frames, the direction continuity criterion is established, the main path information in the forearm rotation track is accurately extracted, the included angle consistency judgment of the shoulder, elbow and wrist paths is matched, and the multi-path collaborative space combination is constructed; after a skeleton offset reference standard is established by means of a linear fitting method of node trajectories, the consistency performance of subsequent samples in spatial offset and direction projection can be quantified, the accurate judgment ability of path trend consistency and time sequence evolution relation can be improved, the recognition stability and judgment reliability of shooting intentions are enhanced, and the shooting intention recognition accuracy is improved. The motion interference caused by attitude change is effectively overcome, and the extraction precision of complex motion intentions is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern classification, in particular to a joint motion reverse push shot intention pattern recognition method. BACKGROUND

[0002] The technical field of pattern classification relates to the research on the identification, classification and judgment of data patterns by computers, including the establishment of mathematical models or the training of identification models to realize the automatic discrimination and classification of patterns in image, sound, action and other data, and the systematic identification and classification of data patterns through sample training, parameter setting and similarity measurement methods. Among them, the joint motion reverse push shot intention pattern recognition method refers to obtaining the dynamic motion information of a series of joints of the human body before shooting, and then judging whether the moving subject has the intention of shooting. It is aimed at the pre-judgment of shooting behavior in sports intention recognition, usually adopts a time sequence analysis method based on key skeleton point trajectory sequence, and completes the recognition by constructing a multi-dimensional action template database and combining posture sequence classification rules according to the relative displacement relationship and speed change characteristics between different joints.

[0003] In the existing shooting intention recognition process, classification and judgment are mainly based on the relative displacement and speed change between joints. When the motion change amplitude is small or the path trend is not obvious, similar trajectory misjudgment or posture fragment matching error may occur. When the moving subject performs non-shooting actions but has local postures close to each other, the system may produce deviation in intention recognition. The time sequence template library relied on lacks recognition ability in dealing with the multi-path differentiation trend in continuous motion, resulting in the problem of motion recognition confusion in diversified shooting preparation actions or fake action situations, reducing the adaptation range and judgment accuracy in actual motion discrimination scenarios. SUMMARY

[0004] The purpose of the present application is to solve the shortcomings in the prior art, and a joint motion reverse push shot intention pattern recognition method is proposed.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: a joint motion reverse push shot intention pattern recognition method, comprising the following steps: S1: obtaining the spatial position sequence of the elbow and wrist nodes of the moving person in the continuous motion frame, constructing a unit direction vector sequence, judging the angle change trend between each pair of adjacent vectors, screening the vector continuous segment with continuous direction change, and determining the main path according to the spatial trend to obtain the forearm rotation trajectory structure; S2: according to the main path direction vector in the forearm rotation trajectory structure, judging the direction angle of the shoulder to elbow path and the elbow to wrist path, screening the path segment with consistent direction and consistent trend, integrating the continuity information between the path segments, and obtaining the joint path coordination segment; S3: According to the shoulder, elbow, wrist node position sequence in the joint path cooperative segment, the spatial coordinate point set corresponding to the continuous time sequence of each node is extracted, a reference path line penetrating through the path set is constructed, the spatial offset of each node to the reference path line is calculated, and the skeleton path fitting reference data is obtained; S4: According to the skeleton path fitting reference data, the point set of the subsequent node path sample is subjected to spatial offset and direction projection judgment, the path segment consistent with the direction of the reference path line is screened, and uniform marking processing is performed on the path segment, so as to obtain the sample path intention set.

[0006] As a further scheme of the application, the forearm rotation trajectory structure includes a main path direction, a direction change trend record, and a vector continuous segment number, the joint path cooperative segment includes a path angle relationship, a spatial direction consistency, and a path segment connection feature, the skeleton path fitting reference data includes a node spatial offset, a fitting path line direction, and an error distribution characteristic, and the sample path intention set includes a path direction projection characteristic, a spatial offset mode, and a marking category information.

[0007] As a further scheme of the application, the specific acquisition steps of the forearm rotation trajectory structure are as follows: S111: The spatial position sequence of the elbow and wrist nodes in the continuous motion frame is obtained, and the unit direction vector sequence is obtained according to the change of the spatial position between adjacent frames; S112: According to the unit direction vector sequence, the direction change trend between any two adjacent vectors is judged, the vector segment with continuous direction change is screened, and the vector continuous segment is obtained; S113: According to the vector continuous segment, the spatial direction information of each segment is extracted, the spatial direction sequence of all segments is integrated, and the forearm rotation trajectory structure is obtained.

[0008] As a further scheme of the application, the specific acquisition steps of the joint path cooperative segment are as follows: S211: According to the main path direction vector in the forearm rotation trajectory structure, the direction vector of each continuous segment in the shoulder-to-elbow path and the elbow-to-wrist path is obtained respectively, the direction feature of each segment path is extracted, and the direction consistency with the main path direction vector is judged, the path segments consistent in direction are identified by comparing whether the spatial direction is within the allowed deviation range, and the path direction judgment result is generated; S212: Based on the path direction judgment result, the segments consistent in direction in the shoulder-to-elbow path segment and the elbow-to-wrist path segment are matched, the frame number and time sequence information of the start and end nodes of each segment are identified, the combination relationship between the two path segments in time and direction is identified, the segment sequence with continuous direction association is screened, and the path continuous segment information is obtained; S213: According to the path segment information, the connection state of each continuous segment in the spatial structure is extracted, the distribution characteristics in the motion sequence are arranged, the combination relationship between the shoulder-to-elbow path segment and the elbow-to-wrist path segment is established, the combination state between each path segment pair is summarized through unified marking, and the joint path coordination segment is obtained.

[0009] As a further scheme of the present application, the specific acquisition step of the skeleton path fitting reference data is: S311: According to the path sequence of the shoulder, elbow and wrist nodes in the joint path coordination segment, the spatial coordinate data under the corresponding time frame is extracted, the three-dimensional position coordinates of the three nodes are recorded in each frame, and the node position set is constructed in time sequence form, the data is grouped and arranged according to the node type and frame number, and the node trajectory set data is generated; S312: Based on the node trajectory set data, the connection line segment with continuity is selected as the path sample set in all path segment sets, the spatial straight line penetrating the whole node distribution is established as the fitting reference path according to the position coordinates of all node connection lines in the three-dimensional space, and the spatial reference direction is constructed along the minimum direction deviation to obtain the reference path line direction data; S313: According to the reference path line direction data, the shortest spatial distance from each node frame position point to the path line is extracted, the offset vector sequence corresponding to each node is constructed, and the spatial offset trend of the node distribution point relative to the fitting path is judged, the distance value of each node to the fitting path is obtained by operation, and the skeleton path fitting reference data is established.

[0010] As a further scheme of the present application, the specific acquisition step of the sample path intention set is: S411: According to the skeleton path fitting reference data, the spatial coordinate point set of the subsequent node path sample is acquired, the positions of the three nodes of shoulder, elbow and wrist are extracted in each frame, and a continuous path point sequence is formed in time sequence, the point cloud set of all nodes is matched with the fitting path reference data, a unified coordinate index structure is established, and the sample node point set structure is obtained; S412: Based on the sample node point set structure, the direction vector of each path segment is extracted, and the direction consistency with the direction vector of the reference path line is judged, according to the preset direction consistency threshold, the path segments consistent with the direction of the reference path line are screened, the path segments meeting the conditions are marked and recorded, and the path direction consistent segment information is obtained. S413: According to the path direction consistent segment information, all path segments meeting the conditions are uniformly marked and processed, the frame number range, start and end position and direction attribute of each path segment are integrated, the path segment identification system is established, the set structure which can be used for subsequent trajectory classification and determination is formed, and the sample path intention set is obtained.

[0011] As a further scheme of the present application, the method further comprises: S5: According to the node arrangement order of each path in the sample path intention set, the time sequence structure of the shoulder-elbow and elbow-wrist path segments is compared in turn, whether the node arrangement keeps a continuous one-way change trend is recorded, the paths meeting the time progression relationship are identified, and a shot intention recognition result is obtained; The shot intention recognition result includes node arrangement order features, path time sequence continuity records, and action trend judgment indexes.

[0012] As a further scheme of the present application, the specific acquisition steps of the shot intention recognition result are: S511: According to the path identification in the sample path intention set, the time sequence index of the shoulder, elbow and wrist nodes in each path segment is extracted, the node arrangement structure of the shoulder-elbow and elbow-wrist two path segments is obtained in turn according to the frame order, the node three-dimensional coordinates are corresponded with the time sequence, the change trend of each node in the continuous frames is identified, the adjacent node sequence from shoulder to elbow and from elbow to wrist is compared, the nodes with increasing time index and continuous spatial displacement are screened, and a node time sequence matrix data is formed; S512: Based on the node time sequence matrix data, for the shoulder-elbow and elbow-wrist two path segments, the time progression relationship of the two path segments is compared based on the time sequence of each frame node position, whether the time order of adjacent nodes keeps consistency is checked, the time change mode of continuous frames is compared, the path segment with a stable change trend in the time dimension is identified, the path segment meeting the time order requirement is marked and stored, and a time progression trend path set is obtained; S513: According to the time progression trend path set, the node arrangement continuity of the shoulder-elbow and elbow-wrist path segments is verified, the path segments not keeping a one-way change trend are screened out, only the path segments meeting the requirements of time sequence and spatial continuity are reserved, the path identification is correspondingly mapped with the path identification in the original sample path intention set, the effective path segment sequence is integrated, and a shot intention recognition result is generated.

[0013] Compared with the prior art, the present application has the advantages and positive effects that: In the application, by constructing the spatial structure of the forearm vector in the continuous frame and establishing the direction continuity criterion, the main path information in the forearm rotation trajectory is accurately extracted, the spatial combination of multiple paths is constructed by cooperating with the angle consistency judgment of the shoulder, elbow and wrist paths, and after the linear fitting method of the node trajectory is relied on to establish the skeleton offset reference standard, the consistency performance of the subsequent samples in the spatial offset and direction projection can be quantified, by comparing and identifying the time sequence characteristics of the path node sequence, the accurate discrimination ability of the path trend consistency and the time sequence evolution relationship can be improved, the recognition stability and judgment reliability of the shooting intention are enhanced, the motion interference caused by the posture change is effectively overcome and the extraction accuracy of the complex motion intention is improved. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 The main step flowchart of the application is shown in the figure. Figure 2 The forearm rotation trajectory structure acquisition flowchart of the application is shown in the figure. Figure 3 The joint path collaborative segment acquisition flowchart of the application is shown in the figure. Figure 4 The skeleton path fitting reference data acquisition flowchart of the application is shown in the figure. Figure 5 The sample path intention set acquisition flowchart of the application is shown in the figure. Figure 6 The shooting intention recognition result acquisition flowchart of the application is shown in the figure. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0016] In the description of the application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the application and simplify the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application. In addition, in the description of the application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0017] Please refer to Figure 1 , the joint motion backstepping shooting intention mode recognition method comprises the following steps: S1: Obtain the spatial position sequence of the elbow and wrist nodes of the moving person in the continuous motion frames, construct a unit direction vector sequence according to the spatial position change between adjacent frames, judge the angle change trend between each pair of adjacent vectors, screen the vector continuous section with continuous direction change, determine the main path according to the spatial trend of the vector continuous section, and obtain the forearm rotation trajectory structure; S2: According to the direction vector of the main path in the forearm rotation trajectory structure, judge the direction angle of the shoulder to elbow path and the elbow to wrist path, screen the path section with consistent direction and consistent trend, integrate the continuity information between the path sections, mark the spatial combination with consistent trend between the paths, and obtain the joint path collaborative section; S3: According to the position sequence of the shoulder, elbow and wrist nodes in the joint path collaborative section, extract the spatial coordinate point set corresponding to the continuous time sequence of each node, establish the path set of the node connection section and construct the reference path line (using the least square method linear fitting, and finding the best fitting straight line by minimizing the error sum of squares) through the path set, measure the spatial offset of each node to the reference path line, and obtain the skeleton path fitting reference data; S4: According to the skeleton path fitting reference data, judge the spatial offset and direction projection of the point set of the subsequent node path sample, screen the path section with consistent direction with the reference path line, and uniformly mark the path section to obtain the sample path intention set; S5: According to the node arrangement order of each path in the sample path intention set, sequentially compare the time sequence structure of the shoulder-elbow and elbow-wrist path sections, record whether the node arrangement maintains a continuous unidirectional change trend, and identify the path that meets the time progression relationship to obtain the shooting intention recognition result.

[0018] The forearm rotation trajectory structure includes the main path trend, direction change trend record and vector continuous section number, the joint path collaborative section includes the path angle relationship, spatial trend consistency and path section connection characteristics, the skeleton path fitting reference data includes the node spatial offset, fitting path line direction and error distribution characteristics, the sample path intention set includes the path direction projection characteristics, spatial offset mode and marked category information, and the shooting intention recognition result includes the node arrangement order characteristics, path time sequence continuity record and action trend judgment index.

[0019] Please refer to Figure 2 , the specific steps of S1 are: S111: Obtain the spatial position sequence of the elbow and wrist nodes in the continuous motion frames, and obtain the unit direction vector sequence according to the spatial position change between adjacent frames; In the process of acquiring the spatial position sequence of the elbow and wrist nodes in the continuous motion frames, first, the high-speed camera device is used to continuously capture 120 frames of image data per second. After calibration, the three-dimensional coordinate point information of the elbow and wrist is extracted. In the example, the position of the elbow in the first frame is (120.2 mm, 85.6 mm, 30.1 mm), and the position of the wrist in the same frame is (230.7 mm, 90.4 mm, 28.8 mm). In the second frame, they are (121.0 mm, 86.1 mm, 30.5 mm) and (232.0 mm, 91.0 mm, 29.3 mm), respectively. The node positions of the two frames are stored in array form for subsequent processing. At the same time, the spatial displacement difference between frames is calculated to obtain the displacement vector of the elbow and wrist between different frames. The displacement vector is obtained from the node coordinate difference. For example, the three-dimensional displacement of the wrist from the first frame to the second frame is (1.3 mm, 0.6 mm, 0.5 mm). In order to compare the spatial direction features between subsequent frames, the displacement vector of each adjacent frame needs to be normalized to a unit vector. The length of the unit vector is standardized to 1 for subsequent analysis of the spatial trend. For example, the displacement vector of the wrist from the first to the second frame (1.3, 0.6, 0.5) is normalized to obtain the corresponding unit direction vector (0.857, 0.395, 0.329). The same operation is applied to all nodes in the subsequent frame sequence. In 20 frames of data, the unit direction vector set between each frame can be obtained in sequence and form a sequence, and finally the unit direction vector sequence is obtained.

[0020] S112: According to the unit direction vector sequence, the change trend of the included angle between any two adjacent vectors is judged, and the vector segment with continuous direction change is selected to obtain the vector continuous segment. According to the unit direction vector sequence, the continuity of the direction change between adjacent vectors needs to be analyzed frame by frame. First, the included angle between adjacent vectors is calculated. The cosine value of the included angle can be obtained by the ratio of the dot product and the length of the vector. Then, the angle is converted to degrees by the inverse cosine function. For example, the dot product of the wrist direction vectors from the first frame to the second frame and from the second frame to the third frame is 0.92, so the corresponding included angle is 23.1°. Based on this, a sequence of included angle changes is established to analyze the direction smoothness. The vector segment with a continuous included angle change less than 30° is classified into the same paragraph. 30° is used as the direction continuity threshold based on experimental data of the human upper limb movement. Through actual testing in the motion experiment of 10 subjects, the statistical results show that the continuous included angle is mostly distributed between 10° and 28° under natural rotation, as shown in Table 1.

[0021] Table 1: Vector included angle experiment statistics

[0022] As shown in Table 1, when the included angle changes more than 30°, it is considered as a direction mutation, and no longer classified into the same continuous vector segment, so that 3 vector continuous segments satisfying the continuity condition are obtained in 20 frames of experimental data.

[0023] S113: According to the vector continuous segment, the spatial trend information of each segment is extracted, the spatial trend sequence of all segments is integrated, and the forearm rotation trajectory structure is obtained; According to the vector continuous segment, the spatial trend information of each segment is extracted and the spatial sequence is established for integration. First, the overall direction of each vector set in the three-dimensional space is determined. The average direction coordinates of all unit direction vectors of each segment are calculated to form a three-dimensional trend vector representing the main direction of each segment. For example, the average components of all unit direction vectors of the first vector continuous segment (frames 1 to 6) in X, Y and Z directions are (0.86, 0.42, 0.30), and similarly, the second segment is (0.82, 0.35, 0.46), and the third segment is (0.77, 0.50, 0.39). Arranging the three spatial trend vectors in frame sequence can obtain the spatial trend sequence. Combined with the frame sequence of the continuous segment and the time stamp of each segment, the complete trajectory of the forearm rotation in the three-dimensional space is finally reconstructed, and the forearm rotation trajectory structure is obtained.

[0024] Please refer to Figure 3 , the specific steps of S2 are: S211: According to the main path direction vector in the forearm rotation trajectory structure, the direction vectors of each continuous segment in the shoulder-to-elbow path and the elbow-to-wrist path are obtained respectively. The direction features of each path segment are extracted, and the direction consistency with the main path direction vector is judged. By comparing whether the spatial trend is within the allowed deviation range, the path segments that remain consistent in direction are identified, and the path direction judgment result is generated; Based on the main path direction vector in the forearm rotation trajectory structure, the spatial direction data of each continuous segment needs to be extracted from the shoulder to elbow and elbow to wrist paths. In this process, the path is disassembled through frame identification and spatial node coordinate recording. Each path segment is constructed with the starting frame and ending frame nodes to build the direction vector. For example, the direction vector of the 2nd to 4th frame segment in the shoulder to elbow path can be obtained by the difference between the 2nd frame position (110.2 mm, 75.6 mm, 34.2 mm) and the 4th frame position (115.4 mm, 80.3 mm, 36.1 mm) of the elbow. After unitization processing of the direction vector, the spatial angle is judged with the forearm main path direction vector. The direction consistency is judged by whether the included angle range is lower than the threshold value of 15°. The threshold value is derived from the average stable rotation angle experiment of 12 subjects. The angle deviation is calculated by the inverse cosine of the ratio of the dot product and the module length. For example, if the direction vector of a path segment is (0.86, 0.44, 0.24) and the main path direction is (0.89, 0.42, 0.21), the dot product is 1.0006, and the corresponding included angle is 9.2°, which meets the direction consistency condition. The threshold value of 15° is the average angle deviation tolerance value set by repeated experiments. The data is shown in Table 2. Table 2 Path direction consistency judgment threshold setting table

[0025] As shown in Table 2, the included angle between the shoulder and elbow segments of 10 of the 12 subjects in the experiment and the main path is less than 15°. Therefore, this angle is used as the basis for direction consistency judgment. Further, the multiple path segments that meet the conditions are identified based on this standard, and the path direction judgment result is obtained.

[0026] S212: Based on the path direction judgment result, the segments with consistent directions in the shoulder to elbow path segment and the elbow to wrist path segment are matched. Based on the frame number and time sequence information of the start and end nodes of each segment, the combination relationship between the two path segments that are continuous in time and consistent in direction is identified. The segment sequence with continuous and related segments is screened, and the path continuous segment information is obtained. After obtaining the path direction judgment result, the path segments meeting the direction consistency condition are arranged in two directions of shoulder to elbow and elbow to wrist respectively, and whether they constitute a continuous time segment is judged by comparing the time frame number sequences of adjacent path segments. The specific steps are as follows: the frame numbers of the shoulder-elbow path direction consistent segments are extracted, such as frame numbers 5 to 7, there is a segment corresponding to frame numbers 6 to 8 in the elbow-wrist path, which means that the two form a continuous overlapping area in frame time, and this is taken as a continuous segment candidate combination. For path segment pairs that partially exist frame jumps but have consistent directions, the maximum frame jump value is set to not more than 2 frames as an allowed window according to the actual delay range. If the elbow-wrist path segment is frame 9 to 11, and the shoulder-elbow segment is frame 6 to 8, the time difference is 1 frame, which can still be regarded as a continuous segment. The judgment basis is based on the allowed frame rate error range of 0.01 seconds to 0.02 seconds in the dynamic sampling process. This time window is derived from the frame rate jitter control result data statistics of different devices in the experiment, and then the path segments that constitute a reasonable continuation in time are retained and marked as combined valid paths. According to the number of continuously appearing valid path combinations, the combinations that constitute a stable continuous sequence in time frames are extracted, for example, there are direction consistent path combination segments in the 10th frame to the 16th frame in a continuous 7 frames in the motion process of a certain subject, which can be included in the continuous segment information structure, and the path continuous segment information is obtained.

[0027] S213: According to the path continuous segment information, the connection state of each continuous segment in the spatial structure is extracted, the distribution characteristics in the motion sequence are arranged, the combination relationship between the shoulder to elbow path segment and the elbow to wrist path segment which are consistent in spatial direction and time trend is established, and the combination state between each path segment pair is summarized through unified marking method, and the joint path coordination segment is obtained. According to the path continuous segment information, the connection relationship between each continuous segment in the spatial structure needs to be further combed, the starting node, the ending node and their spatial coordinates between the path segments are extracted, and the combination is arranged through the direction relationship and the time sequence. If there are multiple path segments that are consistent in spatial direction and have overlapping relationship between the starting node and the ending node, for example, path segment A starts at frame 5 and ends at frame 8, path segment B starts at frame 7 and ends at frame 10, there are time frame intersections frame 7 and frame 8, and the direction vector deviation is less than 10°, then they are merged into the same combination structure. In the combination structure construction process, the paragraph number, spatial trend information and frame time distribution are recorded, the combination identification of the segment pair structure is formed, each structure corresponds to a type of spatial combination label, and the label value is set according to the continuity of the spatial starting and ending points, the time sequence consistency and the direction matching score. The direction matching is taken as a high matching standard that the experimental standard angle deviation is less than 15° and the spatial deviation is within 5 mm. Finally, all the combination relationship structures are numbered, and the structures meeting the consistency marking standard are included in the coordination marking set to establish the joint path coordination segment.

[0028] Please refer to Figure 4The specific steps of S3 are as follows: S311: According to the path sequence of the shoulder, elbow and wrist nodes in the joint path cooperative segment, the spatial coordinate data in the corresponding time frame is extracted, the three-dimensional position coordinates of the three nodes are recorded in each frame, and the node position set is constructed in the form of time sequence. The data is grouped and arranged according to the node type and frame number, and the node trajectory set data is generated; According to the position sequence of the shoulder, elbow and wrist nodes in the joint path cooperative segment, the three-dimensional coordinate information corresponding to each node in the continuous frame sequence is first extracted. The shoulder node is represented by infrared marker point number P1, the elbow node is P2, and the wrist node is P3. The spatial position of each marker point in each frame is a three-dimensional coordinate set. For example, in the first frame, the P1 coordinates are (112.4, 64.8, 30.3), the P2 coordinates are (125.0, 71.2, 34.1), and the P3 coordinates are (138.6, 76.9, 36.5). This process extracts 300 frames of data in the entire measurement period. The frame number and corresponding coordinates of the nodes are recorded by index number, and a node-frame mapping matrix is formed. Then, a time sequence coordinate table is constructed. The coordinates of the same node in adjacent frames are connected in sequence to form a path segment set for each node. Each path group is composed of multiple point pairs. For example, the path segment of the P1 node can be generated by the space segment (frame 1)-(frame 2), (frame 2)-(frame 3), etc. The three-dimensional vector direction of such a segment can be directly calculated from the coordinate difference between adjacent frames. The sampling time between frames is set to 0.0083 seconds, i.e., 120Hz sampling rate. 120 coordinate points are recorded per second, forming a node segment set of about 890, and constructing a path structure table of start and end frame index, node type identification and three-dimensional spatial position data, as shown in Table 3.

[0029] Table 3 Node Path Structure Table

[0030] As shown in Table 3, by summarizing the node position pairs of all frames, the node trajectory set data used for subsequent fitting has been generated.

[0031] S312: Based on the node trajectory set data, select the connected line segments with continuity in all path segment sets as the path sample set. According to the position coordinates of all node connections in the path sample set in the three-dimensional space, establish a space straight line that penetrates the entire node distribution as a fitting reference path, and construct a spatial reference direction along the minimum direction deviation to obtain reference path line direction data; After calling the node trajectory set data, a representative path segment needs to be selected for spatial structure fitting. The midpoint of each node connecting segment is taken as a fitting reference sample point. The midpoint of the shoulder-elbow connecting segment is calculated as the average of the coordinates of the two ends. For example, P1 in the 20th frame is (120.0, 70.5, 31.8) and P2 is (130.4, 75.3, 35.2), and the midpoint is (125.2, 72.9, 33.5). The midpoint of the elbow-wrist connecting segment is obtained in the same way, which is (135.5, 78.4, 36.3). A total of about 580 sets of midpoint data are obtained in all frames. All the midpoints are arranged in time sequence to form a point cloud set, which is used to identify the distribution trend in three-dimensional space. Then, a straight line is drawn through all the sample points based on the midpoint coordinate set, which is used to represent the main direction of the overall node activity. The direction line needs to meet the principle of maximizing the fitting direction of the point cloud. Therefore, the principal component analysis (PCA) method is used to process the point cloud set composed of all the midpoints. The direction of the first principal component is taken as the direction vector of the reference path line. Finally, the two key parameters of the starting point of the direction line and the direction vector are generated, which are used for subsequent offset calculation to obtain the reference path line direction data.

[0032] S313: According to the reference path line direction data, the shortest spatial distance of each node interframe position point to the path line is extracted, the corresponding offset vector sequence of each node is constructed, and the spatial offset trend of the node distribution point relative to the fitted path is judged. The formula is: ; The distance values of all frames of each node to the fitted path are obtained by operation, and the skeleton path fitting reference data is established by integration. Wherein, represents the offset value of the node in the frame, is the node spatial coordinate in the frame, and are the starting and ending point coordinates of the reference path line at the position of the frame, is the length of the path line on the plane; According to the reference path line direction data, the spatial position of each node in each frame needs to be extracted, and the minimum spatial offset relative to the reference path line is determined. First, the path position of the reference path line in the current frame is identified. By setting the spatial parameter equation of the path line, the perpendicular projection point of the node to the straight line is established at the time point corresponding to the node in the current frame. The foot point is taken as the fitting reference. Then, the spatial offset vector is constructed between the node three-dimensional coordinate and the foot point on the reference line. The Euclidean distance between them is taken as the offset distance value. For example, in the 28th frame, the actual spatial coordinate of the shoulder node P1 is , and the position of the corresponding point on the reference path line is the starting point , and the direction vector is set as In this direction, the space straight line equation is constructed, and the current node point position is substituted into the space point to the straight line distance formula to calculate its offset value. Each parameter is substituted into the formula: , , ; , , ; , The direction unit vector direction is 1.0:0.5; Direction component difference: , ; The parameter equation of the reference path line is calculated, if the starting point of the path line is (xo, yo, Zo), the direction vector is (a, b, c), then the coordinates of a point on the line are (xo+at, yo+bt, zo+ct), by projecting the node (xi, yi) onto the XY plane of the straight line t value, that is, z(ti)=Zo+ct, the reference value of the Z direction is 31.5, which is adjusted with the linear change of the node in the main direction path length, and is strictly coupled with the slope of the direction vector, to reflect the fitting positioning of the height position.

[0033] Then the distance of the plane part is: ; The height direction offset part is: ; The final offset value is: ; The spatial offset value of this node in this frame is 2.324mm, which belongs to the range of offset value exceeding the reference threshold value, the offset value reference interval is set to [0mm, 2.0mm], and the setting basis is the statistical result of the node position offset amount concentrated distribution interval of the subjects in the shoulder, elbow and wrist cooperative movement of the subjects in the natural barrier-free state in multiple experiments. Specifically, 9 subjects, 30 frames of motion, the node offset value is measured according to the Euclidean distance, and the offset value of each frame is accumulated and counted according to the node category, and the average offset value is concentrated between 1.2mm and 1.8mm, and the standard deviation is less than 0.5mm. Therefore, 2.0mm is set as the upper limit of the normal fluctuation interval, which may be slightly expanded when the node motion speed rises or the direction mutation frequency increases, but remains stable within the standard step frequency range (0.6Hz to 1.2Hz).

[0034] During the whole measurement process, the offset of all frames and three nodes is processed frame by frame, and the offset value list of each frame node is obtained, and the maximum offset, minimum offset and mean value of the node appearing in all frames are counted. If the offset direction (such as X direction) of a node continuously increases or continuously decreases in more than 5 consecutive frames, it is marked as an offset trend continuation area. The trend is judged by the consistent section of the sign sequence of the offset value, and the frame segment length is counted, and the offset direction is marked. In addition, the outlier point of the offset value of all frames is determined. If the offset value is more than 4.5mm, it is marked as an outlier in the statistics. If the proportion of all outliers is more than 10% of the total number of points, it is warned as an abnormal trajectory segment. The offset data statistical results will be used for motion evaluation reference, and the skeleton path fitting reference data is finally established.

[0035] In the formula, the first part represents the two-dimensional vector cross product absolute value of the node position and the reference path line starting point and the direction vector The geometric meaning is the vertical distance from the node to the path line in the plane multiplied by the length of the path line. By dividing by , the length of the path line is normalized, so that this part of the result directly represents the minimum offset distance of the node to the path line in the XY plane. The second part reflects the height offset of the node in the Z direction. The calculation method is the absolute value of the difference between the node Z coordinate and the estimated height of the reference path line at the corresponding position of this frame , so as to ensure that the offset value is positive no matter whether the node is above or below the path line. The denominator is 1 in order to maintain dimensional consistency, which is convenient for subsequent direct addition of the plane offset.

[0036] Finally, by directly adding the plane offset and the height offset, the comprehensive spatial offset value of the node relative to the reference path line is obtained.

[0037] Please refer to Figure 5 , the specific steps of S4 are: S411: According to the skeleton path fitting reference data, obtain the spatial coordinate point set of the subsequent node path sample, extract the positions of the shoulder, elbow and wrist nodes in each frame, and form a continuous path point sequence in time sequence. Match the point cloud set of all nodes with the fitting path reference data to establish a unified coordinate index structure, and obtain the sample node point set structure; According to the skeleton path fitting reference data, first, the spatial coordinate point set of the subsequent node path sample is obtained, in this process, the three-dimensional coordinates of the shoulder, elbow and wrist nodes are extracted from each frame, and they are arranged in time sequence by frame number to form a complete path point sequence, after all the path sample points are uniformly arranged, an index structure containing nodes, frame numbers, spatial coordinates and corresponding time stamps is constructed, on this basis, each node sample point is matched with the reference path, the position deviation of the three-dimensional coordinate position of all points is corrected to ensure the accuracy of the data, for example, if the Euclidean distance between the node coordinates in a frame and the corresponding point of the fitted reference path is greater than 2.0mm, it is marked as an offset abnormal point and excluded in the subsequent calculation, through this method, a consistent quality path sample point set is formed, and finally a classified, time-continuous and traceable point cloud data structure is generated, which is used for subsequent path judgment and screening to obtain the sample node point set structure.

[0038] Table 4 Node sample point space coordinate table

[0039] As shown in Table 4, the node sample points maintain continuity in time sequence, ensuring the formation of a complete three-dimensional path trajectory data structure, providing basic data support for subsequent direction judgment.

[0040] S412: Based on the sample node point set structure, the direction vector of each path segment is extracted, and the direction consistency with the direction vector of the reference path line is judged, according to the preset direction consistency threshold, the path segments consistent with the direction of the reference path line are screened out, and the path segments meeting the conditions are marked and recorded to obtain the path direction consistent segment information; The sample node point set structure is called, the direction vector of each path segment is extracted, and the consistency with the reference path line direction vector is judged, first, the node coordinates of the continuous frames in each path segment are obtained, the direction vector set of the path segment is constructed by using the spatial position difference between frames, and then it is compared with the reference path direction vector segment by segment, in this process, the direction consistency threshold is set to be the included angle ≤15°, which is determined by experimental statistics, for example, in the dynamic sample data of 12 subjects, the direction deviation of the shoulder to wrist path segment is 11.2°, and the standard deviation is 2.6°, 15° can cover more than 90% of the normal path segments, the path segments with direction deviation less than the threshold are classified into the effective path segment set, and the corresponding frame interval and direction data are recorded.

[0041] Table 5 Path segment direction consistency judgment table

[0042] As shown in Table 5, the path segments highly consistent with the reference path direction can be screened out through direction consistency judgment, which provides a basic condition for subsequent path segment marking and integration, and the path direction consistent segment information is obtained.

[0043] S413: According to the path direction consistent segment information, all qualified path segments are uniformly marked, the frame number range, start and end position and direction attribute of each path segment are integrated, a path segment identification system is established, a collection structure is formed which can be used for subsequent trajectory classification and determination, and a sample path intention collection is obtained; According to the path direction consistent segment information, all qualified path segments are uniformly marked, the frame number range, start and end position and direction attribute of each path segment are integrated, a path segment identification system is established, a collection structure is formed which can be used for subsequent trajectory classification and determination, and a sample path intention collection is obtained;

[0044] Table 6 sample path intention marking table

[0045] As shown in Table 6, all path segments consistent with the reference path direction are uniformly marked, and the final identification system can be directly used for sample action classification and path intention recognition, and a sample path intention collection is obtained.

[0046] Please refer to Figure 6 , the specific steps of S5 are: S511: According to the path identification in the sample path intention collection, the time sequence index of the shoulder, elbow and wrist nodes in each path segment is extracted, the node arrangement structure of the shoulder-elbow and elbow-wrist two path segments is obtained in sequence according to the frame order, the three-dimensional coordinates of the nodes are corresponded with the time sequence, the change trend of each node in the continuous frame is marked, the adjacent node sequence of the shoulder to the elbow and the elbow to the wrist is compared, the nodes with increasing time index and continuous spatial displacement are screened, and a node time sequence matrix data is formed; According to the path identifier in the sample path intention set, the three-dimensional coordinates of the shoulder, elbow, and wrist nodes of each path segment and the corresponding frame number are first extracted, and a complete time sequence index structure is established by combining the original timestamp data collected by the high-speed camera equipment, to ensure that the spatial position and time position of each frame node are accurately corresponding. In the shooting scene, assuming that data collection is performed on 10 testers in a continuous shooting experiment, the sampling rate of each action is 120 fps, and the total number of frames is about 120. By reading the shoulder, elbow, and wrist node coordinates in the frame sequence, an initial node point cloud dataset is established. Next, according to the spatial continuity requirement of adjacent nodes, the three-dimensional distance between the shoulder-elbow and elbow-wrist in the same frame is detected. If the distance between adjacent nodes exceeds the reference threshold, the frame is marked as an abnormal frame and is excluded. The threshold is set based on statistical experimental data. Taking the spatial distance measurement of the shoulder-elbow and elbow-wrist segments of the 10 testers during shooting as an example, the inter-frame average displacement of the shoulder-elbow segment is between 8.2 mm and 13.6 mm, and the inter-frame average displacement of the elbow-wrist segment is between 10.1 mm and 17.4 mm. Through analysis, the effective threshold range under normal circumstances is 5 mm to 15 mm for the shoulder-elbow segment and 8 mm to 18 mm for the elbow-wrist segment, as shown in Table 7. In a typical shooting sample, the x, y, and z coordinates of the shoulder node of frame numbers 1 to 10 are between (210, 1300, 1450) and (225, 1315, 1470), the elbow node is between (330, 1105, 980) and (340, 1120, 1000), and the wrist node is between (460, 980, 620) and (475, 1000, 640). Through spatial calculation verification, the distance change of the consecutive frames all conforms to the threshold range. The qualified nodes are retained and arranged in time sequence to generate a three-dimensional coordinate matrix, forming a node time sequence matrix data.

[0047] Table 7: Reference threshold table for node displacement in shooting samples

[0048] As shown in Table 7, the threshold range is calculated from the measured samples. By excluding abnormal frames, the continuity and data validity of the time sequence matrix in subsequent analysis are ensured, and finally the node time sequence matrix data is obtained.

[0049] S512: Based on the node time sequence matrix data, for the shoulder-elbow and elbow-wrist two-path segments, the time progression relationship of the two-path segments is compared based on the time sequence of each frame node position, the time sequence of adjacent nodes is checked for consistency, the change trend of the path segment that is stable in the time dimension is identified by comparing the time change pattern of consecutive frames, the path segment that meets the time sequence requirement is marked and stored, and a time progression trend path set is obtained. The node timing matrix data is called, the timestamp sequence of each frame is extracted for the shoulder-elbow and elbow-wrist two path segments in turn, and each frame acquisition time is bound with the corresponding three-dimensional node coordinates to construct a path time sequence set. In the specific experiment, the sampling rate is 120 fps, and the frame acquisition interval is about 4.17 ms. For example, in a shooting sample, the shoulder-elbow segment continuous frame time sequence is 0.000 s, 0.004 s, 0.008 s to 1.236 s, and the elbow-wrist segment time sequence is 0.016 s, 0.020 s, 0.024 s to 1.256 s. By detecting whether the time increment between adjacent frames is greater than 0, it can be judged whether the time sequence meets the continuity requirement of one-way increasing; if there is a time rollback or sampling loss between adjacent frames, for example, the time difference between the 58th frame and the 59th frame of the elbow-wrist segment is-0.012 s, it is marked as invalid and the frame is excluded. Then, the effective time intervals of the two path segments are compared and contrasted to ensure that the shoulder-elbow segment time interval is completely before or synchronized with the elbow-wrist segment. In a sample, the effective time interval of the shoulder-elbow segment is [0.000 s, 1.236 s], and the effective time interval of the elbow-wrist segment is [0.016 s, 1.256 s], and there is no time overlap between the two path segments, which meets the standard time evolution requirement of continuous shooting action. The time sequences of all effective path segments are uniformly marked and numbered to obtain a time progression trend path set for subsequent node trend analysis.

[0050] S513: According to the time progression trend path set, the node arrangement continuity of the shoulder-elbow and elbow-wrist path segments is verified, the path segments that do not maintain a one-way change trend are screened out, only the path segments that meet the requirements of time sequence and spatial continuity are reserved, the path identification in the original sample path intention set is correspondingly mapped, the effective path segment sequence is integrated, and a shooting intention recognition result is generated. According to the time progression trend path set, further integrate all shoulder-elbow and elbow-wrist path segments meeting the time sequence and spatial continuity requirements, map the effective path segments back to the original sample path intention set, and establish a unified identifier for each path segment. In specific operation, the start frame number, end frame number, start and end node three-dimensional coordinate range, and corresponding time interval of each path segment are extracted, for example, in sample A, the start and end frames of the shoulder-elbow segment are 12 to 64 frames, the time range is [0.050s, 1.020s], and the node coordinate range is between (210, 1280, 1440) and (260, 1360, 1500); the start and end frames of the elbow-wrist segment are 20 to 74 frames, the time range is [0.080s, 1.200s], and the coordinate range is between (340, 1120, 990) and (380, 1190, 1050). After numbering all effective path segments, combine the original sample path identifier to establish a path-to-time sequence mapping table, and realize consistent alignment across path segments. After data integration is completed, all path segments that do not meet the time progression or spatial continuity requirements are removed, high-quality trajectory data during the shooting process is retained, and finally the shooting intention recognition result that can be directly used for action pattern recognition is obtained.

[0051] The above is only a preferred embodiment of the present application, and does not limit the present application in other forms. Any skilled person in the art can modify or change the above disclosed technical content to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made according to the technical essence of the present application to the above embodiments without departing from the technical solution content of the present application still falls within the protection scope of the present application.

Claims

1. A method for identifying a motion-reversal shooting intention pattern, characterized in that, The method comprises the following steps: S1: obtaining a sequence of spatial positions of elbow and wrist nodes of a moving person in continuous motion frames, constructing a sequence of unit direction vectors, judging a change trend of an included angle between each pair of adjacent vectors, screening a vector continuous segment in which the direction change remains continuous, and determining a main path according to a spatial trend to obtain a forearm rotation trajectory structure; S2: judging a direction included angle of a shoulder-to-elbow path and an elbow-to-wrist path according to a direction vector of the main path in the forearm rotation trajectory structure, screening a path segment in which the direction is consistent and the trend is consistent, integrating continuity information between path segments, and obtaining a joint path cooperative segment; S3: extracting a set of spatial coordinates corresponding to a continuous time sequence of each node according to a sequence of positions of shoulder, elbow, and wrist nodes in the joint path cooperative segment, constructing a reference path line penetrating through the path set, measuring a spatial offset of each node to the reference path line, and obtaining skeleton path fitting reference data; S4: judging a spatial offset and a direction projection of a point set of a subsequent node path sample according to the skeleton path fitting reference data, screening a path segment in which the direction is consistent with the reference path line, uniformly marking the path segment, and obtaining a sample path intention set.

2. The method of pitch pattern recognition of claim 1, wherein, The forearm rotation trajectory structure comprises a main path trend, a direction change trend record, and a number of vector continuous segments, the joint path cooperative segment comprises a path included angle relationship, a spatial trend consistency, and a path segment connection feature, the skeleton path fitting reference data comprises a node spatial offset, a fitting path line direction, and an error distribution characteristic, and the sample path intention set comprises a path direction projection characteristic, a spatial offset mode, and marked category information.

3. The method of pitch pattern recognition of claim 1, wherein, The specific acquisition steps of the forearm rotation trajectory structure are as follows: S111: obtaining a sequence of spatial positions of elbow and wrist nodes in continuous motion frames, and obtaining a sequence of unit direction vectors according to a change in spatial positions between adjacent frames; S112: judging a change trend of an included angle between any two adjacent vectors according to the sequence of unit direction vectors, screening a vector segment in which the direction change remains continuous, and obtaining a vector continuous segment; S113: extracting spatial trend information of each segment according to the vector continuous segment, integrating a sequence of spatial trends of all segments, determining a main path from the sequence, and obtaining a forearm rotation trajectory structure.

4. The articulation reverse-pivot shot intention pattern recognition method of claim 1, wherein, The specific acquisition steps of the joint path cooperative segment are as follows: S211: obtaining direction vectors of each continuous segment in a shoulder-to-elbow path and an elbow-to-wrist path according to a direction vector of the main path in the forearm rotation trajectory structure, extracting a direction feature of each path segment, judging a direction consistency with the direction vector of the main path, identifying a path segment in which the direction remains consistent by comparing spatial trends of each continuous segment within an allowed deviation range, and generating a path direction judgment result; S212: matching a segment in which the direction is consistent in a shoulder-to-elbow path segment and an elbow-to-wrist path segment based on the path direction judgment result, identifying a combination relationship in which two path segments are continuous in time and consistent in direction based on frame number and time sequence information of start and end nodes of each segment, screening a segment sequence having a continuous trend association, and obtaining path continuous segment information; S213: According to the path segment information, the connection state of each continuous segment in the spatial structure is extracted, the distribution characteristics in the motion sequence are arranged, the combination relationship between the shoulder-to-elbow path segment and the elbow-to-wrist path segment is established, the combination state between each path segment pair is summarized through unified marking, and the joint path coordination segment is obtained.

5. The articulation reverse-pivot shot intention pattern recognition method of claim 1, wherein, The specific acquisition steps of the skeleton path fitting reference data are: S311: According to the path sequence of the shoulder, elbow and wrist nodes in the joint path coordination segment, the spatial coordinate data under the corresponding time frame is extracted, the three-dimensional position coordinates of the three nodes are recorded in each frame, and the node position set is constructed in time sequence form, the data is grouped and arranged according to the node type and frame number, and the node trajectory set data is generated; S312: Based on the node trajectory set data, select the connected line segment with continuity in all path segment sets as the path sample set, establish a straight line through the whole node distribution in the three-dimensional space as the fitting reference path according to the position coordinates of all node connecting lines in the path sample set, and construct a spatial reference direction along the minimum direction deviation to obtain the reference path line direction data; S313: According to the reference path line direction data, the shortest spatial distance from each node frame position point to the path line is extracted, the offset vector sequence corresponding to each node is constructed, and the spatial offset trend of the node distribution point relative to the fitting path is judged, the distance value of each node to the fitting path in all frames is calculated, and the skeleton path fitting reference data is established.

6. The articulation reverse-pivot shot intention pattern recognition method of claim 1, wherein, The specific acquisition steps of the sample path intention set are: S411: According to the skeleton path fitting reference data, the spatial coordinate point set of the subsequent node path sample is acquired, the positions of the three nodes of shoulder, elbow and wrist are extracted in each frame, and a continuous path point sequence is formed in time sequence, the point cloud set of all nodes is matched with the fitting path reference data, and a unified coordinate index structure is established to obtain the sample node point set structure; S412: Based on the sample node point set structure, the direction vector of each path segment is extracted, and the direction consistency with the direction vector of the reference path line is judged, according to the preset direction consistency threshold, the path segments consistent with the direction of the reference path line are selected, the path segments meeting the conditions are marked and recorded, and the path direction consistent segment information is obtained; S413: According to the path direction consistent segment information, all path segments meeting the conditions are uniformly marked and processed, the frame number range, start and end position and direction attribute of each path segment are integrated, the path segment identification system is established, and the sample path intention set is obtained.

7. The method of pitch pattern recognition of claim 1, wherein, The method further comprises: S5: According to the node arrangement order of each path in the sample path intention set, the time sequence structure of the shoulder-elbow and elbow-wrist path segments is compared in turn, whether the node arrangement maintains a continuous and single-direction change trend is recorded, and the path meeting the time progression relationship is identified to obtain the shooting intention recognition result; The shooting intention recognition result includes node arrangement order characteristics, path time sequence continuity record and motion trend judgment index.

8. The articulation reverse-pivot shot intent pattern recognition method of claim 7, wherein, The specific acquisition steps of the shooting intention recognition result are: S511: According to the path identification in the sample path intention set, the time sequence index of the shoulder, elbow and wrist nodes in each path segment is extracted, the node arrangement structure of the shoulder-elbow and elbow-wrist two segments is obtained in turn according to the frame order, the node three-dimensional coordinates are corresponded with the time sequence, the change trend of each node in the continuous frame is identified, the adjacent node sequence of the shoulder to the elbow and the elbow to the wrist is compared, the nodes with the time index increasing and the spatial displacement continuous are screened, and the node time sequence matrix data is formed; S512: Based on the node time sequence matrix data, for the shoulder-elbow and elbow-wrist two segments, the time progression relationship of the two segments is compared based on the time sequence of the node position of each frame, whether the time order of the adjacent nodes is consistent is checked, the time change mode of the continuous frames is compared, the path segment with the change rate in the time dimension remaining in the predetermined range is identified, the path segment meeting the time order requirement is marked and stored, and the time progression trend path set is obtained; S513: According to the time progression trend path set, the node arrangement continuity of the shoulder-elbow and elbow-wrist path segments is verified, the path segment not maintaining the one-way change trend is screened out, only the path segment meeting the requirements of the time sequence and the spatial continuity is reserved, the path segment sequence is integrated corresponding to the path identification in the original sample path intention set, and the shot intention recognition result is generated.

Images