Motion signal processing method and apparatus, computer device and readable storage medium

Abstract

The application discloses a motion signal processing method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a motion signal sequence of a target user, and acquiring a sequence balance position of the motion signal sequence; determining a sequence zero-crossing point of the motion signal sequence according to the sequence balance position, segmenting the motion signal sequence based on the sequence zero-crossing point, and obtaining a sequence period of the motion signal sequence; generating a sequence feature vector of the motion signal sequence according to the sequence period, and predicting a target motion result of the target user according to the sequence feature vector. By using the application, the accuracy and efficiency of obtaining the target motion result of the target user can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more particularly to a motion signal processing method, apparatus, computer device, and readable storage medium. Background Technology

[0002] In many scenarios, there is a need to analyze a user's motion signals, such as analyzing a user's motion speed signals or analyzing a user's motion amplitude signals.

[0003] Current technology typically allows for direct detection (e.g., via sensors) of a user's motion speed and amplitude signals. These signals can then be manually analyzed by staff. For example, analysis of motion speed might determine the stability of the user's speed, while analysis of motion amplitude might determine the stability of the user's amplitude. However, because different staff members may use different standards for analyzing user motion signals, the results are generally inaccurate and require significant human resources. Summary of the Invention

[0004] This application provides a motion signal processing method, apparatus, computer device, and readable storage medium, which can improve the accuracy and efficiency of acquiring target motion results from a target user.

[0005] This application provides a motion signal processing method, including:

[0006] Obtain the motion signal sequence of the target user and the sequence equilibrium position of the motion signal sequence;

[0007] The zero-crossing point of the motion signal sequence is determined based on the equilibrium position of the sequence. The motion signal sequence is then segmented based on the zero-crossing point to obtain the sequence period of the motion signal sequence.

[0008] Generate a sequence feature vector of the motion signal sequence based on the sequence period, and predict the target motion result of the target user based on the sequence feature vector.

[0009] The acquisition of the target user's motion signal sequence includes:

[0010] Acquire sensor motion parameters of the target user during movement;

[0011] Determine the motion characteristic parameters of the target user based on the sensor motion parameters;

[0012] Generate motion signal sequences for the target user based on motion characteristic parameters.

[0013] This application provides a motion signal processing device, comprising:

[0014] The equilibrium position acquisition module is used to acquire the motion signal sequence of the target user and obtain the sequence equilibrium position of the motion signal sequence.

[0015] The period segmentation module is used to determine the zero-crossing point of the motion signal sequence based on the sequence equilibrium position, and to segment the motion signal sequence based on the zero-crossing point to obtain the sequence period of the motion signal sequence.

[0016] The motion assessment module is used to generate a sequence feature vector of the motion signal sequence based on the sequence period, and to predict the target motion result of the target user based on the sequence feature vector.

[0017] The equilibrium position acquisition module includes:

[0018] The extreme value detection unit is used to acquire the smoothed signal sequence of the motion signal sequence and detect the extreme values ​​of the smoothed signal sequence.

[0019] The balance line determination unit is used to determine the balance line of a motion signal sequence based on the sequence extreme values.

[0020] The position determination unit is used to determine the position of the equilibrium line as the equilibrium position of the sequence.

[0021] The extreme value detection unit includes:

[0022] The frequency domain transformation subunit is used to transform a motion signal sequence in the time domain to the frequency domain to obtain the frequency domain motion signal of the motion signal sequence;

[0023] The frequency filtering subunit is used to obtain the user's motion frequency threshold, and filter the frequency domain motion signal based on the user's motion frequency threshold to obtain a smooth motion signal in the frequency domain.

[0024] The time-domain transformation subunit is used to transform the smoothed motion signal to the time domain space to obtain a smoothed signal sequence of the motion signal sequence.

[0025] The smooth signal sequence includes sampled values ​​corresponding to at least two sampling points; the at least two sampling points include sampling point nl, sampling point n, and sampling point n+l, where n is a positive integer less than or equal to the total number of at least two sampling points, and l is the sampling point interval between sampling point nl and sampling point n+l respectively; the sequence extrema include the sequence maximum and the sequence minimum.

[0026] The extreme value detection unit includes:

[0027] The matrix acquisition sub-unit is used to acquire the difference matrix corresponding to at least two sampling points. Here, l is a positive integer less than or equal to the total number of rows in the difference matrix. The l-th row of the difference matrix includes element values ​​mapped by the sampling difference values ​​corresponding to sampling point nl and sampling point n+l. The sampling difference value corresponding to sampling point nl refers to the difference between the sampled value at sampling point nl and the sampled value at sampling point n, and the sampling difference value corresponding to sampling point n+l refers to the difference between the sampled value at sampling point n+l and the sampled value at sampling point n. When both the sampling difference value corresponding to sampling point nl and sampling point n+l are less than the difference boundary value, the element value is the first element value; when both the sampling difference value corresponding to sampling point nl and sampling point n+l are greater than the difference boundary value, the element value is the second element value.

[0028] The target row is determined by counting the number of first elements in each row of the difference matrix containing the first element value. The row with the most first elements is determined as the target row k1. The second element in each row of the difference matrix is ​​counted by counting the number of second elements in each row containing the second element value. The row with the most second elements is determined as the target row k2. k1 and k2 are both positive integers less than or equal to the total number of rows in the difference matrix.

[0029] The extreme value determination subunit is used to determine the sampled value corresponding to sampling point n as the maximum value of the sequence when the first element value to the k1st element value in the nth column of the difference matrix are all the first element value, and to determine the sampled value corresponding to sampling point n as the minimum value of the sequence when the first element value to the k2th element value in the nth column of the difference matrix are all the second element value.

[0030] Among them, sequence extrema include sequence maxima and sequence minima;

[0031] The equilibrium line determination unit includes:

[0032] The median acquisition sub-unit is used to obtain the median of the maximum values ​​of the sequence maxima and the median of the minimum values ​​of the sequence minima;

[0033] The balance line determination sub-unit is used to determine the balance line of a motion signal sequence based on the median of the maximum and minimum values.

[0034] The periodic segmentation module includes:

[0035] The signal wave acquisition unit is used to acquire the motion signal wave of a smoothed signal sequence of a motion signal sequence;

[0036] The zero-crossing point acquisition unit is used to acquire the intersection point between the equilibrium line and the motion signal wave, which serves as the zero-crossing point of the sequence.

[0037] The sequence must have at least two zero-crossing points.

[0038] The periodic segmentation module includes:

[0039] A frequency detection unit is used to obtain a sampling frequency threshold and, based on the sampling frequency threshold, sequentially detect the interval frequency between adjacent zero-crossing points of at least two sequences.

[0040] An anomaly determination unit is used to determine the zero-crossing points of adjacent sequences whose interval frequency is less than the sampling frequency threshold among the detected at least two sequence zero-crossing points as abnormal sequence zero-crossing points.

[0041] The correction determination unit is used to filter out abnormal sequence zero-crossings from at least two sequence zero-crossings to obtain corrected sequence zero-crossings;

[0042] The correction segmentation unit is used to segment the smoothed signal sequence based on the zero-crossing point of the correction sequence to obtain the sequence period.

[0043] The exercise assessment module includes:

[0044] The information acquisition unit is used to acquire a motion evaluation standard table and motion prediction types for target users; the motion evaluation standard table includes the correlation between at least two motion prediction types and their corresponding motion feature types.

[0045] The feature type determination unit is used to determine the motion feature types that are related to the motion prediction type of the target user in the motion evaluation standard table as the detection feature types for the target user;

[0046] The vector generation unit is used to generate signal feature parameters of the motion signal sequence under the detection feature type based on the sequence period, and to generate sequence feature vectors based on the signal feature parameters.

[0047] The number of detection feature types is at least two;

[0048] Vector generation unit, including:

[0049] The period information acquisition subunit is used to obtain the period duration and period amplitude of the sequence period;

[0050] The feature parameter determination subunit is used to determine the signal feature parameters of the motion signal sequence under each detection feature type based on the period duration and period amplitude.

[0051] The feature splicing subunit is used to splice the signal feature parameters of the motion signal sequence under each detection feature type to obtain the sequence feature vector.

[0052] The equilibrium position acquisition module includes:

[0053] The key point determination unit is used to obtain the motion prediction type for the target user and determine the motion key points for the target user based on the motion prediction type.

[0054] The key information detection unit is used to acquire the motion video data of the target user and detect the key point motion information of the target user in the motion video data based on the motion key points;

[0055] The first sequence generation unit is used to generate a motion signal sequence of the target user based on key point motion information.

[0056] The equilibrium position acquisition module includes:

[0057] The sensor parameter acquisition unit is used to acquire the sensor motion parameters of the target user during the movement process;

[0058] The motion feature determination unit is used to determine the motion feature parameters of the target user based on the sensor motion parameters.

[0059] The second sequence generation unit is used to generate a motion signal sequence of the target user based on motion feature parameters.

[0060] The exercise assessment module includes:

[0061] The feature input unit is used to input the sequence feature vector into the motion prediction model; the motion prediction model is trained based on the sample sequence feature vector of the sample motion signal sequence.

[0062] The result output unit is used to output the target motion result for the target user in the motion prediction model.

[0063] The aforementioned device also includes:

[0064] The sample position acquisition module is used to acquire the sample motion signal sequence of the sample user and the actual motion result of the sample user, and to acquire the sample sequence equilibrium position of the sample motion signal sequence.

[0065] The sample period segmentation module is used to determine the zero-crossing point of the sample motion signal sequence based on the equilibrium position of the sample sequence, and to segment the sample motion signal sequence based on the zero-crossing point to obtain the sample period of the sample motion signal sequence.

[0066] The sample feature generation module is used to generate sample sequence feature vectors of sample motion signal sequences based on the sample sequence period.

[0067] The sample result output module is used to input the sample sequence feature vector and the actual motion result into the initial motion prediction model, and output the sample motion result for the sample motion signal sequence in the initial motion prediction model;

[0068] The parameter correction module is used to correct the model parameters of the initial motion prediction model based on the sample motion results and the actual motion results, so as to obtain the motion prediction model.

[0069] This application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the method of this application.

[0070] This application provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described above.

[0071] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative embodiments described above.

[0072] This application first acquires the motion signal sequence of the target user and obtains the sequence equilibrium position of the motion signal sequence. Next, it determines the sequence zero-crossing points of the motion signal sequence based on the sequence equilibrium position, and segments the motion signal sequence based on these zero-crossing points to obtain the sequence period. Furthermore, it generates a sequence feature vector of the motion signal sequence based on the sequence period, and predicts the target user's target motion result based on the sequence feature vector. Therefore, the method proposed in this application can periodically segment the motion signal sequence using the sequence equilibrium position. Since this sequence equilibrium position can evenly distribute the sampled values ​​in the motion signal sequence, it can accurately segment the period of the motion signal sequence. Furthermore, it can generate a sequence feature vector using the accurately segmented sequence period, thereby achieving accurate prediction of the target user's target motion result. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1This is a schematic diagram of a network architecture for motion assessment provided in an embodiment of this application;

[0075] Figure 2 This is a schematic diagram of a scenario for obtaining the motion result of a target, as provided in this application;

[0076] Figure 3 This is a flowchart illustrating a motion signal processing method provided in this application;

[0077] Figure 4 This is a schematic diagram of a scenario for acquiring motion key points provided in this application;

[0078] Figure 5 This is a schematic diagram illustrating a scenario for obtaining the period of a sequence, as provided in this application.

[0079] Figure 6 This is a schematic diagram of a scenario for predicting the motion result of a target, as provided in this application;

[0080] Figure 7 This is a schematic diagram of a scenario for obtaining target motion results provided in this application;

[0081] Figure 8 This is a flowchart illustrating a method for obtaining the balanced position of a sequence provided in this application;

[0082] Figure 9 This is a schematic diagram of a scenario for detecting sequence extrema provided in this application;

[0083] Figure 10 This is a schematic diagram of the structure of a motion signal processing device provided in this application;

[0084] Figure 11 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0085] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0086] This application relates to technologies related to artificial intelligence (AI). AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0087] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0088] This application primarily concerns machine learning within artificial intelligence. Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0089] The machine learning involved in this application mainly refers to how to train a motion prediction model to evaluate a user's movements. This motion prediction model can be applied to routine user motion assessment as well as to medical-assisted motion assessment. See below for details. Figure 3 The description in the corresponding embodiments.

[0090] Please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture for motion assessment provided in an embodiment of this application. Figure 1 As shown, the network architecture may include server 200 and a terminal device cluster. The terminal device cluster may include one or more terminal devices; the number of terminal devices is not limited here. Figure 1As shown, the multiple terminal devices may specifically include terminal device 100a, terminal device 101a, terminal device 102a, ..., terminal device 103a; as Figure 1 As shown, terminal devices 100a, 101a, 102a, ..., 103a can all connect to server 200 via the network, so that each terminal device can interact with server 200 via the network.

[0091] like Figure 1 The server 200 shown can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart TV, or other smart terminal. The following description uses the communication between terminal device 100a and server 200 as an example to illustrate the specific implementation of this application.

[0092] Please see also Figure 2 , Figure 2 This is a schematic diagram of a scenario for obtaining the motion result of a target, as provided in this application. Figure 2 As shown, terminal device 100a can be the terminal device of the target user, and the target user can be any user. The target user can upload their motion video data to server 200 through terminal device 100a to request server 200 to evaluate their motion based on the uploaded motion video data. Furthermore, server 200 can extract the target user's motion signal sequence 100c from the motion video data uploaded by terminal device 100a. The specific process of generating motion signal sequence 100c from motion video data can also be found in the following... Figure 3 The description in the corresponding embodiments.

[0093] Furthermore, the server 200 can filter out noise and interference signals in the motion signal sequence 100c to obtain a smoothed signal sequence 101c of the motion signal sequence 100c. This smoothed signal sequence can be a sequence in the time domain, therefore, the smoothed signal sequence can be made time continuous to obtain the signal wave 103c of the smoothed signal sequence.

[0094] Server 200 can detect the equilibrium position (which can be called the sequence equilibrium position) of the smoothed signal sequence 101c, which is the location of the equilibrium line 102c. The equilibrium line 102c can nearly uniformly divide the signal wave 103c into upper and lower parts. The specific process for obtaining the sequence equilibrium position of the equilibrium line of the smoothed signal sequence can be found below. Figure 8 The description in the corresponding embodiments.

[0095] Furthermore, the server 200 can perform periodic segmentation of the signal wave 103c using the balance line 102c. Periodic segmentation of the signal wave 103c is equivalent to periodic segmentation of the motion signal sequence or smoothed signal sequence, thereby obtaining one or more sequence periods 104c of the motion signal sequence (including, for example,...). Figure 2 The sequence includes one sequence period and the second sequence period. A sequence period can include two peaks and one trough, and two adjacent sequence periods can share a peak. The specific process of periodically dividing the signal wave 103c using the balance line 102c can be found below. Figure 3 The description in the corresponding embodiments.

[0096] Furthermore, server 200 can calculate the period duration 106c and period amplitude 105c of each sequence period 104c. Then, server 200 can generate a sequence feature vector 107c for the motion signal sequence 100c using this period duration 106c and period amplitude 105c. This sequence feature vector 107c is the final generated vector describing the motion features contained in the motion signal sequence 100c.

[0097] Server 200 can input the generated sequence feature vector 107c into motion prediction model 108c, which can then output a target motion result 109c for the target user. This target motion result can be a motion evaluation score. The motion prediction model 108c can be a pre-trained model used to predict the user's target motion result. The training process of motion prediction model 108c can be described below. Figure 3 The relevant descriptions in the corresponding embodiments.

[0098] Server 200 can return the predicted target motion result 109c for the target user to terminal device 100a, so that terminal device 100a can output the target motion result 109c on the terminal page for the target user to view.

[0099] Using the method provided in this application, the sequence period of the motion signal sequence 100c can be accurately segmented through the balance line, thereby enabling accurate prediction of the target motion result of the target user.

[0100] Please see Figure 3 , Figure 3 This is a flowchart illustrating a motion signal processing method provided in this application. Figure 3 As shown, the method may include:

[0101] Step S101: Obtain the motion signal sequence of the target user and obtain the sequence equilibrium position of the motion signal sequence;

[0102] Specifically, the execution entity in this application embodiment can be a single computer device or a cluster of multiple computer devices. The computer device can be a server or a terminal device. Therefore, the execution entity in this application embodiment can be a server, a terminal device, or a combination of both.

[0103] Computer equipment can acquire the motion signal sequence of a target user, who can be any user. The motion signal sequence can be generated based on the target user's movements. This application allows for various types of motion assessment of the target user, such as predictions of hand movements, waist movements, or foot movements. Hand movement predictions can be further divided into several types, such as predictions of palm rotation, palm opening and closing / clenching, or finger opening and closing movements (e.g., between the thumb and index finger). Similarly, foot movement predictions can also be divided into several types, such as predictions of stomping or walking movements. Therefore, the motion signal sequence of the target user can be determined based on the type of motion assessment (referred to as the motion prediction type), which can be any of the hand, waist, or foot movement prediction types described above. The specific motion prediction type for the target user is determined based on the actual application scenario and is not limited thereto.

[0104] Optionally, the computer device can acquire the motion signal sequence of the target user through motion video data of the target user, or the computer device can acquire the motion signal sequence of the target user through sensors.

[0105] One method for a computer device to acquire a target user's motion signal sequence from the target user's motion video data is as follows: The computer device can acquire the target user's motion prediction type, which can be a prediction type specified by a motion assessment person (such as a doctor or developer) in the computer device. Furthermore, the computer device can determine the motion keypoints for the target user based on this motion prediction type; these motion keypoints are the body point locations that need to be assessed when evaluating the target user's motion.

[0106] For example, if the predicted motion type for the target user is a prediction of the target user's hand flipping motion, then the motion key points could include the key points at the tips of the target user's thumb and little finger. Alternatively, if the predicted motion type for the target user is a prediction of the target user's hand opening, closing, and clenching motion, then the motion key points could also include the location points of all the finger joints of the target user's hand. Or, if the predicted motion type for the target user is a prediction of the opening and closing motion between the fingers (such as between the thumb and index finger) (also known as finger-opposing motion), then the motion key points could include the location points at the tips of the target user's thumb and index finger. Or, if the predicted motion type for the target user is a prediction of stomping motion, then the motion key point could be the location point at the target user's heel.

[0107] Please see Figure 4 , Figure 4 This is a schematic diagram of a scenario for acquiring motion key points provided in this application. For example... Figure 4 As shown, the computer device first obtains the target user's motion prediction type 101d. If the motion prediction type 101d is a prediction type of finger opening and closing movements (which can be called finger-opposing movements), and specifically a prediction type of finger-opposing movements between the tips of the thumb and index finger, then the motion keypoints 102d for the target user can be keypoints 4 and 8 in the hand 100d, where keypoint 8 is the tip of the index finger and keypoint 4 is the tip of the thumb. For example, if the motion prediction type 101d is a prediction type of fist clenching movements, then the motion keypoints 102d for the target user can be all the keypoints in the hand 100d, including keypoints 0 to 20.

[0108] Furthermore, computer devices can also acquire motion video data corresponding to the target user's predicted motion type. This motion video data can be recorded during on-site guidance from exercise assessors, uploaded by the target user, or retrieved from the internet. This motion video data is essentially the video data used for motion assessment of the target user. For example, it could be video data of the target user performing hand rotations, finger opening and closing movements, or foot stomping, and so on.

[0109] Therefore, computer equipment can detect the motion information of key points in the motion video data of a target user, which can be referred to as the target user's key point motion information. For example, if the key point is the location of the target user's heel, and the motion video data is video data of the target user stomping, then the computer equipment can detect and identify the motion information of the target user's heel location in the motion video data. This motion information can include information such as the amplitude, angle, and speed of the movement of the target user's heel location.

[0110] Furthermore, computer equipment can generate a motion signal sequence for the target user by detecting keypoint motion information of the target user in the target user's motion video data. This motion signal sequence can include multiple sampled values ​​corresponding to multiple sampling points that sample the motion parameters of the target user's motion process, with one sampled value corresponding to one sampling point. Each sampled value can be referred to as a frame, and each sampling point can correspond to an index value. This index value indicates the frame number of the corresponding sampling point (i.e., which frame was sampled in). In other words, the index value indicates which sampling point the corresponding sampling point is, and the sampling order of the sampled values ​​corresponding to each sampling point can be distinguished by the index value. The sampled values ​​in the motion signal sequence can be calculated based on the target user's keypoint motion information. For different motion prediction types, the type of sampled values ​​in the motion signal sequence is usually different.

[0111] For example, if the motion prediction type for a target user is the prediction type for the target user's hand opening, closing, and clenching movements, then the area of ​​the hand (which can be understood as the projected area of ​​the hand) will change during the clenching process. The target user's hand area is largest when the palm is fully open and smallest when the palm is fully clenched. Therefore, the computer device can calculate the target user's hand area at each sampling point (such as each sampling time point) using the target user's keypoint motion information. Thus, the multiple sample values ​​corresponding to the multiple sampling points included in the target user's motion signal sequence can be the values ​​of the target user's hand area at each sampling point.

[0112] For example, if the predicted motion type for the target user is the opening and closing motion between the user's fingers (such as between the thumb and index finger), then the distance between the fingertips of the target user's thumb and index finger will change as the user performs this opening and closing motion. The distance is greatest when the user opens their thumb and index fingertips to their maximum extent, and smallest when they close and touch. Therefore, the computer device can calculate the distance between the fingertips of the target user's thumb and index finger at various sampling points (such as various sampling time points) using the target user's keypoint motion information. Thus, the sampled values ​​corresponding to the multiple sampling points included in the target user's motion signal sequence can be the values ​​of the distance between the fingertips of the target user's thumb and index finger at each sampling point.

[0113] Similarly, the process by which a computer device acquires the motion signal sequence of a target user through sensors can be as follows: during the motion process, the target user can wear sensor devices at the corresponding key points of motion based on the motion prediction type. Then, the computer device can acquire the sensor motion parameters of the target user during the motion process through the sensor devices. These sensor motion parameters may include the motion amplitude and motion speed of the target user's key points of motion.

[0114] The sensor motion parameters can be understood as the motion information of the aforementioned key points. Therefore, the computer device can also use these sensor motion parameters to calculate motion characteristic parameters for the target user. These motion characteristic parameters can be the calculated sample values ​​of the target user at each sampling point. For example, if the motion prediction type for the target user is the prediction type of the target user's hand opening and closing / clenching motion, then the motion characteristic parameter could be the value of the hand area of ​​the target user at each sampling point. As another example, if the motion prediction type for the target user is the prediction type of the opening and closing motion between the target user's fingers (such as between the thumb and index finger), the motion characteristic parameter could also be the value of the distance between the fingertips of the target user's thumb and index finger. Therefore, the computer device can generate a motion signal sequence for the target user using the calculated motion characteristic parameters.

[0115] After the computer device generates a motion signal sequence for the target user, it can also obtain the sequence equilibrium position of this motion signal sequence. This equilibrium position is the equilibrium position of the smoothed signal sequence, which is the signal sequence obtained after filtering out noise and interference signals from the motion signal sequence. The process of obtaining the smoothed signal sequence of the motion signal sequence is detailed below. Figure 8 As described in the corresponding embodiment, the smoothed signal sequence may also include sampled values ​​corresponding to multiple sampling points. The sampling points included in the smoothed signal sequence are the same as those included in the motion signal sequence, but the sampled values ​​corresponding to the sampling points included in the smoothed signal sequence may be different from those corresponding to the sampling points included in the motion signal sequence.

[0116] The equilibrium position of this sequence refers to the location of the equilibrium line in the motion signal sequence. Optionally, when the sampled values ​​in the motion signal sequence are unevenly distributed, the equilibrium position can be obtained from the maxima (peaks) and minima (troughs) in the motion signal sequence. This allows for a comprehensive consideration of the maxima and minima of the motion signal sequence, resulting in a more accurate differentiation between them. Therefore, this equilibrium position can also more accurately distinguish the sampled values ​​in the upper and lower regions of the motion signal sequence. It can be understood that this method uses the equilibrium line to divide the maxima and minima in the smoothed signal sequence into a relatively balanced distribution. The combined distance between one or more maxima in the smoothed signal sequence and the equilibrium line approaches the combined distance between one or more minima in the smoothed signal sequence and the equilibrium line. For details on how to detect the extreme values ​​(including maxima and minima) in a smoothed signal sequence and how to obtain the equilibrium position of the smoothed signal sequence from these extreme values, please refer to the following... Figure 8The corresponding embodiment describes this. The equilibrium position of the smooth signal sequence is also the equilibrium position of the motion signal sequence.

[0117] Alternatively, when the sampled values ​​in the smoothed signal sequence of a motion signal sequence are relatively uniformly distributed, the balance line can also be obtained by the mean of the sampled values ​​in the smoothed signal sequence. In this case, the resulting balance line can evenly divide the sampled values ​​in the smoothed signal sequence into two parts: one part with larger element values ​​and the other part with smaller element values. See below for details. Figure 8 The relevant descriptions in the corresponding embodiments.

[0118] In the smoothed signal sequence described above, the indices of each sampling point (e.g., the index indicating the nth sampling point) can be distributed on the horizontal axis, and the sampled value corresponding to each sampling point in the smoothed signal sequence can be the vertical axis corresponding to the index of each sampling point. Therefore, it should be understood that the above-mentioned balance line can be a horizontal line determined by the vertical axis.

[0119] Through the above process, the computer device can obtain the motion signal sequence of the target user, as well as the sequence equilibrium position of the motion signal sequence, which is also the position of the equilibrium line of the smoothed signal sequence of the motion signal sequence.

[0120] Step S102: Determine the zero-crossing point of the motion signal sequence based on the sequence equilibrium position, and segment the motion signal sequence based on the zero-crossing point to obtain the sequence period of the motion signal sequence;

[0121] In this application, the aforementioned motion signal sequence is a time series, meaning it is a signal in the time domain. Therefore, the smoothed signal sequence of the motion signal sequence is also a time series. A computer device can make this smoothed signal sequence time-continuous, thus obtaining the signal wave of the smoothed signal sequence, which can be called the motion signal wave. Therefore, the computer device can obtain the intersection point between the motion signal wave and the equilibrium line at the sequence's equilibrium position, which can be called the sequence zero-crossing point of the motion signal sequence. The equilibrium line and the sequence zero-crossing point are mutually deterministic; that is, the equilibrium line can be determined through the sequence zero-crossing point, and the sequence zero-crossing point can also be determined through the equilibrium line.

[0122] There are usually multiple (at least two) zero-crossing points in the above sequence. The computer equipment can correct and filter the multiple zero-crossing points of the sequence according to the sampling frequency threshold. The sampling frequency threshold is the minimum sampling frequency corresponding to the sampling point in the motion signal sequence. In other words, the sampling rate corresponding to the sampling value in the motion signal sequence must be greater than or equal to the sampling frequency threshold, and cannot be less than the sampling frequency threshold. This can ensure that the analog signal can be recovered without distortion.

[0123] The sampling frequency threshold can be calculated using the Nyquist sampling theorem. This sampling frequency domain threshold is equal to f / 2w, where f is the sampling frequency corresponding to the sampled value in the motion signal sequence (which can be understood as the frame rate, with one frame corresponding to one sampling point). For example, f can be equal to 50Hz. w is the highest frequency of human movement. For example, when the motion prediction type is hand movement, w can be equal to the physiological limit of human hand movement, which is 6-7Hz. Alternatively, w can be given some margin, so that w is slightly larger than the physiological limit of human hand movement, which is 6-7Hz, such as 8Hz.

[0124] Therefore, normally, the frequency interval between two sequence zero-crossing points needs to be greater than the aforementioned sampling frequency threshold to ensure undistorted recovery of the analog signal. Thus, computer equipment can sequentially detect the frequency interval between any two adjacent sequence zero-crossing points from multiple sequence zero-crossing points, and can identify adjacent sequence zero-crossing points whose detected frequency interval is less than the aforementioned sampling frequency threshold as abnormal sequence zero-crossing points.

[0125] Furthermore, the computer device can filter out (i.e. remove) the abnormal sequence zero-crossing from all sequence zero-crossings. The sequence zero-crossing remaining after removing the abnormal sequence zero-crossing is called the corrected sequence zero-crossing. Therefore, the computer device can segment the smoothed signal sequence of the motion signal sequence using the corrected sequence zero-crossing, thereby obtaining the period of the motion signal sequence, which can be called the sequence period.

[0126] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating a scenario for obtaining the periodicity of a sequence, as provided in this application. Figure 5 As shown, by detecting the intersection point between the equilibrium line 101g and the motion signal wave 100g of the smooth signal sequence, multiple sequence zero-crossing points can be obtained. These multiple sequence zero-crossing points can be sequence zero-crossing point 101h, sequence zero-crossing point 102h, sequence zero-crossing point 103h, sequence zero-crossing point 104h, sequence zero-crossing point 105h, and sequence zero-crossing point 106h in sequence.

[0127] The computer device can sequentially detect the frequency interval between any two adjacent zero-crossing points in the plurality of sequence zero-crossing points. For example, if it first detects that the frequency interval between adjacent sequence zero-crossing points 101h and 102h in box 102g is greater than the sampling frequency threshold, then the frequency interval between sequence zero-crossing points 101h and 102h meets the requirements; then it further detects that the frequency interval between adjacent sequence zero-crossing points 102h and 103h in box 103g is greater than the sampling frequency threshold. If the frequency interval between sequence zero-crossing points 102h and 103h meets the requirements, then it is detected that the frequency interval between adjacent sequence zero-crossing points 103h and 104h in box 104g is less than the sampling frequency threshold. Therefore, the frequency interval between sequence zero-crossing points 103h and 104h does not meet the requirements, and sequence zero-crossing points 103h and 104h are abnormal sequence zero-crossing points 106g, which need to be removed.

[0128] Furthermore, since the frequency interval between sequence zero-crossing points 102h and 103h is greater than the sampling frequency threshold, the frequency interval between sequence zero-crossing points 102h and 105h is also definitely greater than the sampling frequency threshold. Therefore, the computer device does not need to further detect the frequency interval between sequence zero-crossing points 102h and 105h. The computer device can continue to detect the frequency interval between adjacent sequence zero-crossing points 105h and 106h in frame 105g, and detect that the frequency interval between sequence zero-crossing points 105h and 106h is greater than the sampling frequency threshold, and the frequency interval between sequence zero-crossing points 105h and 106h meets the requirements.

[0129] Therefore, by filtering out the abnormal sequence zero-crossing point 106g from all sequence zero-crossing points, the corrected sequence zero-crossing point 107g can be obtained. This corrected sequence zero-crossing point 107g includes sequence zero-crossing points 101h, 102h, 105h, and 106h. By periodically dividing the motion signal wave 100g using this corrected sequence zero-crossing point 107g, the sequence period 108g of the smoothed signal sequence can be obtained.

[0130] One method of segmenting the smoothed signal sequence by correcting the zero-crossing points of the sequence is as follows: Considering the actual situation of user movement, a sequence period can be composed of peak-valley-peak. For example, for the prediction type of opening and closing fist movements, when the user opens their palm, it corresponds to a peak (this peak can represent the projected area of ​​the open palm at this time). When the user moves from opening their palm to closing their fist, it corresponds to a valley (this valley can represent the projected area of ​​the clenched fist at this time). When the user moves from clenching their fist back to opening their palm, it corresponds to a peak (this peak can represent the projected area of ​​the open palm at this time). As can be seen from the above, the process from opening the palm to clenching the fist and then back to opening the palm is a complete process. Peaks usually refer to the state of maximum value, and valleys usually refer to the state of minimum value.

[0131] Understandably, there can be a peak or a trough between every two adjacent zero-crossing points of the corrected sequence. Each sequence zero-crossing point can have a direction, which can be upward or downward. When the direction of a sequence zero-crossing point is upward, one or more sample values ​​to the right of the zero-crossing point are greater than one or more sample values ​​to the left of the zero-crossing point. Similarly, when the direction of a sequence zero-crossing point is downward, one or more sample values ​​to the left of the zero-crossing point are less than one or more sample values ​​to the right of the zero-crossing point. For example, the above... Figure 2 The direction of the sequence to the left of the first peak of the "first sequence period" is upward, corresponding to the zero point ①. Figure 2 The direction of the sequence to the right of the first peak of the "first sequence period" is downward.

[0132] Therefore, there can be a peak between an upward-pointing corrected sequence zero-crossing point (located first) and a downward-pointing corrected sequence zero-crossing point (located second). The peak value corresponding to this peak can be called the sequence peak value, which can be the maximum value among the sampled values ​​included between the two adjacent corrected sequence zero-crossing points. Conversely, there can be a valley between a downward-pointing corrected sequence zero-crossing point (located first) and an upward-pointing corrected sequence zero-crossing point (located second). The valley value corresponding to this valley can be called the sequence valley value, which can be the minimum value among the sampled values ​​included between the two adjacent corrected sequence zero-crossing points.

[0133] As shown above, a smoothed signal sequence can be divided into one or more sequence periods by correcting the zero-crossing points of the sequence. A sequence period can have two peaks and one valley. The two peaks correspond to a sequence peak value, and the valley corresponds to a sequence valley value. For example, assuming a peak in a sequence period is peak z1, and the two corrected sequence zero-crossing points surrounding peak z1 are zero-crossing points g1 and g2, then the sequence peak value corresponding to peak z1 can be the maximum value of the sampled values ​​in the smoothed signal sequence whose abscissa lies between the abscissas of zero-crossing points g1 and g2. Similarly, assuming a valley in a sequence period is z2, and the two corrected sequence zero-crossing points surrounding valley z2 are zero-crossing points g3 and g4, then the sequence valley value corresponding to valley z2 can be the minimum value of the sampled values ​​in the smoothed signal sequence whose abscissa lies between the abscissas of zero-crossing points g3 and g4.

[0134] In this application, the zero-crossing points of a smooth signal sequence are obtained through the sequence equilibrium position, and these zero-crossing points can be adaptively corrected and filtered to obtain the final corrected sequence zero-crossing points. Correcting and filtering the sequence zero-crossing points is equivalent to correcting and filtering the element values ​​in the obtained smooth signal sequence (which also represents a motion signal sequence), thus enabling accurate periodic segmentation of the smooth signal sequence. Furthermore, if the sequence equilibrium position is obtained through the sequence extrema, then correcting and filtering the sequence zero-crossing points is equivalent to correcting the detected sequence extrema in reverse, which also demonstrates that the method provided in this application can achieve accurate periodic segmentation of the smooth signal sequence.

[0135] Step S103: Generate a sequence feature vector of the motion signal sequence based on the sequence period, and predict the target motion result of the target user based on the sequence feature vector;

[0136] Specifically, computer equipment can generate a sequence feature vector of a motion signal sequence based on the sequence period. This sequence feature vector is the final vector used to characterize the motion features of the motion signal sequence for the target user. Please refer to the following description; the process of generating a sequence feature vector of a motion signal sequence can be as follows:

[0137] First, the computer equipment can calculate the period duration and period amplitude of each sequence period. The period duration of a sequence period can be equal to the duration between two peaks in that sequence period. For example, the period duration t of the i-th sequence period... i It can be obtained through the following formula (1):

[0138]

[0139] Among them, y i2Let y be the index value corresponding to the next peak in the i-th sequence period (this index value can be any index value of the sampling point, such as the index value of the nth sampling). i1 Let f be the index value corresponding to the previous peak of the i-th sequence period, and f be the sampling frequency.

[0140] The period amplitude a of the i-th sequence period i Please refer to the following formula (2):

[0141]

[0142] in, This indicates the peak value of the sequence corresponding to the next peak in the i-th sequence period. This indicates the peak value of the sequence corresponding to the previous peak of the i-th sequence period. Let be the valley value of the i-th sequence period.

[0143] The computer can acquire the period duration and period amplitude of each sequence period using the same principle as acquiring the period duration and period amplitude of the i-th sequence period. Furthermore, the computer device can generate the sequence feature vector of the aforementioned motion signal sequence using the acquired period duration and period amplitude of each sequence period, as described below:

[0144] Optionally, the computer device can also acquire a motion assessment standard table, which may include motion feature types for assessing multiple motion prediction types. In other words, different motion prediction types may have different motion feature types, and the motion of a certain motion prediction type can be assessed by using the feature parameters under the corresponding motion feature type. For example, the motion assessment standard table may be the MDS-UPDRS (Parkinson's Disease Comprehensive Assessment Scale).

[0145] For example, for the prediction type of hand opening and closing / clenching motion, the corresponding motion feature types can include motion pause feature types, motion amplitude feature types, motion speed feature types, and motion amplitude decay feature types, etc. As another example, for the prediction type of toe tapping motion, the corresponding motion feature types can be motion pause feature types, motion amplitude feature types, and motion speed feature types, etc.

[0146] In other words, the motion evaluation criteria table is actually a standard table used to evaluate users based on their movements. The table can include the relationships between various motion prediction types and their corresponding motion feature types. When evaluating a motion of a specific motion prediction type, the motion features under the corresponding motion feature type can be used for evaluation.

[0147] Therefore, the computer device can use the motion feature type associated with the motion prediction type of the target user in the motion evaluation standard table as the detection feature type for the target user. Furthermore, the computer device can generate signal feature parameters of the motion signal sequence under this detection feature type based on the sequence duration and sequence amplitude of the obtained sequence period, and can generate the sequence feature vector corresponding to the motion signal sequence of the target user based on these signal feature parameters.

[0148] For example, the criteria for predicting hand movements in the movement assessment criteria table can be found in Table 1 below.

[0149]

[0150] Table 1

[0151] If the detection feature type is obtained through Table 1 above, there can be multiple detection feature types, such as speed (slowness) feature type, amplitude feature type, amplitude decay feature type, hesitation / freeze feature type, interruption feature type, and action completion feature type.

[0152] Furthermore, different motion sequence signals may be segmented into different numbers of sequence periods. Assuming the motion sequence signal of the target user is segmented into m sequence periods, each sequence period has a duration and an amplitude. Therefore, the duration of each of these m sequence periods can be represented as a sequence... Period duration t i Let be the period duration of the i-th sequence period. Similarly, the period amplitude of these m sequence periods can also be represented as a sequence... Period duration a i Let be the period amplitude of the i-th sequence period.

[0153] Furthermore, computer equipment can... and To generate signal feature parameters for motion signal sequences under each detection feature type: The signal feature parameters of the motion signal sequence generated by the computer device under the velocity (slowness) feature type can be the sequence... The median of the signal, the signal characteristic parameter is 1-dimensional; the signal characteristic parameter of the motion signal sequence generated by the computer device under the amplitude characteristic type can be the sequence... The median of the signal characteristic parameter is one-dimensional; the number of sequence periods m considered can be a multiple of 3 (or other values, and the corresponding signal characteristic parameter can also change accordingly). Therefore, the signal characteristic parameter of the motion signal sequence generated by the computer device under the amplitude decay characteristic type can be... The first-order difference of the signal, the signal characteristic parameters are (m / 3)-1 dimensional. The signal characteristic parameters of a motion signal sequence generated by a computer device under the hesitant freeze characteristic type can be the sequence... Maximum value and sequence The number of outliers indicates that the signal feature parameter is 2-dimensional. The signal feature parameter of a motion signal sequence generated by a computer device, under the type of interruption feature, can be the sequence... Minimum value and sequence The number of outliers in the signal is 2-dimensional. The signal feature parameter of a motion signal sequence generated by a computer device under the action completion feature type can be min(1, k / m), which is 1-dimensional. Here, min(1, k / m) represents the minimum value between 1 and m / M, where M is the maximum number of cycles the user is expected to complete when evaluating their motion. For example, if M equals 12, then if the user completes 12 cycles of motion, the user's action completion is considered perfect.

[0154] In addition to generating signal feature parameters under the detection feature types in the aforementioned motion evaluation standard table, the computer device can also consider additional statistical characteristics of the target user's motion signal sequence, such as sequence... Statistical characteristics and sequences of interquartile range The interquartile range has two statistical characteristics. Therefore, computer equipment can calculate the sequence. interquartile range and sequence The interquartile range is also used as a signal feature parameter of the generated motion signal sequence of the target user; this signal feature parameter is 2-dimensional. Through the sequence... interquartile range and sequence The interquartile range can be used to assess the motion stability and rhythm consistency of a target user.

[0155] Assuming the number of sequence periods m obtained from segmenting the motion signal sequence of the target user is 12, the computer device can obtain 1-dimensional signal feature parameters under the type of velocity (slowness) feature, 1-dimensional signal feature parameters under the type of amplitude feature, 3-dimensional (i.e., 12 / 3)-dimensional signal feature parameters under the type of amplitude decay feature, 2-dimensional signal feature parameters under the type of hesitation / freeze feature, 2-dimensional signal feature parameters under the type of interruption feature, 1-dimensional signal feature parameters under the type of action completion feature, and through the sequence interquartile range and sequence The 2D signal feature parameters obtained from the interquartile range are a total of 12-dimensional signal feature parameters.

[0156] Therefore, computer equipment can concatenate all the acquired signal feature parameters to obtain the sequence feature vector of the target user's motion signal sequence. For example, the 12-dimensional signal feature parameters obtained above can be concatenated to obtain a 12-dimensional sequence feature vector.

[0157] Furthermore, the computer device can input the generated sequence feature vector into a motion prediction model. This motion prediction model can be pre-trained using the sample sequence feature vectors of the sample motion signal sequence and can be used to output an evaluation score for the user's motion. This evaluation score can correspond to the aforementioned motion evaluation standard table, as shown in Table 1 above, and can be any score among 0 / 1 / 2 / 3 / 4. The principle for obtaining the sample sequence feature vector of the sample motion signal sequence is the same as the principle for obtaining the sequence feature vector of the target user's motion signal sequence, as described above. By inputting the generated sequence feature vector of the target user's motion signal sequence into the motion prediction model, the motion prediction model can output the target motion result for the target user, which can be any evaluation score among 0 / 1 / 2 / 3 / 4.

[0158] Specifically, by generating signal feature parameters using motion feature types directly corresponding to the motion assessment criteria table, and then using these signal feature parameters to generate a sequence feature vector, the user's motion can be evaluated using this sequence feature vector. This approach ensures that the predicted target motion results for the target user have high interpretability. In other words, the target motion results for the target user predicted by the sequence feature vector generated from the motion feature types directly corresponding to the motion assessment criteria table can directly correspond to the scoring criteria in the table, which improves the medical interpretability of motion assessments.

[0159] More specifically, the process of training a motion prediction model can be as follows: A computer device can acquire a sample motion signal sequence from a sample user and the actual motion result of that sample motion signal sequence. The computer device can acquire the sample motion signal sequence in the same way as it acquires the motion signal sequence of the target user. For example, the computer device can also acquire the equilibrium position of the sample motion signal sequence and segment the sample motion signal sequence using this equilibrium position to obtain the sample sequence period. Furthermore, the computer device can generate a sample sequence feature vector based on this sample sequence period. The actual motion result of the sample motion signal sequence can be an evaluation score obtained by researchers (such as doctors) based on a motion evaluation standard table during the user's movement. This evaluation score can also be any one of the evaluation scores 0 / 1 / 2 / 3 / 4 in Table 1.

[0160] The motion prediction model can be trained from an initial motion prediction model. The computer device can input the generated sample sequence feature vectors and the actual motion results for the sample user into the initial motion prediction model. This initial motion prediction model can then predict the sample motion results for the sample user. Furthermore, the computer device can use the sample motion results and the actual motion results for the sample user to correct the model parameters of the initial motion prediction model. The corrected initial motion prediction model can then be used as the aforementioned motion prediction model. The principle of correcting the model parameters of the initial motion prediction model is to adjust the model parameters so that the sample motion results predicted by the initial motion prediction model approximate the actual motion results for the sample user.

[0161] Furthermore, the initial motion prediction model mentioned above can be either a decision tree-based ensemble learning model (GBDT, Gradient Boosting Tree Model) or a random forest model. Experimental results demonstrate that the random forest model has better evaluation accuracy than GBDT; therefore, the random forest model can be preferentially used as the initial motion prediction model. The experimental results are shown in Table 2 below:

[0162] accuracy Cohen's kappa GBDT 86.25％ 0.8025 random forest 87.82％ 0.8224

[0163] Table 2

[0164] Cohen's kappa is a statistical measure of the correlation between scores, ranging from -1 to 1. A value between 0.81 and 1 indicates a very high predictive accuracy (e.g., accurate score prediction). Therefore, the GBDT model has a prediction accuracy of 86.25%, while the random forest model has an accuracy of 87.82%. The Cohen's kappa for the GBDT model is 0.8025, and for the random forest model it is 0.8224. Therefore, in practical motion assessment scenarios, the random forest model can be used to train a motion prediction model, which can accurately assess a user's motion.

[0165] Please see Figure 6 , Figure 6 This is a scene illustration of a prediction of target motion results provided in this application. For example... Figure 6 As shown, the computer device can generate a sample sequence feature vector 101e from the sample motion signal sequence 100e of the sample user. The computer device can input the sample sequence feature vector 101e and the actual motion result 102e of the sample user into the initial motion prediction model 103e, and then output the sample motion result 104e of the sample user through the initial motion prediction model 103e.

[0166] The computer device can backpropagate the sample motion result 104e to the initial motion prediction model 103e, so as to correct the model parameters of the initial motion prediction model 103e through the sample motion result 104e and the actual motion result 102e, and thus obtain the motion prediction model 107e with corrected model parameters.

[0167] Furthermore, the computer device can input the sequence feature vector 106e generated by the motion signal sequence 105e of the target user into the trained motion prediction model 107e, and then output the target motion result 108e for the target user through the motion prediction model.

[0168] Please see Figure 7 , Figure 7 This is a schematic diagram of a scenario for obtaining the motion result of a target, as provided in this application. Figure 7As shown, the computer device can input a motion signal sequence into the filtering module 100i to obtain a smoothed signal sequence. Then, the computer device can perform extremum detection on the smoothed signal sequence using the extremum detection model 101i to obtain the sequence extrema. The computer device can also use the obtained sequence extrema to determine the sequence equilibrium position of the balanced signal sequence. Furthermore, the waveform segmentation module 102i can use this sequence equilibrium position to periodically segment the motion signal wave of the smoothed signal sequence to obtain the sequence period.

[0169] Furthermore, the motion index extraction module 103i can extract motion indices from the segmented sequence period. The extracted motion indices can be the signal feature parameters calculated from the sequence period. The target motion result prediction module 104i can generate the sequence feature vector from the signal feature parameters and can call the motion prediction model. By inputting the sequence feature vector into the motion prediction model, the target motion result for the target user can be output.

[0170] The method provided in this application can be used to assess the daily movement of normal users, or it can be used to assess the movement of patients with movement disorders. By using the method provided in this application to predict the target movement results of target users, doctors can be assisted in assessing the user's movement. In other words, the target movement results of target users generated by this application can be provided to researchers as auxiliary information to assess the user's movement.

[0171] This application first acquires the motion signal sequence of the target user and obtains the sequence equilibrium position of the motion signal sequence. Next, it determines the sequence zero-crossing points of the motion signal sequence based on the sequence equilibrium position, and segments the motion signal sequence based on these zero-crossing points to obtain the sequence period. Furthermore, it generates a sequence feature vector of the motion signal sequence based on the sequence period, and predicts the target user's target motion result based on the sequence feature vector. Therefore, the method proposed in this application can periodically segment the motion signal sequence using the sequence equilibrium position. Since this sequence equilibrium position can evenly distribute the sampled values ​​in the motion signal sequence, it can accurately segment the period of the motion signal sequence. Furthermore, it can generate a sequence feature vector using the accurately segmented sequence period, thereby achieving accurate prediction of the target user's target motion result.

[0172] Please see Figure 8 , Figure 8 This is a flowchart illustrating a method for obtaining the balanced position of a sequence provided in this application. Figure 8 As shown, the method may include:

[0173] Step S201: Obtain the smoothed signal sequence of the motion signal sequence and detect the sequence extrema of the smoothed signal sequence;

[0174] The execution subject in this application embodiment is the same as the one described above. Figure 3 The executing entity in the corresponding application embodiments is the same, which is a computer device. The motion signal sequence here is the one described above. Figure 3 The corresponding embodiment refers to the motion signal sequence acquired for the target user. During the acquisition of the motion signal sequence, some noise and interference signals may be collected. These noise and interference signals manifest as spikes and glitches in the waveform of the motion signal sequence (the continuous signal of the motion signal sequence). Therefore, the computer device can filter the motion signal sequence based on the frequency limits of human physiological movement, that is, filter out the noise and interference signals in the motion signal sequence. The signal sequence after filtering out the noise and interference signals can be called the smoothed motion signal sequence.

[0175] Specifically, filtering a motion signal sequence to obtain a smooth signal sequence can be done as follows:

[0176] Computer devices can acquire a user's motion frequency threshold, which is the frequency domain of the human physiological motion limit. In other words, this user motion frequency threshold is the maximum motion frequency that humans can achieve during movement. Different motion prediction types can have different user motion frequency thresholds. For example, assuming the motion prediction type for the target user is the prediction type for the target user's hand movements, and the physiological motion limit of the human hand is 6-7 Hz, then the aforementioned user motion frequency threshold could be 6-7 Hz.

[0177] The computer device can retain a margin based on the user's motion frequency threshold. This margin can be determined according to the actual application scenario, and it can also be 0. For example, if the margin is 1Hz, the computer device can add 1Hz to the user's motion frequency threshold, and then filter the motion signal sequence by adding the 1Hz user motion frequency threshold to obtain a smoothed motion signal sequence. For instance, if the motion prediction type for the target user is the prediction type for the target user's hand movements, and the user motion frequency domain threshold is the physiological limit of human hand movements (6-7Hz), then 1Hz can be added to this 7Hz, using 8Hz to filter the motion signal sequence to obtain a smoothed signal sequence. Please refer to the following description.

[0178] The aforementioned motion signal sequence can be a signal in the time domain. Therefore, computer equipment can first transform the motion signal sequence from the time domain to the frequency domain to obtain the signal of the motion signal sequence in the frequency domain. The signal of the motion signal sequence in the frequency domain can be called a frequency domain motion signal.

[0179] Furthermore, computer equipment can use a user motion frequency threshold with some margin (such as 8Hz obtained by adding a margin of 1Hz to 7Hz as mentioned above) to filter the frequency domain motion signal, thereby filtering out high-frequency components in the frequency domain motion signal (for example, filtering out frequency components greater than 8Hz), that is, filtering out noise and interference signals in the frequency domain motion signal. The filtered frequency domain motion signal can be called a smooth motion signal.

[0180] Furthermore, the computer device can transform the smoothed motion signal in the frequency domain to the time domain, thus obtaining a smoothed signal sequence after filtering the motion signal sequence. By transforming the motion signal sequence to the frequency domain before filtering, instead of filtering the motion signal sequence directly in the time domain, it is possible to achieve filtering without adjusting and setting hyperparameters related to the time-domain filtering window, which can improve the accuracy of signal filtering.

[0181] After obtaining the smoothed signal sequence of the above motion signal sequence, the computer device can obtain the sequence extrema of the smoothed signal sequence. The sequence extrema of the smoothed signal sequence are the sequence extrema of the motion signal sequence. The sequence extrema include the maximum value (which can be called the sequence maximum value, and is reflected in the waveform as a peak) and the minimum value (which can be called the sequence minimum value, and is reflected in the waveform as a trough) in the motion signal sequence. The following describes in detail how to obtain the sequence extrema in the smoothed signal sequence.

[0182] Computer equipment can detect sequence extrema in smooth signal sequences using automatic multiscale-based peak detection (AMPD, a signal detection algorithm). By employing the AMPD algorithm to detect sequence extrema in smooth signal sequences, no additional hyperparameter settings are required during the detection process; the entire detection process is adaptively parameter-set. Since setting hyperparameters during detection requires constant adjustment and setting for different signal sequences, this application eliminates the need for additional hyperparameter settings. Therefore, it is universally applicable to any type of signal sequence and improves the efficiency and adaptability of signal detection.

[0183] The following describes in detail how to use the AMPD algorithm to detect sequence extrema in a smooth signal sequence:

[0184] A smoothed signal sequence can include sampled values ​​from multiple sampling points (at least two sampling points, one sampling point corresponding to one time point). Each sampling point can correspond to one sampled value, and the sampled values ​​from these multiple sampling points are arranged sequentially according to their sampling time in the smoothed signal sequence. Therefore, the local maxima scalogram (LMS, a type of adaptive filtering) matrix of the smoothed signal sequence can be obtained through the AMPD algorithm. The sequence maxima and sequence minima can each correspond to an LMS matrix; that is, the sequence maxima correspond to one LMS matrix, and the sequence minima correspond to another LMS matrix.

[0185] As shown in the following formula (3), the LMS matrix J1 corresponding to the sequence maxima can be expressed as:

[0186]

[0187] Where N is the total number of sampling points, which is a positive integer. In other words, a smooth signal sequence can include a total of N sampling points, which can be sampling point 1, sampling point 2, ... and sampling point N in sequence, where n is less than or equal to N. This represents the sample value at the nth sampling point in the smoothed signal sequence. This represents the sample value of the l-th sample point preceding the n-th sample point in the smoothed signal sequence. This represents the sample value of the l-th sample point following the n-th sample point in a smooth signal sequence; it can be understood as the sample value. The corresponding sampling points nl and The corresponding sampling points n+l are respectively related to the sampled values The sampling interval between corresponding sampling points n is l, where l is less than or equal to the total number of rows L of matrix J1. l,n Let L represent the value of the element at the l-th row and n-th column in matrix J1. To ensure the algorithm's validity, L can be equal to...

[0188] Therefore, it can be understood that the first row of matrix J1 includes the relationship between the sampled value corresponding to each sampling point and the sampled value corresponding to the sampling point at a distance of 1 sampling point; the second row of matrix J1 includes the relationship between the sampled value corresponding to each sampling point and the sampled value corresponding to the sampling point at a distance of 2 sampling points; and so on. The Lth row of matrix J1 includes the relationship between the sampled value corresponding to each sampling point and the sampled value corresponding to the sampling point at a distance of L sampling points.

[0189] The element value d in the above matrix J1 l,n The method for determining this is as follows: when the sampled value Compare and sampled values When both need to be large, then let d l,n It is 0, and when Compare and sampled values If any one or two sampled values ​​are smaller than d, then let d l,n Let be 1 + δ, where δ is a random variable. Therefore, one column of the matrix J1 above corresponds to a sampling point. For example, the nth column of matrix J1 contains the relationship between the sampled value corresponding to sampling point n and the sampled values ​​corresponding to sampling points at different intervals.

[0190] Therefore, the way to obtain the maximum value of a sequence is:

[0191] First, it is important to understand that a maximum value is defined as being larger than the value within a certain range around it (which can be called the maximum range). Therefore, computer devices can first determine the maximum range when obtaining the maximum value of a sequence.

[0192] The computer device can sum the element values ​​in each row of matrix J1. The sum of the element values ​​in each row of matrix J1 is called the element sum of that row. The computer device can take the row with the smallest element sum in matrix J1 as the target row of matrix J1. The number of the target row can be denoted as k1. Therefore, the range of maxima can be the range of k1 sampling points away from each sampling point. For example, if k1 equals 2, then the sampling points in the range of maxima of sampling point n include the two sampling points adjacent to sampling point n before and the two sampling points adjacent to sampling point n after.

[0193] Therefore, the computer device can detect columns in matrix J1 where the first to the k1th element values ​​from top to bottom are all 0, and take the sampling points corresponding to the detected columns where the first to the k1th element values ​​are all 0 as the maximum points, and take the sampling values ​​corresponding to the maximum points as the sequence maxima in the smooth signal sequence.

[0194] Similarly, as shown in formula (4) below, the LMS matrix J2 corresponding to the sequence minimum can be expressed as:

[0195]

[0196] Where N is the total number of sampling points, which is a positive integer. In other words, a smooth signal sequence can include a total of N sampling points, which can be sampling point 1, sampling point 2, ... and sampling point N in sequence, where n is less than or equal to N. This represents the sample value at the nth sampling point in the smoothed signal sequence. This represents the sample value of the l-th sample point preceding the n-th sample point in the smoothed signal sequence. This represents the sample value of the l-th sample point following the n-th sample point in a smooth signal sequence; it can be understood as the sample value. The corresponding sampling points nl and The corresponding sampling points n+l are respectively related to the sampled values The sampling interval between corresponding sampling points n is l, where l is less than or equal to the total number of rows L of matrix J1. l,n Let L represent the value of the element in the l-th row and n-th column of matrix J2. To ensure the algorithm's validity, L can be equal to...

[0197] Therefore, it can be understood that the first row of matrix J2 includes the relationship between the sampled value corresponding to each sampling point and the sampled value corresponding to the sampling point at a distance of 1 sampling point; the second row of matrix J2 includes the relationship between the sampled value corresponding to each sampling point and the sampled value corresponding to the sampling point at a distance of 2 sampling points; and so on. The Lth row of matrix J2 includes the relationship between the sampled value corresponding to each sampling point and the sampled value corresponding to the sampling point at a distance of L sampling points.

[0198] The element value x in the above matrix J2 l,n The method for determining this is as follows: when the sampled value Compare and sampled values If both need to be in hours, then let d ln It is 0, and when Compare and sampled values If any one or two sampled values ​​are larger, then let d l,n Let be 1 + δ, where δ is a random variable. Therefore, one column of the matrix J2 above corresponds to a sampling point. For example, the nth column of matrix J2 contains the relationship between the sampled value corresponding to sampling point n and the sampled values ​​corresponding to sampling points at different intervals.

[0199] Therefore, the way to obtain the minimum value of a sequence is:

[0200] First, it is important to understand that the definition of a minimum value is that it is smaller than the values ​​within a certain range around it (which can be called the minimum value range). Therefore, computer devices can first determine the minimum value range when obtaining the minimum value of a sequence.

[0201] The computer device can sum the element values ​​in each row of matrix J2. The sum of the element values ​​in each row of matrix J2 is called the element sum of that row. The computer device can take the row with the smallest element sum in matrix J2 as the target row of matrix J2. The number of the target row can be denoted as k2. Therefore, the range of minimum values ​​can be the range of k2 sampling points away from each sampling point. For example, if k2 equals 3, then the sampling points in the range of minimum values ​​of sampling point n include the 3 sampling points adjacent to sampling point n before sampling point n and the 3 sampling points adjacent to sampling point n after sampling point n.

[0202] Therefore, the computer device can detect the column in each column of matrix J2 where the first element value to the k2th element value from top to bottom are all 0, and take the sampling point corresponding to the detected column where the first element value to the k2th element value are all 0 as the minimum point, and take the sampling value corresponding to the minimum point as the sequence minimum in the smooth signal sequence.

[0203] Please see Figure 9 , Figure 9 This is a schematic diagram illustrating a scenario for detecting sequence extrema provided in this application. For example... Figure 9 As shown, matrix 100f is matrix J1 mentioned above, used to obtain the maximum value of the sequence, and matrix 101f is matrix J2 mentioned above, used to obtain the minimum value of the sequence. In matrix 100f, one column corresponds to one sampling point, and one row corresponds to the same sampling point interval. Similarly, in matrix 101f, one column also corresponds to one sampling point, and one row corresponds to the same sampling point interval. Both matrices 100f and 101f are L rows and N columns.

[0204] As shown in box 102f, the computer device can truncate matrix 100f according to the row with the smallest sum (i.e., the target row k1 in matrix J1 above) to obtain matrix 103f. It can also truncate matrix 101f according to the row with the smallest sum (i.e., the target row k2 in matrix J2 above) to obtain matrix 104f. Therefore, matrix 103f only includes rows 1 to k1 of matrix 100f, and matrix 104f only includes rows 1 to k2 of matrix 101f.

[0205] Therefore, as shown in box 105f, the computer device can take the sampling points corresponding to the columns in matrix 103f where all element values ​​are 0 as the maxima points, and take the sampling values ​​corresponding to the maxima points in the smoothed sequence signal as the sequence maxima 107f. Similarly, the computer device can take the sampling points corresponding to the columns in matrix 104f where all element values ​​are 0 as the minima points, and take the sampling values ​​corresponding to the minima points in the smoothed sequence signal as the sequence minima 108f.

[0206] Optionally, both the minimum and maximum values ​​of the sequence can be obtained simultaneously using an LMS matrix, which can be called a difference matrix. This difference matrix has the same number of rows (L) and columns (N) as described above. Each column of the difference matrix corresponds to a sampling point; that is, each column contains the relationship between the sampled value corresponding to sampling point n and the sampled values ​​corresponding to sampling points at different intervals.

[0207] same, This represents the sample value at the nth sampling point in the smoothed signal sequence. This represents the sample value of the l-th sample point preceding the n-th sample point in the smoothed signal sequence. This represents the sample value of the l-th sample point following the n-th sample point in a smoothed signal sequence. The sample value can be... Subtract sample value The difference is called the sampling difference corresponding to sampling point nl, and the sampling values ​​can be... Subtract sample value The difference is called the sampling difference corresponding to sampling point n+l.

[0208] Therefore, the element values ​​in the difference matrix can be determined as follows: the element value at the nth column and lth row of the difference matrix can be called the element value that is jointly mapped by the sampling difference corresponding to sampling point n+l and the sampling difference corresponding to sampling point nl. When the sampled value Subtract sample value The difference and sampled values Subtract sample value When all differences are less than 0 (this 0 can be called the difference threshold), it indicates that the sampled values ​​are... At the same time greater than the sample value and sampled values At this point, the element value at the nth column and lth row of the difference matrix can be the first element value, which can be represented by any single character.

[0209] And when the sampled value Subtract sample value The difference and sampled values Subtract sample value When all differences are greater than 0 (i.e., greater than the difference threshold), it indicates that the sampled values ​​are... At the same time, it is less than the sample value. and sampled values At this point, the value of the element in the nth column and lth row of the difference matrix can be the second element value, which can also be represented by any character.

[0210] And when the sampled value Subtract sample value The difference is less than or equal to 0, and the sampled values Subtract sample value When the difference is greater than or equal to 0, the element value in the nth column and lth row of the difference matrix can be the third element value, which can also be represented by any character. Similarly, when the sampled values... Subtract sample value The difference is greater than or equal to 0, and the sampled values Subtract sample value When the difference is less than or equal to 0, the value of the element in the nth column and lth row of the difference matrix can also be the value of the third element.

[0211] Through the above process, the difference matrix can be calculated using the sampled values ​​corresponding to each sampling point in the smoothed signal sequence. The computer can count the number of first element values ​​in each row of the difference matrix; this number can be called the first element count, with one first element count per row. The computer can choose the row with the highest first element count in the difference matrix as the target row k1. This is equivalent to choosing the row with the smallest sum of elements in matrix J1 as the target row of matrix J1. This target row k1 is the k1th row in the difference matrix.

[0212] Similarly, a computer device can count the number of second element values ​​in each row of the difference matrix; this number can be called the second element count, with one second element count per row. The computer device can then use the row with the highest number of second elements in the difference matrix as the target row k2. This is equivalent to using the row with the smallest sum of elements in matrix J2 as the target row of matrix J2. This target row k2 is the k2-th row in the difference matrix.

[0213] Therefore, when the computer device detects that the first to the k1th elements from top to bottom in the nth column of the difference matrix are all the first element value, the sampling point n can be taken as a maximum point, and the sampled value corresponding to sampling point n is the maximum value of the aforementioned sequence. Similarly, when the computer device detects that the first to the k2th elements from top to bottom in the nth column of the difference matrix are all the second element value, the sampling point n can be taken as a minimum point, and the sampled value corresponding to sampling point n is the minimum value of the aforementioned sequence.

[0214] Through the above process, the sequence maxima and minima in the smooth signal sequence can be detected.

[0215] Step S202: Determine the equilibrium line of the motion signal sequence based on the extreme values ​​of the sequence, and determine the position of the equilibrium line as the equilibrium position of the sequence.

[0216] In this application, there can be multiple maxima and minima of the sequence. The computer device can obtain the median among the multiple maxima, which can be called the median of the maxima. For example, if the multiple maxima include 1, 2, and 3, then the median of the maxima is 2. As another example, if the multiple maxima include 1, 2, 3, and 4, then the median of the maxima can be (2+3) / 2, which equals 2.5. Similarly, the computer device can also obtain the median among the multiple minima, which can be called the median of the minima. For example, if the multiple minima include 4, 5, and 6, then the median of the minima is 5.

[0217] Furthermore, the indices of each sampling point in the smoothed signal sequence (such as indices 1 to N mentioned above) can be distributed on the horizontal axis, and the sampled value corresponding to each sampling point in the smoothed signal sequence can be the vertical axis corresponding to the index of each sampling point. Computer equipment can use the horizontal line at the average of the median of the maximum and the median of the minimum as the balance line of the smoothed signal sequence; in other words, the average of the median of the maximum and the median of the minimum is the vertical axis of this balance line. This balance line also represents the balance line of the motion signal sequence, and the position of this balance line can be called the sequence balance position. This balance line effectively distinguishes the sequence maxima and minima in the smoothed signal sequence, making the combined distance between multiple sequence maxima and the balance line approach the combined distance between multiple sequence minima and the balance line.

[0218] Alternatively, when the sampled values ​​in the smoothed signal sequence of the motion signal sequence are relatively uniformly distributed, the computer device can also calculate the average value of the elements contained in the smoothed signal sequence. The computer device can then use the horizontal line at this average value as the balance line of the smoothed signal sequence. In other words, the ordinate of the balance line can be the average value of the elements contained in the smoothed signal sequence. The balance line obtained in this way can better distinguish the sampled values ​​in the smoothed signal sequence, that is, it can more evenly separate the element values ​​in the smoothed signal sequence that are greater than the ordinate of the balance line and the element values ​​that are less than the ordinate of the balance line.

[0219] As can be seen from the above process, the sampling AMPD algorithm is used in this application to detect the extreme values ​​of the smooth signal sequence, and then obtain the balance line (i.e. the balance position of the sequence) of the smooth signal sequence. No additional hyperparameters need to be set in this process, thus improving the efficiency of balance line detection and increasing the applicability to signal sequences.

[0220] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a motion signal processing device provided in this application. Figure 10As shown, the motion signal processing device 1 may include: a balance position acquisition module 101, a period segmentation module 102, and a motion evaluation module 103;

[0221] The equilibrium position acquisition module 101 is used to acquire the motion signal sequence of the target user and acquire the sequence equilibrium position of the motion signal sequence.

[0222] The period segmentation module 102 is used to determine the zero-crossing point of the motion signal sequence based on the sequence equilibrium position, and to segment the motion signal sequence based on the zero-crossing point to obtain the sequence period of the motion signal sequence.

[0223] The motion assessment module 103 is used to generate a sequence feature vector of the motion signal sequence based on the sequence period, and to predict the target motion result of the target user based on the sequence feature vector.

[0224] For details on the implementation of the equilibrium position acquisition module 101, the period segmentation module 102, and the motion evaluation module 103, please refer to [link to relevant documentation]. Figure 3 Steps S101-S103 in the corresponding embodiments will not be described again here.

[0225] The equilibrium position acquisition module 101 includes: an extreme value detection unit 1011, an equilibrium line determination unit 1012, and a position determination unit 1013.

[0226] The extreme value detection unit 1011 is used to acquire a smoothed signal sequence of a motion signal sequence and detect the extreme values ​​of the smoothed signal sequence.

[0227] The balance line determination unit 1012 is used to determine the balance line of the motion signal sequence based on the sequence extreme values;

[0228] The position determination unit 1013 is used to determine the position of the balance line as the sequence balance position.

[0229] For details on the implementation of the extreme value detection unit 1011, the equilibrium line determination unit 1012, and the position determination unit 1013, please refer to [link to relevant documentation]. Figure 3 Step S101 in the corresponding embodiment will not be described again here.

[0230] The extreme value detection unit 1011 includes: a frequency domain transformation subunit 10111, a frequency filtering subunit 10112, and a time domain transformation subunit 10113.

[0231] The frequency domain transformation subunit 10111 is used to transform a motion signal sequence in the time domain space to the frequency domain space to obtain a frequency domain motion signal of the motion signal sequence.

[0232] The frequency filtering subunit 10112 is used to obtain the user's motion frequency threshold, and filter the frequency domain motion signal based on the user's motion frequency threshold to obtain a smooth motion signal of the frequency domain motion signal.

[0233] The time-domain transformation subunit 10113 is used to transform the smoothed motion signal to the time domain space to obtain a smoothed signal sequence of the motion signal sequence.

[0234] For details on the functional implementation of the frequency domain transformation subunit 10111, the frequency filtering subunit 10112, and the time domain transformation subunit 10113, please refer to [link to relevant documentation]. Figure 3 Step S101 in the corresponding embodiment will not be described again here.

[0235] The smooth signal sequence includes sampled values ​​corresponding to at least two sampling points; the at least two sampling points include sampling point nl, sampling point n, and sampling point n+l, where n is a positive integer less than or equal to the total number of at least two sampling points, and l is the sampling point interval between sampling point nl and sampling point n+l respectively; the sequence extrema include the sequence maximum and the sequence minimum.

[0236] The extreme value detection unit 1011 includes: a matrix acquisition subunit 10114, a target row determination subunit 10115, and an extreme value determination subunit 10116;

[0237] Matrix acquisition subunit 10114 is used to acquire the difference matrix corresponding to at least two sampling points; where l is a positive integer less than or equal to the total number of rows of the difference matrix, and the l-th row of the difference matrix includes element values ​​jointly mapped by the sampling difference value corresponding to sampling point nl and the sampling difference value corresponding to sampling point n+l. The sampling difference value corresponding to sampling point nl refers to the difference between the sampling value corresponding to sampling point nl and the sampling value corresponding to sampling point n, and the sampling difference value corresponding to sampling point n+l refers to the difference between the sampling value corresponding to sampling point n+l and the sampling value corresponding to sampling point n. When both the sampling difference value corresponding to sampling point nl and the sampling difference value corresponding to sampling point n+l are less than the difference boundary value, the element value is the first element value; when both the sampling difference value corresponding to sampling point nl and the sampling difference value corresponding to sampling point n+l are greater than the difference boundary value, the element value is the second element value.

[0238] The target row determination subunit 10115 is used to count the number of first elements of the first element value contained in each row of the difference matrix, and the row with the most first elements is determined as the target row k1. The number of second elements of the second element value contained in each row of the difference matrix is ​​counted, and the row with the most second elements is determined as the target row k2. k1 and k2 are both positive integers less than or equal to the total number of rows of the difference matrix.

[0239] The extreme value determination subunit 10116 is used to determine the sampled value corresponding to the sampling point n as the maximum value of the sequence when the first element value to the k1th element value in the nth column of the difference matrix are all the first element value, and to determine the sampled value corresponding to the sampling point n as the minimum value of the sequence when the first element value to the k2th element value in the nth column of the difference matrix are all the second element value.

[0240] For details on the implementation of the matrix acquisition subunit 10114, the target row determination subunit 10115, and the extreme value determination subunit 10116, please refer to [link to relevant documentation]. Figure 3 Step S101 in the corresponding embodiment will not be described again here.

[0241] Among them, sequence extrema include sequence maxima and sequence minima;

[0242] The balance line determination unit 1012 includes: a median acquisition subunit 10121 and a balance line determination subunit 10122;

[0243] The median acquisition subunit 10121 is used to obtain the median of the maximum values ​​of the sequence maxima and the median of the minimum values ​​of the sequence minima;

[0244] Balance line determination subunit 10122 is used to determine the balance line of the motion signal sequence based on the median of the maximum and minimum values.

[0245] For details on the implementation of the median acquisition subunit 10121 and the balance line determination subunit 10122, please refer to [link to details]. Figure 3 Step S101 in the corresponding embodiment will not be described again here.

[0246] The period segmentation module 102 includes: a signal wave acquisition unit 1021 and a zero-crossing point acquisition unit 1022;

[0247] The signal wave acquisition unit 1021 is used to acquire the motion signal wave of the smoothed signal sequence of the motion signal sequence;

[0248] The zero-crossing point acquisition unit 1022 is used to acquire the intersection point between the equilibrium line and the motion signal wave as the sequence zero-crossing point.

[0249] For details on the specific functional implementation of the signal wave acquisition unit 1021 and the zero-crossing point acquisition unit 1022, please refer to [link to relevant documentation]. Figure 3 Step S102 in the corresponding embodiment will not be described again here.

[0250] The sequence must have at least two zero-crossing points.

[0251] The period segmentation module 102 includes: a frequency detection unit 1023, an anomaly determination unit 1024, a correction determination unit 1025, and a correction segmentation unit 1026;

[0252] The frequency detection unit 1023 is used to obtain a sampling frequency threshold and, based on the sampling frequency threshold, sequentially detect the interval frequency between adjacent zero-crossing points of at least two sequences.

[0253] The anomaly determination unit 1024 is used to determine the zero-crossing points of adjacent sequences whose interval frequency is less than the sampling frequency threshold among the detected at least two sequence zero-crossing points as abnormal sequence zero-crossing points.

[0254] The correction determination unit 1025 is used to filter out abnormal sequence zero crossings from at least two sequence zero crossings to obtain corrected sequence zero crossings;

[0255] The correction segmentation unit 1026 is used to segment the smooth signal sequence based on the zero-crossing point of the correction sequence to obtain the sequence period.

[0256] For details on the specific functional implementation of the frequency detection unit 1023, the anomaly determination unit 1024, the correction determination unit 1025, and the correction segmentation unit 1026, please refer to [link to relevant documentation]. Figure 3 Step S102 in the corresponding embodiment will not be described again here.

[0257] The motion evaluation module 103 includes: an information acquisition unit 1031, a feature type determination unit 1032, and a vector generation unit 1033.

[0258] The information acquisition unit 1031 is used to acquire a motion evaluation standard table and motion prediction types for the target user; the motion evaluation standard table includes the correlation between at least two motion prediction types and their corresponding motion feature types.

[0259] The feature type determination unit 1032 is used to determine the motion feature types that are related to the motion prediction type of the target user in the motion evaluation standard table as the detection feature types for the target user;

[0260] The vector generation unit 1033 is used to generate signal feature parameters of the motion signal sequence under the detection feature type according to the sequence period, and generate sequence feature vectors according to the signal feature parameters.

[0261] For details on the specific functional implementation of the information acquisition unit 1031, the feature type determination unit 1032, and the vector generation unit 1033, please refer to [link to relevant documentation]. Figure 3 Step S103 in the corresponding embodiment will not be described again here.

[0262] The number of detection feature types is at least two;

[0263] The vector generation unit 1033 includes: a periodic information acquisition subunit 10331, a feature parameter determination subunit 10332, and a feature splicing subunit 10333;

[0264] The period information acquisition subunit 10331 is used to acquire the period duration and period amplitude of the sequence period;

[0265] The feature parameter determination subunit 10332 is used to determine the signal feature parameters of the motion signal sequence under each detection feature type based on the period duration and period amplitude.

[0266] The feature splicing subunit 10333 is used to splice the signal feature parameters of the motion signal sequence under each detection feature type to obtain the sequence feature vector.

[0267] For details on the implementation of the periodic information acquisition subunit 10331, the feature parameter determination subunit 10332, and the feature splicing subunit 10333, please refer to [link to relevant documentation]. Figure 3 Step S103 in the corresponding embodiment will not be described again here.

[0268] The equilibrium position acquisition module 101 includes: a key point determination unit 1014, a key information detection unit 1015, and a first sequence generation unit 1016.

[0269] The key point determination unit 1014 is used to obtain the motion prediction type for the target user and determine the motion key points for the target user based on the motion prediction type.

[0270] The key information detection unit 1015 is used to acquire the motion video data of the target user and detect the key point motion information of the target user in the motion video data based on the motion key points.

[0271] The first sequence generation unit 1016 is used to generate a motion signal sequence of the target user based on key point motion information.

[0272] For details on the specific functional implementation of the key point determination unit 1014, the key information detection unit 1015, and the first sequence generation unit 1016, please refer to [link to relevant documentation]. Figure 3 Step S101 in the corresponding embodiment will not be described again here.

[0273] The equilibrium position acquisition module 101 includes: a sensor parameter acquisition unit 1017, a motion feature determination unit 1018, and a second sequence generation unit 1019.

[0274] The sensor parameter acquisition unit 1017 is used to acquire sensor motion parameters of the target user during the motion process;

[0275] The motion feature determination unit 1018 is used to determine the motion feature parameters of the target user based on the sensor motion parameters.

[0276] The second sequence generation unit 1019 is used to generate a motion signal sequence of the target user based on motion feature parameters.

[0277] For details on the specific functional implementation of the sensor parameter acquisition unit 1017, the motion feature determination unit 1018, and the second sequence generation unit 1019, please refer to [link to relevant documentation]. Figure 3 Step S101 in the corresponding embodiment will not be described again here.

[0278] The motion evaluation module 103 includes a feature input unit 1034 and a result output unit 1035.

[0279] The feature input unit 1034 is used to input the sequence feature vector into the motion prediction model; the motion prediction model is trained based on the sample sequence feature vector of the sample motion signal sequence.

[0280] The result output unit 1035 is used to output the target motion result for the target user in the motion prediction model.

[0281] For details on the specific functional implementation of the feature input unit 1034 and the result output unit 1035, please refer to [link / reference needed]. Figure 3 Step S103 in the corresponding embodiment will not be described again here.

[0282] The device 1 mentioned above also includes: a sample location acquisition module 104, a sample period segmentation module 105, a sample feature generation module 106, a sample result output module 107, and a parameter correction module 108.

[0283] The sample position acquisition module 104 is used to acquire the sample motion signal sequence of the sample user and the actual motion result of the sample user, and to acquire the sample sequence equilibrium position of the sample motion signal sequence.

[0284] The sample period segmentation module 105 is used to determine the zero-crossing point of the sample motion signal sequence based on the equilibrium position of the sample sequence, and to segment the sample motion signal sequence based on the zero-crossing point of the sample sequence to obtain the sample sequence period of the sample motion signal sequence.

[0285] The sample feature generation module 106 is used to generate a sample sequence feature vector of the sample motion signal sequence based on the sample sequence period.

[0286] The sample result output module 107 is used to input the sample sequence feature vector and the actual motion result into the initial motion prediction model, and output the sample motion result for the sample motion signal sequence in the initial motion prediction model;

[0287] The parameter correction module 108 is used to correct the model parameters of the initial motion prediction model based on the sample motion results and the actual motion results, so as to obtain the motion prediction model.

[0288] For details on the implementation of the sample location acquisition module 104, sample period segmentation module 105, sample feature generation module 106, sample result output module 107, and parameter correction module 108, please refer to [link to relevant documentation]. Figure 8 Step S103 in the corresponding embodiment will not be described again here.

[0289] This application first acquires the motion signal sequence of the target user and obtains the sequence equilibrium position of the motion signal sequence. Next, it determines the sequence zero-crossing points of the motion signal sequence based on the sequence equilibrium position, and segments the motion signal sequence based on these zero-crossing points to obtain the sequence period. Furthermore, it generates a sequence feature vector of the motion signal sequence based on the sequence period, and predicts the target user's target motion result based on the sequence feature vector. Therefore, the method proposed in this application can periodically segment the motion signal sequence using the sequence equilibrium position. Since this sequence equilibrium position can evenly distribute the sampled values ​​in the motion signal sequence, it can accurately segment the period of the motion signal sequence. Furthermore, it can generate a sequence feature vector using the accurately segmented sequence period, thereby achieving accurate prediction of the target user's target motion result.

[0290] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in this application. For example... Figure 11As shown, computer device 1000 may include: a processor 1001, a network interface 1004, and a memory 1005. Furthermore, computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 11 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0291] exist Figure 11 In the computer device 1000 shown, the network interface 1004 provides network communication functionality; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0292] Obtain the motion signal sequence of the target user and the sequence equilibrium position of the motion signal sequence;

[0293] The zero-crossing point of the motion signal sequence is determined based on the equilibrium position of the sequence. The motion signal sequence is then segmented based on the zero-crossing point to obtain the sequence period of the motion signal sequence.

[0294] Generate a sequence feature vector of the motion signal sequence based on the sequence period, and predict the target motion result of the target user based on the sequence feature vector.

[0295] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 and Figure 8 The description of the motion signal processing method in the corresponding embodiment can also be performed as described above. Figure 10 The description of the motion signal processing device 1 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0296] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the aforementioned motion signal processing device 1. The computer program includes program instructions, which, when executed by a processor, enable the execution of the aforementioned... Figure 3 and Figure 8 The description of the motion signal processing method in any of the corresponding embodiments is already provided, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0297] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0298] One aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in one aspect of the embodiments of this application.

[0299] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0300] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0301] The methods and related apparatuses provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0302] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method of processing a motion signal, characterized by, The method comprises: obtaining a motion signal sequence of a target user through motion video data of the target user or a sensor device worn by the target user; obtaining a smoothed signal sequence of the motion signal sequence, detecting sequence extreme values of the smoothed signal sequence, determining a balance line of the motion signal sequence according to the sequence extreme values, and taking a position of the balance line as a sequence balance position; determining sequence zero-crossing points of the motion signal sequence according to the sequence balance position, filtering abnormal sequence zero-crossing points based on a sampling frequency threshold to obtain corrected sequence zero-crossing points, and segmenting the motion signal sequence based on the corrected sequence zero-crossing points to obtain sequence periods of the motion signal sequence; obtaining a motion evaluation standard table and a motion prediction type of the target user, wherein the motion evaluation standard table comprises an association relationship between at least two motion prediction types and corresponding motion feature types; determining, as a detection feature type of the target user, a motion feature type in the motion evaluation standard table that has the association relationship with the motion prediction type of the target user, wherein the number of the detection feature types is at least two; obtaining a period length and a period amplitude of the sequence periods, and determining signal feature parameters of the motion signal sequence under each detection feature type according to the period length and the period amplitude; splicing the signal feature parameters of the motion signal sequence under each detection feature type and signal feature parameters corresponding to additional statistical features to obtain a sequence feature vector; inputting the sequence feature vector into a motion prediction model, and outputting a target motion result of the target user in the motion prediction model, wherein the motion prediction model is trained based on sample sequence feature vectors of sample motion signal sequences.

2. The method of claim 1, wherein, The method comprises: transforming the motion signal sequence in a time domain space to a frequency domain space to obtain a frequency domain motion signal of the motion signal sequence; obtaining a user motion frequency threshold, filtering the frequency domain motion signal based on the user motion frequency threshold to obtain a smoothed motion signal of the frequency domain motion signal; transforming the smoothed motion signal to the time domain space to obtain the smoothed signal sequence of the motion signal sequence.

3. The method of claim 1, wherein, The smoothed signal sequence includes sampled values ​​corresponding to at least two sampling points; wherein, the at least two sampling points include sampling points. Sampling points and sampling points , The number of samples is a positive integer less than or equal to the total number of the at least two sampling points. For the sampling point and sampling points Each with the sampling points The sampling point interval between them; the sequence extrema include the sequence maxima and the sequence minima; The method comprises: Obtain the difference matrix corresponding to the at least two sampling points; wherein, The integer less than or equal to the total number of rows of the difference matrix, the th row of the difference matrix. The row includes the sampling points The corresponding sampling difference and the sampling point The element values ​​that are jointly mapped by the corresponding sampling differences, the sampling points The corresponding sampling difference refers to the sampling points Corresponding sampled values ​​and sampled points The difference between corresponding sampled values, the sampled points The corresponding sampling difference refers to the sampling points Corresponding sampled values ​​and sampled points The difference between the corresponding sampled values; when the sampling point The corresponding sampling difference and the sampling point When all corresponding sampling differences are less than the difference threshold, the element value is the first element value; when the sampling point The corresponding sampling difference and the sampling point When all the corresponding sampling differences are greater than the difference threshold, the element value is the second element value; counting a first element number of the first element values contained in each row of the difference matrix, determining a target row k1 having the most first element numbers, counting a second element number of the second element values contained in each row of the difference matrix, and determining a target row k2 having the most second element numbers, wherein k1 and k2 are positive integers less than or equal to a total number of rows of the difference matrix; When it is detected that the first element value to the k1th element value in the first column of the difference matrix are the first element value, the sampling value corresponding to the sampling point is determined as the sequence maximum value, when it is detected that the first element value to the k2th element value in the first column of the difference matrix are the second element value, the sampling value corresponding to the sampling point is determined as the sequence minimum value. when it is detected that the first element value to the k2th element value in the first column of the difference matrix are the second element value, 4. The method of claim 1, wherein, the sequence extreme values comprise sequence maximum values and sequence minimum values; The method comprises: obtaining a maximum value median of the sequence maximum values and a minimum value median of the sequence minimum values; determining the balance line of the motion signal sequence according to the maximum value median and the minimum value median.

5. The method of claim 1, wherein, The sequence zero-crossing point of the motion signal sequence is determined according to the sequence balance position, and the method comprises the steps that: a motion signal wave of the smoothed signal sequence of the motion signal sequence is obtained; an intersection point between the balance line and the motion signal wave is obtained as the sequence zero-crossing point.

6. The method of claim 5, wherein, The number of the sequence zero-crossing points is at least two; The abnormal sequence zero-crossing points are filtered based on a sampling frequency threshold value to obtain modified sequence zero-crossing points, and the motion signal sequence is segmented based on the modified sequence zero-crossing points to obtain a sequence period of the motion signal sequence, which comprises the steps that: a sampling frequency threshold value is obtained, and the interval frequency between adjacent sequence zero-crossing points in at least two sequence zero-crossing points is detected based on the sampling frequency threshold value; adjacent sequence zero-crossing points in the at least two sequence zero-crossing points whose interval frequency is less than the sampling frequency threshold value are determined as abnormal sequence zero-crossing points; the abnormal sequence zero-crossing points are filtered from the at least two sequence zero-crossing points to obtain modified sequence zero-crossing points; the smoothed signal sequence is segmented based on the modified sequence zero-crossing points to obtain the sequence period.

7. The method of claim 1, wherein, The motion signal sequence of the target user is obtained, which comprises the steps that: a motion prediction type for the target user is obtained, and a motion key point for the target user is determined according to the motion prediction type; motion video data of the target user is obtained, and key point motion information of the target user is detected in the motion video data based on the motion key point; the motion signal sequence of the target user is generated based on the key point motion information.

8. The method of claim 1, wherein, The method further comprises the steps that: the sample motion signal sequence of a sample user and an actual motion result for the sample user are obtained, and a sample sequence balance position of the sample motion signal sequence is obtained; sample sequence zero-crossing points of the sample motion signal sequence are determined according to the sample sequence balance position, and the sample motion signal sequence is segmented based on the sample sequence zero-crossing points to obtain a sample sequence period of the sample motion signal sequence; the sample sequence feature vector of the sample motion signal sequence is generated according to the sample sequence period; the sample sequence feature vector and the actual motion result are input into an initial motion prediction model, and a sample motion result for the sample motion signal sequence is output in the initial motion prediction model; model parameters of the initial motion prediction model are corrected based on the sample motion result and the actual motion result to obtain the motion prediction model.

9. A motion signal processing apparatus, characterized by comprising: The method comprises the steps that: a balance position acquisition module is configured to obtain a motion signal sequence of a target user through motion video data of the target user or a sensor device worn by the target user, obtain a smoothed signal sequence of the motion signal sequence, detect sequence extreme values of the smoothed signal sequence, determine a balance line of the motion signal sequence according to the sequence extreme values, and take the position of the balance line as a sequence balance position; A period division module is configured to determine sequence zero-crossing points of the motion signal sequence according to the sequence balance position, filter abnormal sequence zero-crossing points based on a sampling frequency threshold to obtain corrected sequence zero-crossing points, and divide the motion signal sequence based on the corrected sequence zero-crossing points to obtain sequence periods of the motion signal sequence. A motion evaluation module is configured to obtain a motion evaluation standard table and a motion prediction type of the target user, the motion evaluation standard table including an association relationship between at least two motion prediction types and corresponding motion feature types, determine a detection feature type of the target user as a motion feature type in the motion evaluation standard table that has the association relationship with the motion prediction type of the target user, obtain a period length and a period amplitude of the sequence period, determine signal feature parameters of the motion signal sequence under each detection feature type according to the period length and the period amplitude, splice the signal feature parameters of the motion signal sequence under each detection feature type and signal feature parameters corresponding to additional statistical features to obtain a sequence feature vector, input the sequence feature vector into a motion prediction model, and output a target motion result of the target user in the motion prediction model, wherein the motion prediction model is trained based on sample sequence feature vectors of sample motion signal sequences.

10. A computer device, comprising: A computer readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor to execute the steps of the method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor to execute the steps of the method in any one of claims 1-8.

Images