Motion scene recognition method, apparatus, device, medium and product

Abstract

This application discloses a motion scene recognition method, belonging to the field of robotics. The method is executed by an exoskeleton device, which includes sensors. The method includes: acquiring a test parameter sequence and a sensor parameter sequence, wherein the test parameter sequence includes motion parameters of different test objects under different motion scenes, and the sensor parameter sequence is a set of motion parameters acquired from the sensors; calibrating the sensor parameter sequence based on the test parameter sequence to obtain a calibrated sensor parameter sequence; and determining the motion scene in which the exoskeleton device is located based on the calibrated sensor parameter sequence. This method uses a pre-tested test parameter sequence to calibrate the sensor parameter sequence acquired by the sensors in the exoskeleton device, thereby identifying the motion scene in which the exoskeleton device is located based on the calibrated sensor parameter sequence, ensuring the reliability of scene recognition by the exoskeleton device.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method, apparatus, device, medium, and product for motion scene recognition. Background Technology

[0002] An exoskeleton is a wearable mechanical structure that assists or enhances the human body's motor abilities. Exoskeletons not only need to provide functional enhancements for different limb movements in terms of power output, but also need to provide perceptual predictions for different movement scenarios in terms of intent decision-making, such as recognizing uphill and downhill scenarios.

[0003] In related technologies, the motion scene of the wearer is usually determined based on the sensor parameters obtained by various sensors in the exoskeleton device. However, due to the accuracy of the sensors themselves and the data offset problem that the sensors are prone to after strenuous exercise, the sensors often need to be calibrated before the wearer uses the exoskeleton device.

[0004] This will significantly increase the time and cost for wearers using exoskeleton devices. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for motion scene recognition, the technical solution of which is as follows:

[0006] According to one aspect of this application, a motion scene recognition method is provided, the method being performed by an exoskeleton device, the exoskeleton device including sensors, the method comprising:

[0007] The test parameter sequence and sensor parameter sequence are obtained. The test parameter sequence includes motion parameters of different test objects under different motion scenarios, and the sensor parameter sequence is a set of motion parameters obtained from the sensors.

[0008] Based on the test parameter sequence, the sensor parameter sequence is calibrated to obtain the calibrated sensor parameter sequence;

[0009] Based on the calibrated sensor parameter sequence, the motion scenario of the exoskeleton device is determined.

[0010] According to one aspect of this application, a motion scene recognition device is provided, the device comprising:

[0011] The acquisition module is used to acquire test parameter sequences and sensor parameter sequences. The test parameter sequences include motion parameters of different test objects under different motion scenarios, and the sensor parameter sequences are a set of motion parameters obtained from sensors.

[0012] A calibration module is used to calibrate the sensor parameter sequence based on the test parameter sequence to obtain a calibrated sensor parameter sequence.

[0013] The identification module is used to determine the motion scene of the exoskeleton device based on the calibrated sensor parameter sequence.

[0014] According to one aspect of this application, a computer device is provided, the computer device comprising: a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement a motion scene recognition method.

[0015] According to one aspect of this application, an exoskeleton device is provided for performing a motion scene recognition method.

[0016] According to one aspect of this application, a computer storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement a motion scene recognition method.

[0017] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform a motion scene recognition method.

[0018] The beneficial effects of the technical solution provided in this application include at least the following:

[0019] By using pre-collected test parameter sequences, the sensor parameter sequences acquired by the exoskeleton device during use are calibrated, and the motion scenario of the exoskeleton device is determined based on the calibrated sensor parameter sequences. On one hand, using test parameter sequences to assist calibration eliminates the need for manual sensor calibration before use, allowing the exoskeleton device to be used immediately after wearing, reducing latency and manpower costs for the wearer. On the other hand, calibration using test parameter sequences ensures that a highly reliable sensor parameter sequence is obtained even when the exoskeleton device is worn. This avoids data shifts in the exoskeleton device's sensors during wear due to intense movement or extreme environments, which could lead to inaccurate motion environment judgments. Simultaneously, using calibrated sensor parameters to identify the motion scenario improves the accuracy of motion scenario recognition, ensuring that the exoskeleton device can provide different assistance to the wearer in different motion scenarios, thus enhancing the adaptability and reliability of the exoskeleton device. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 An architectural diagram of a computer system provided in an exemplary embodiment of this application is shown;

[0022] Figure 2 A flowchart of a motion scene recognition method provided in an exemplary embodiment of this application is shown;

[0023] Figure 3 A flowchart of a motion scene recognition method provided in an exemplary embodiment of this application is shown;

[0024] Figure 4 A flowchart of a motion scene recognition method provided in an exemplary embodiment of this application is shown;

[0025] Figure 5 A schematic diagram of a motion scene recognition method provided in an exemplary embodiment of this application is shown;

[0026] Figure 6 A schematic diagram of a motion scene recognition method provided in an exemplary embodiment of this application is shown;

[0027] Figure 7A schematic diagram of a motion scene recognition method provided in an exemplary embodiment of this application is shown;

[0028] Figure 8 A schematic diagram of a motion scene recognition method provided in an exemplary embodiment of this application is shown;

[0029] Figure 9 This invention provides a structural block diagram of a motion scene recognition device according to an exemplary embodiment of the present application.

[0030] Figure 10 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0034] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the settings and operation information involved in this application were obtained with full authorization.

[0035] It should be understood that although the terms first, second, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0036] First, let me introduce the relevant terms used in this application:

[0037] Exoskeleton devices: Wearable mechanical structures. Exoskeleton devices can assist or enhance the movements and abilities of the wearer.

[0038] Cutoff frequency: This refers to the transition point of a filter from the passband (the frequency range that signals are allowed to pass through) to the stopband (the frequency range that signals are blocked from passing through). For a low-pass filter, signals can pass through when their frequency is below the cutoff frequency; when the signal frequency is above the cutoff frequency, the signal output will be significantly attenuated. For a high-pass filter, signals can pass through when their frequency is above the cutoff frequency; when the signal frequency is below the cutoff frequency, the signal output will be significantly attenuated. In other words, the cutoff frequency is used to determine the frequency of the signal passing through the filter.

[0039] Figure 1 A schematic diagram of the architecture of a control system provided in one embodiment of this application is shown. The control system may include an exoskeleton device 100.

[0040] Optionally, the exoskeleton device 100 includes a controller 110 and at least one sensor.

[0041] Optionally, the controller 110 is used to control various components in the exoskeleton device 100. For example, the controller 110 controls the various components of the exoskeleton device 100 to switch the exoskeleton device 100 to a "sitting" posture, or the controller 110 controls the various components of the exoskeleton device 100 to switch the exoskeleton device 100 to a flat-ground assist mode, which is used to assist the wearer in walking on flat ground, and so on. Exemplarily, the controller 110 includes a processor, which may be a microprocessor (MCU), a CPU (Central Processing Unit), etc. Exemplarily, the controller 110 is a computer device, such as a microcomputer.

[0042] Optionally, the controller 110 is connected to at least one sensor, which is used to collect the motion state of the exoskeleton device 100, such as posture, speed, acceleration, and position. The controller 110 can determine the control method for the exoskeleton device 100 based on the sensor parameters collected by the at least one sensor.

[0043] For example, in this embodiment of the application, the controller 110 determines the motion scenario of the exoskeleton device 100 based on sensor parameters collected by at least one sensor, and sets different assist modes of the exoskeleton device 100 for different motion scenarios.

[0044] Exoskeleton devices can provide assist torque that follows the wearer's gait, reducing physical exertion during walking, running, climbing, and other activities, making movement easier. To achieve this, and to provide the most suitable assistance for various motion scenarios while optimizing the quality of assistance, adaptive recognition of the wearer's motion environment is necessary. For adaptive recognition of inclines and declines, current technologies typically use IMU (Inertial Measurement Unit) inertial navigation to obtain the wearer's Euler angles or barometers to detect altitude changes. However, commonly used low-cost inertial navigation systems (accelerometers, gyroscopes, magnetometers) can experience data shifts if not calibrated before use or after prolonged operation. Furthermore, they perform poorly in scenarios requiring vigorous movement, making accurate recognition difficult. Barometers, on the other hand, are susceptible to environmental influences, failing to accurately measure altitude changes in turbulent environments; and they have low sensitivity to dynamic changes and lack real-time performance, making it difficult to detect scene changes promptly during the wearer's movement. To address these issues, this application proposes an adaptive recognition algorithm that integrates data fusion from sensors such as IMU, barometer, and hip joint motor encoder. This algorithm, with strong robustness and real-time performance, can accurately identify the application scenario of the wearable object and improve the quality of assistance.

[0045] Figure 2 A flowchart illustrating a motion scene recognition method provided in an exemplary embodiment of this application is shown. The method is performed by an exoskeleton device, which may be... Figure 1 The exoskeleton device shown in the image. The method includes:

[0046] Optionally, the exoskeleton device includes sensors, or the exoskeleton device is connected to sensors.

[0047] Optionally, the exoskeleton device includes at least one sensor. Different sensors are used to collect different motion parameters. Motion parameters refer to parameters collected when the wearer of the exoskeleton device is wearing the device, such as height, speed, acceleration, angular velocity, pressure, electromyography signals, etc. For example, the sensor includes a sampling period, which indicates the time interval between two consecutive samples. If the sampling period is 2ms, then the sensor will sample once every 2ms to obtain motion parameters (i.e., the motion parameters in the sensor parameter sequence below).

[0048] Step 210: Obtain the test parameter sequence and sensor parameter sequence. The test parameter sequence includes the motion parameters of different test objects under different motion scenarios, and the sensor parameter sequence is a set of motion parameters obtained from the sensors.

[0049] The test parameter sequence and / or sensor parameter sequence represent motion parameters collected over a continuous period of time, such as velocity within 1 second. If the sampling period is 2 ms, then the test parameter sequence and / or sensor parameter sequence includes velocities collected every 2 ms. Since the sampling duration is 1 second (=1000 ms), the test parameter sequence and / or sensor parameter sequence includes 1000 / 2=500 velocities.

[0050] Optionally, the test parameter sequence includes one or more sets of motion parameters, each set of motion parameters corresponding to a test object and a motion scenario. That is, each set of motion parameters is collected for a test object in a motion scenario. For example, if there are two test objects (test object a and test object b) and two motion scenarios to be tested (motion scenario 1 and motion scenario 2), then four sets of motion parameters can be collected: one set of motion parameters for test object a in motion scenario 1, one set of motion parameters for test object a in motion scenario 2, one set of motion parameters for test object b in motion scenario 1, and one set of motion parameters for test object b in motion scenario 2. When obtaining the test parameter sequence, all four sets of motion parameters can be used as the test parameter sequence, or any one of the four sets of motion parameters can be selected as the test parameter sequence, or multiple sets of the four sets of motion parameters can be selected as the test parameter sequence. This application embodiment does not limit this, but the protection scope of this application embodiment is not limited thereto.

[0051] Optionally, the motion parameters for different test subjects in different motion scenarios can be collected by sensors in the wearable exoskeleton device from different test subjects in different motion scenarios. In this implementation, the sensors in the exoskeleton device are calibrated, meaning their accuracy is guaranteed. Alternatively, the motion parameters for different test subjects in different motion scenarios can also be collected by highly reliable sensors from different test subjects in different motion scenarios. These highly reliable sensors can be installed on the exoskeleton device or other test devices. A highly reliable sensor means that the motion parameters collected by the sensor do not change over time under the same test environment; that is, a highly reliable sensor does not exhibit data shift, or in other words, a highly reliable sensor does not experience data shift when testing the motion parameters in the above-mentioned test parameter sequence. Data shift refers to the phenomenon where the motion parameters output by the sensor change over time when the input remains constant.

[0052] Optionally, the test parameter sequence can be considered a set of highly reliable motion parameters, or in other words, the reliability of the test parameter sequence is higher than that of the sensor parameter sequence. This is because changes in the internal structure of the sensor due to time or collisions during use, or changes in the external environment during sensor use, can cause data shifts in the motion parameters in the sensor parameter sequence acquired by the sensor. In other words, the reliability of the sensor parameter sequence decreases with use. Related technologies typically employ manual calibration of the sensors in the exoskeleton device before use to ensure the reliability of the sensors during use. However, if vigorous exercise is performed while wearing the exoskeleton device, data shifts in the motion parameters acquired by the sensors may still occur. The method shown in the embodiments of this application can solve the above problems.

[0053] Optionally, the test parameter sequence and the sensor parameter sequence have the same sequence length, or the number of motion parameters included in the test parameter sequence and the sensor parameter sequence is the same. That is, the sampling period and sampling duration corresponding to each set of motion parameters included in the test parameters are equal to the sampling period and sampling duration of the sensor. For example, if the sensor sampling period is 10ms and the sampling duration is 1s (=1000ms), then there are 1000 / 10=100 data points in one set of motion parameters included in the sensor parameter sequence; then the corresponding sampling period of the test parameter sequence is 10ms, the sampling duration is 1s, and each set of motion parameters in the test parameter sequence includes 100 data points. Optionally, each set of motion parameters in the test parameter sequence is obtained by extracting a portion of the motion parameters from a set of candidate motion parameters based on the sensor's sampling period and sampling duration. For example, if a set of candidate motion parameters is collected for different test objects under different motion scenarios, and the sampling period for this set of candidate motion parameters is 2ms and the sampling duration is 2s, then firstly, 1s of candidate motion parameters are obtained from the candidate motion parameters corresponding to these 2s, and then the motion parameters in the test parameter sequence are obtained by resampling from these 1s candidate motion parameters according to the sampling period of 10ms. The resampling process described above is as follows: Assume that there is a set of continuous motion parameters {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} in the original candidate motion parameters. Since the sampling period corresponding to the candidate motion parameters is 2ms, assume that the above motion parameter "1" was collected at 0ms, the above motion parameter "2" was collected at 2ms, the above motion parameter "3" was collected at 4ms, ..., the above motion parameter "10" was collected at 20ms. Resampling according to the sensor's sampling period (10ms) means obtaining the motion parameters according to the above parameters at a sampling period of 10ms. For example, we can obtain the above motion parameter "1" at 0ms, then obtain the motion parameter "5" at 10ms after a 10ms interval, and then obtain the motion parameter "10" at 20ms after a 10ms interval. Of course, the starting point of resampling can also start from other time points, such as first obtaining the motion parameter "3" collected at 4ms, and then obtaining the motion parameter "7" at 14ms after a 10ms interval. The resampling method shown above is based on discrete motion parameters. In actual implementation, it can also be based on continuous motion parameters. That is, after fitting the discrete motion parameters, a motion parameter curve is obtained. Then, based on the sensor's sampling period and sampling duration, the motion parameters in the test parameter sequence are resampled from the motion parameter curve. This application does not limit this, but the protection scope of this application is not limited thereto.

[0054] Optionally, when the test parameter sequence includes multiple sets of motion parameters, the number of motion parameters in each set of motion parameters is the same, that is, the sequence length of the test parameter sequence is the number of motion parameters included in each set of motion parameters; similarly, the sequence length of the sensor parameter sequence is also the number of motion parameters included in the sensor parameter sequence.

[0055] Step 220: Based on the test parameter sequence, calibrate the sensor parameter sequence to obtain the calibrated sensor parameter sequence.

[0056] During the use of the exoskeleton device, the sensor parameter sequence is calibrated based on the test parameter sequence to obtain the calibrated sensor parameter sequence. For example, after the exoskeleton device is started, the sensor parameter sequence is collected, and then the sensor parameter sequence is calibrated based on the test parameter sequence to obtain the calibrated sensor parameter sequence; or, during the use of the exoskeleton device, calibration is performed once at preset intervals, that is, the sensor parameter sequence is calibrated based on the test parameter sequence at preset intervals to obtain the calibrated sensor parameter sequence.

[0057] Optionally, the preset duration can be set by the developer, the user wearing the exoskeleton device, or the user. For example, when the user wearing the exoskeleton device is a person, the preset duration can be set by the user wearing the exoskeleton device; when the user wearing the exoskeleton device is an animal, the preset duration can be set by the animal's owner or keeper.

[0058] Optionally, the test parameter sequence used to calibrate the sensor parameter sequence consists of motion parameters of different test subjects under different motion scenarios; or, for the wearer of the exoskeleton device, a test parameter sequence is selected, and one or more sets of candidate motion parameters from test subjects similar to the wearer under different motion scenarios are used to calibrate the sensor parameter sequence. For example, one or more sets of motion parameters from test subjects with similar height and / or weight and / or BMI (Body Mass Index) to the wearer under different motion scenarios are selected.

[0059] Step 230: Determine the motion scenario of the exoskeleton device based on the calibrated sensor parameter sequence.

[0060] For the calibrated sensor parameter sequence, analyze the changes in motion parameters (also known as sensor parameters) in the calibrated sensor parameter sequence to determine the motion scenario in which the exoskeleton device is located.

[0061] For example, given a calibrated sensor parameter sequence, the trend of motion parameter changes within the sampling duration corresponding to the calibrated sensor parameter sequence determines the motion scenario of the exoskeleton device. If the sensor is a barometric pressure sensor (used to collect the height of the wearable object), the sensor parameter sequence is a sensor height sequence, meaning the height collected by the barometric pressure sensor over a period of time. If the height in the sensor parameter sequence increases, the exoskeleton device is determined to be in an ascending motion scenario; if the height in the sensor parameter sequence decreases, the exoskeleton device is determined to be in a descending motion scenario. Alternatively, if the sensor is a velocity sensor, the sensor parameter sequence is a sensor velocity sequence, meaning the velocity collected by the velocity sensor over a period of time. If the velocity in the sensor parameter sequence increases, the exoskeleton device is determined to be in an accelerating motion scenario; if the velocity in the sensor parameter sequence decreases, the exoskeleton device is determined to be in a decelerating motion scenario. Alternatively, the sensor is an accelerometer, and the sensor parameter sequence is a sensor acceleration sequence, that is, the acceleration collected by the accelerometer over a period of time. If the acceleration in the sensor parameter sequence is increasing, it is determined that the exoskeleton device is in an accelerating motion scenario; if the acceleration in the sensor parameter sequence is decreasing, it is determined that the exoskeleton device is in an accelerating motion scenario with decreasing acceleration, or a decelerating motion scenario with decreasing acceleration, and so on.

[0062] In summary, the method provided in this application uses pre-collected test parameter sequences to calibrate sensor parameter sequences acquired by sensors during the use of the exoskeleton device, and then determines the motion scenario of the exoskeleton device based on the calibrated sensor parameter sequences. On one hand, by using test parameter sequences to assist calibration, manual sensor calibration is eliminated before use, allowing the exoskeleton device to be used immediately after wearing, reducing latency and manpower costs for the wearer. On the other hand, using test parameter sequences to assist calibration ensures that a highly reliable sensor parameter sequence can be obtained even when the exoskeleton device is worn. This avoids data shifts in the exoskeleton device's sensors during wear due to intense movement of the wearer or extreme environments, which could lead to inaccurate motion environment judgments. Furthermore, using calibrated sensor parameters to identify the motion scenario of the exoskeleton device improves the accuracy of motion scenario recognition, ensuring that the exoskeleton device can provide different assistance to the wearer in different motion scenarios, thus improving the adaptability and reliability of the exoskeleton device.

[0063] The following section illustrates the specific method for calibrating the sensor parameter sequence based on the test parameter sequence.

[0064] • Calibration process.

[0065] Based on Figure 2 In an alternative embodiment, such as Figure 3 As shown, step 220 can be implemented as steps 221 and 222.

[0066] Step 221: Input the sensor parameter sequence into the first filter to obtain the sensor parameter sequence output by the first filter.

[0067] Optionally, the first filter is used to adjust the various motion parameters in the sensor parameter sequence; or, the first filter is used to eliminate noise introduced into the sensor parameter sequence due to structural changes in the sensor; or, the first filter is used to eliminate noise introduced into the sensor parameter sequence due to the movement of the exoskeleton device.

[0068] Optionally, if the introduced noise is low-frequency information, the first filter is a high-pass filter, meaning it retains the high-frequency information in the input sensor parameter sequence while filtering out the low-frequency information (i.e., the introduced noise) in the sensor parameter sequence. If the introduced noise is high-frequency information, the first filter is a low-pass filter, meaning it retains the low-frequency information in the input sensor parameters while filtering out the high-frequency information in the sensor parameter sequence.

[0069] Optionally, the first filter is a digital filter, which is a filter whose input and output are both digital signals. Digital signals can be understood as discrete data; the sensor parameter sequence shown in this embodiment is a set of discrete data. Alternatively, the first filter is an analog filter, which is a filter whose input and output are both analog signals. When the first filter is an analog filter, a digital-to-analog conversion needs to be performed on the sensor parameter sequence to convert it into an analog signal before inputting it into the first filter. Simultaneously, the analog signal output by the first filter may need to undergo analog-to-digital conversion to obtain the sensor parameter sequence output by the first filter.

[0070] For example, the first filter is a Butterworth filter, such as a Butterworth high-pass filter. It should be noted that the first filter can also be other filters, such as a Bessel filter, a Chebyshev filter, etc.

[0071] Step 222: Based on the test parameter sequence, determine that the sensor parameter sequence output by the first filter is the calibrated sensor parameter sequence.

[0072] Optionally, the test parameter sequence is used to determine whether the sensor parameter sequence output by the first filter meets the calibration conditions. If the sensor parameter sequence output by the first filter meets the calibration conditions, the sensor parameter sequence output by the first filter is determined to be the calibrated sensor parameter sequence.

[0073] Optionally, step 222 above can be implemented as: steps 222-1 to 222-4.

[0074] Step 222-1: Calculate the similarity between the test parameter sequence and the sensor parameter sequence after the (i-1)th filtering, where i is an integer greater than 1.

[0075] Optionally, the sensor parameter sequence after the first filtering is the same as the sensor parameter sequence obtained in step 221 above. The sensor parameter sequence after the (i-1)th filtering is the same as the sensor parameter sequence obtained after inputting the first filter for the (i-1)th time.

[0076] Optionally, inputting the sensor parameter sequence into the first filter does not change the sequence length of the sensor parameter sequence. That is, if the sequence length of the test parameter sequence is the same as the sequence length of the sensor parameter sequence, the sequence length of the test parameter sequence is also the same as the sequence length of the sensor parameter sequence after the (i-1)th filtering.

[0077] Optionally, the similarity between the test parameter sequence and the sensor parameters after the (i-1)th filtering refers to the similarity between each motion parameter in the test parameter sequence and each motion parameter in the sensor parameters after the (i-1)th filtering.

[0078] For example, RMSE (Root Mean Square Error) is used to determine the similarity between the test parameter sequence and the sensor parameters after the (i-1)th filtering.

[0079]

[0080] In the formula, n represents the sequence length of the test parameter sequence or the sequence length of the sensor parameter sequence, or the number of motion parameters included in the test parameter sequence or the number of motion parameters included in the sensor parameter sequence. i This represents the i-th motion parameter in the test parameter sequence. i This represents the i-th motion parameter in the sensor parameter sequence.

[0081] Optionally, the motion parameters in the test parameter sequence are arranged in order of sampling time, that is, the motion parameters sampled earlier are placed earlier in the test parameter sequence; the motion parameters in the sensor parameter sequence are also arranged in order of sampling time.

[0082] Optionally, a larger RMSE between the test parameter sequence and the sensor parameter sequence after the (i-1)th filtering indicates a smaller similarity between them; conversely, a smaller RMSE indicates a greater similarity. In other words, the similarity is inversely proportional to the RMSE.

[0083] Step 222-2: If the similarity is greater than or equal to the similarity threshold, determine the sensor parameter sequence after the (i-1)th filtering as the calibrated sensor parameter sequence.

[0084] Optionally, if the similarity is greater than or equal to the similarity threshold, the sensor parameter sequence after the (i-1)th filtering is determined as the calibrated sensor parameter sequence; or, if the RMSE is less than or equal to the RMSE threshold, the sensor parameter sequence after the (i-1)th filtering is determined as the calibrated sensor parameter sequence.

[0085] Step 222-3: If the similarity is less than the similarity threshold, update the cutoff frequency of the first filter; input the sensor parameter sequence into the updated first filter to obtain the sensor parameter sequence after the i-th filtering.

[0086] Optionally, if the similarity is less than a similarity threshold, the cutoff frequency of the first filter is updated; the sensor parameter sequence is then input into the updated first filter to obtain the sensor parameter sequence after the i-th filtering. Alternatively, if the RMSE is greater than the RMSE threshold, the cutoff frequency of the first filter is updated; the sensor parameter sequence is then input into the updated first filter to obtain the sensor parameter sequence after the i-th filtering.

[0087] It should be noted that, for the case where the similarity equals the similarity threshold, step 222-2 can be executed as shown in the embodiments of this application. In some optional embodiments, step 222-3 can also be executed. That is, when the similarity equals the similarity threshold, the embodiments of this application do not limit the steps to be executed. Similarly, the embodiments of this application do not limit the steps to be executed when RMSR equals the RMSE threshold.

[0088] Step 222-4: Repeat at least two of the above three steps until the calibrated sensor parameter sequence is determined.

[0089] Optionally, steps 222-1 and 222-3 above are repeated until step 222-2 is performed to determine the calibrated sensor parameter sequence.

[0090] For example, the calibration process described above is illustrated using a sensor that includes an accelerometer, a gyroscope, and a magnetometer, with the sensor parameter sequence being the sensor acceleration sequence and the test parameter sequence being the test acceleration sequence.

[0091] Optionally, the test acceleration sequence includes z-axis acceleration data of test subjects of different heights, weights, and BMIs walking slowly in different motion scenarios (such as flat ground, gentle slopes, steep slopes, gentle slopes, and steep slopes). Specifically, the z-axis acceleration data are the z-axis components of acceleration in a world coordinate system. A world coordinate system such as the Northeast-Northeast World Coordinate System (NWWS) is a coordinate system with the observer's position as the origin, geographic east as the x-axis, geographic north as the y-axis, and the z-axis pointing towards the sky.

[0092] First, the sensor acceleration sequence obtained from the accelerometer is in a body-dependent coordinate system. This body-dependent coordinate system is a three-dimensional coordinate system established based on the exoskeleton device, and it moves and rotates with the exoskeleton device. For example, the body-dependent coordinate system has the exoskeleton device's center of mass as the origin, the forward direction of the exoskeleton device as the x-axis, and the rightward direction of the exoskeleton device as the y-axis. Since the sensor acceleration is in a body-dependent coordinate system, it should be transformed to the world coordinate system.

[0093] For example, the sensor acceleration is transformed from the body coordinate system to the world coordinate system based on the Direction Cosine Matrix (DCM). The components of the sensor acceleration along each coordinate axis in the body coordinate system are different from those in the world coordinate system. Since the test acceleration refers to the z-axis acceleration in the world coordinate system, the sensor acceleration and the test acceleration must be in the same coordinate system to correct the sensor acceleration.

[0094] Optionally, the DCM is calculated based on the accelerometers (accelerometer sensors), gyroscopes, and magnetometers (magnetic sensors) in the exoskeleton device. Specifically, the exoskeleton device's quadruple (used to indicate the exoskeleton device's attitude information) is calculated using the Explicit Complementary Filtering Method (MahonyAHRS).

[0095] Specifically, the MahonyAHRS algorithm is a commonly used attitude estimation algorithm that combines data from accelerometers, magnetometers, and gyroscopes to estimate the attitude (i.e., the quaternion) of an object relative to a reference coordinate system (i.e., the aforementioned N / A world coordinate system). The algorithm utilizes the accelerometer to sense the gravitational component to calculate the object's pitch and roll angles, the magnetometer to sense the direction of the Earth's magnetic field to calculate the magnetic north heading, and the gyroscope to measure angular velocity to calculate attitude changes. Through complementary filtering, the algorithm combines the dynamic response characteristics of the gyroscope with the stability of the accelerometer / magnetometer to improve measurement accuracy and the dynamic performance of the system. For example, the quaternion is first initialized, and then the initial attitude is calculated based on the measurements from the accelerometer and magnetometer. Next, the attitude changes are integrated using the gyroscope measurements, and the accumulated errors of the gyroscope are corrected using the accelerometer and magnetometer measurements.

[0096] Optionally, the DCM is constructed based on the quadruples, taking the Northeast-Sky world coordinate system as an example. The calculation formula for the DCM is shown below.

[0097]

[0098] If the sensor acceleration at a certain moment in the body coordinate system is expressed as Sensor acceleration in world coordinate system is represented as Then we have:

[0099]

[0100] The main motion scene identification involved in this application embodiment includes flat ground, gentle slope, steep slope, gentle slope, and steep slope. The key to identifying these motion scenes lies in the z-axis acceleration of the exoskeleton device, that is, the acceleration in the vertical direction. The greater the z-axis acceleration, the greater the probability that the motion scene is a steep slope; the smaller the z-axis acceleration (the larger the negative z-axis acceleration), the greater the probability that the motion scene is a steep slope.

[0101] Next, the similarity at time k between the test acceleration at time k in the test acceleration sequence and the sensor acceleration at time k in the sensor acceleration sequence is determined based on the RMSE. If the similarity at time k is greater than or equal to a preset threshold, the sensor acceleration at time k is determined to be the calibrated sensor acceleration. If the similarity at time k is less than the preset threshold, the sensor acceleration at time k is input into a Butterworth high-pass filter for filtering to obtain a new sensor acceleration. After obtaining the new sensor acceleration, the cutoff frequency of the Butterworth high-pass filter is adjusted. The RMSE between the test acceleration at time k and the new sensor acceleration at time k is calculated, and the similarity is judged again until the calibrated sensor acceleration is confirmed. The cutoff frequency of the Butterworth high-pass filter indicates the transition point from the passband (the frequency range that allows signals to pass) to the stopband (the frequency range that blocks signals from passing). Butterworth high-pass filters retain high-frequency information from the input while eliminating low-frequency information. For sensor acceleration, passing through a Butterworth high-pass filter can preserve the high-frequency changes in the sensor acceleration, that is, the acceleration during rapid motion.

[0102] For example, we can first calculate the RMSE at time k between the sensor acceleration sequence and the test acceleration sequence at time k and n-1 times prior to time k, or in other words, calculate the RMSE at time k between the n sensor velocity sequences from time kn to time k in the sensor acceleration sequence and the n test acceleration sequences from time kn to time k in the test acceleration sequence. Specifically, the formula for calculating RMSE is shown below.

[0103]

[0104] Among them, a i It is the sensor acceleration at time i. Let n be the test acceleration at time i, where i and n are both positive integers.

[0105] After calculating the RMSE at time k, the similarity at time k can be determined based on the RMSE at time k. For example, the similarity at time k is equal to 1 minus the RMSE at time k.

[0106] In summary, the method provided in this application illustrates a calibration method for sensor parameter sequences. Instead of manual calibration, the sensor parameters are calibrated based on a pre-tested parameter sequence. This allows the exoskeleton device to calibrate its sensor parameter sequence during operation. On one hand, this avoids data shifts in the exoskeleton device's sensors during wear due to vigorous movement of the wearer or extreme environments, which could reduce the reliability of the tested sensor parameter sequence. This indirectly improves the reliability of the exoskeleton device, as well as the reliability and stability of motion scene recognition based on the calibrated sensor parameter sequence. On the other hand, eliminating the need for pre-wearing calibration reduces the cost of using the exoskeleton device.

[0107] Furthermore, it is shown how to use a first filter to calibrate a sensor parameter sequence based on a test parameter sequence. The first filter is used because noise errors introduced by the sensor's device are usually traceable. Using the first filter to mimic the manual calibration process removes errors in the sensor parameters, improving the reliability of the sensor parameter sequence and thus enhancing the reliability of the exoskeleton device.

[0108] Scene recognition process.

[0109] In some embodiments, the exoskeleton device includes an accelerometer and a barometric pressure sensor, and the sensor parameter sequence is a sensor acceleration sequence. That is, the sensor parameter sequence that needs to be calibrated is the sensor acceleration sequence.

[0110] Based on Figure 2 In an alternative embodiment, such as Figure 4 As shown, step 230 can be implemented as steps 231 to 233.

[0111] Step 231: Obtain the sensor altitude sequence, which is obtained from the barometric pressure sensor.

[0112] Optionally, a barometric pressure sensor is used to acquire sensor height sequences. Specifically, the barometric pressure sensor measures atmospheric pressure and calculates the current height of the exoskeleton device based on changes in atmospheric pressure.

[0113] Optionally, the sensor height collected by the barometric pressure sensor is either absolute altitude or relative altitude.

[0114] Step 232: Based on the calibrated sensor acceleration sequence and sensor height sequence, a set of prediction parameter sequences is obtained. The set of prediction parameter sequences includes at least one of the prediction height sequence, prediction velocity sequence, and prediction acceleration sequence.

[0115] Optionally, based on the calibrated sensor acceleration and sensor altitude sequences at the current moment, a set of predicted parameter sequences for the next moment is predicted. The set of predicted parameter sequences includes at least one of a predicted altitude sequence, a predicted velocity sequence, and a predicted acceleration sequence.

[0116] Optionally, the implementation of predicting a set of predicted parameter sequences based on the calibrated sensor acceleration and height sequences requires that the state transitions between the various predicted parameter sequences in the set are known. For example, the height, velocity, and acceleration shown in the embodiments of this application can be interconverted. For instance, differentiating the expression for height over time yields the expression for velocity; differentiating the expression for velocity over time (i.e., taking the derivative) yields the expression for acceleration; differentiating the expression for height over time yields the expression for acceleration; integrating the expression for acceleration over time yields the expression for velocity; integrating the expression for acceleration over time yields the expression for height; and integrating the expression for velocity over time yields the expression for height.

[0117] Optionally, based on the state transition relationship between each prediction parameter sequence in the prediction parameter sequence set, the prediction parameters in the prediction parameter sequence at the next time step can be predicted; or, based on the prediction parameters in the prediction parameter sequence at the previous time step, the prediction parameters in the prediction parameter sequence at the current time step can be predicted.

[0118] Optionally, step 232 above can be implemented as follows: calculating the first variance of the calibrated sensor acceleration sequence; and calculating the second variance of the sensor height sequence; based on the first and second variances, determining the measurement noise variance matrix and the process noise variance matrix, wherein the measurement noise variance matrix is ​​used to simulate the measurement noise caused by sensor error in the motion parameters in the predicted parameter sequence set, and the process noise variance matrix is ​​used to indicate the noise generated during prediction; and based on the measurement noise variance matrix, the process noise variance matrix, and the predicted parameter vector at time k, using the second filter to predict the predicted parameter vector at time k+1.

[0119] Optionally, the prediction parameter vector includes at least one prediction parameter to be predicted, such as at least one of predicted height, predicted velocity, and predicted acceleration. The prediction parameter vector at time k can be understood as the prediction parameter at time k in each prediction parameter sequence of the prediction parameter sequence set. For example, the prediction parameter sequence set includes a prediction height sequence, a prediction velocity sequence, and a prediction acceleration sequence; then the prediction height sequence includes the predicted height at each time, the prediction velocity sequence includes the predicted velocity at each time, and the prediction acceleration sequence includes the predicted acceleration at each time. The time periods corresponding to the prediction height sequence, prediction velocity sequence, and prediction acceleration sequence are consistent. For example, if the prediction height sequence includes the predicted height from time 0 to time n, then the prediction velocity sequence also includes the predicted velocity from time 0 to time n, and the prediction acceleration sequence also includes the predicted acceleration from time 0 to time n. Or, in other words, the time periods associated with each prediction parameter sequence in the prediction parameter sequence set are consistent.

[0120] Optionally, the prediction parameter vector at time k includes the prediction parameters at time k in each prediction parameter sequence. The prediction parameter vector at time 0 is the initial parameter vector, or in other words, the predicted height at time 0 is the initial height, the predicted velocity at time 0 is the initial velocity, and the predicted acceleration at time 0 is the initial acceleration. The initial parameter vector is set based on expert experience, or based on prior knowledge, or based on sampled random noise. That is, the initial height, initial velocity, and initial acceleration mentioned above are set based on expert experience, or based on prior knowledge, or based on sampled random noise.

[0121] Optionally, when predicting the prediction parameter vector at time k+1 based on the prediction parameter vector at time k, the calibrated sensor acceleration sequence includes the sensor acceleration acquired and calibrated at time k+1, and the sensor height sequence includes the sensor height acquired at time k+1. This can also be understood as the latest sensor acceleration in the calibrated sensor acceleration sequence being the sensor acceleration at time k+1, and the latest sensor acceleration in the sensor height sequence being the sensor height at time k+1; that is, the sensor acceleration sequence does not include the sensor acceleration at time k+2, and the sensor height sequence does not include the sensor height at time k+2.

[0122] Optionally, based on the measurement noise variance matrix, the process noise variance matrix, and the predicted parameter vector at time k, a second filter is used to predict the predicted parameter vector at time k+1. This includes: based on the predicted parameter vector at time k, using the second filter to predict the first predicted parameter vector at time k+1; and based on the measurement noise variance matrix and the process noise variance matrix, using the second filter to update the first predicted parameter vector at time k+1, thus obtaining the second predicted parameter vector at time k+1. That is, it includes two processes: a prediction process and an update process. Optionally, both of the above processes are implemented based on the second filter. Optionally, the first predicted parameter vector is generated during the prediction process, and the second predicted parameter vector is generated during the update process. The first predicted parameter vector is determined based on the predicted parameter vector at the previous time, while the second predicted parameter vector is determined based on the first predicted parameter vector at the current time.

[0123] Optionally, the prediction parameter vector at time k in the above prediction process can be either the first prediction parameter vector at time k or the second prediction parameter vector at time k. This application's embodiment uses the example of the second prediction parameter vector at time k as the example for illustration, but it is not intended to limit the scope of the application.

[0124] Optionally, in the update process, in addition to using the measurement noise variance matrix and the process noise variance matrix, the calibrated sensor acceleration and the sensor height at the current time can also be used. That is, the above-mentioned update of the first predicted parameter vector at time k+1 based on the measurement noise variance matrix and the process noise variance matrix to obtain the second predicted parameter vector at time k+1 can be implemented as follows: based on the measurement noise variance matrix, the process noise variance matrix, the calibrated sensor acceleration at time k+1, and the sensor height at time k+1, update the first predicted parameter vector at time k+1 to obtain the second predicted parameter vector at time k+1. Here, the sensor acceleration and sensor height at time k+1 can also be collectively referred to as the observation value at time k.

[0125] Optionally, the second filter is used to obtain an optimal estimate (or state variable, state variable, predicted value, etc., such as each predicted parameter in the prediction parameter sequence set) by removing noise based on dynamic observation information (or measurement information, observation value, measurement value, etc., such as the calibrated sensor parameter sequence and sensor height sequence). That is, the second filter supports the analysis of state transition relationships to perform state prediction.

[0126] For example, the second filter is a Kalman filter. In the case where the set of prediction parameter sequences includes a prediction height sequence, a prediction velocity sequence, and a prediction acceleration sequence, the second filter is a third-order Kalman filter.

[0127] For example, the above-described method of predicting the predicted parameter vector at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted parameter vector at time k includes: when the set of predicted parameter sequences includes a predicted height sequence, predicting the predicted height at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted height at time k. Further, predicting the predicted height at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted height at time k includes: predicting the first predicted height at time k+1 using a second filter based on the predicted height at time k; and updating the first predicted height at time k+1 using a second filter based on the measurement noise variance matrix and the process noise variance matrix to obtain the second predicted height at time k+1.

[0128] For example, the above-described method of predicting the predicted parameter vector at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted parameter vector at time k includes: when the set of predicted parameter sequences includes a predicted velocity sequence, predicting the predicted velocity at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted velocity at time k. Further, predicting the predicted velocity at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted velocity at time k includes: predicting the first predicted velocity at time k+1 using a second filter based on the predicted velocity at time k; and updating the first predicted velocity at time k+1 using a second filter based on the measurement noise variance matrix and the process noise variance matrix to obtain the second predicted velocity at time k+1.

[0129] For example, the above-described method of predicting the predicted parameter vector at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted parameter vector at time k includes: when the set of predicted parameter sequences includes a predicted acceleration sequence, predicting the predicted acceleration at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted acceleration at time k. Further, predicting the predicted acceleration at time k+1 using a second filter based on the measurement noise variance matrix, the process noise variance matrix, and the predicted acceleration at time k includes: predicting the first predicted acceleration at time k+1 using a second filter based on the predicted height at time k; and updating the first predicted acceleration at time k+1 using a second filter based on the measurement noise variance matrix and the process noise variance matrix to obtain the second predicted acceleration at time k+1.

[0130] Specifically, the filtering process of the second filter can be referred to in the following "Example of the filtering process of the second filter", which will not be repeated here.

[0131] Step 233: Determine the motion scene of the exoskeleton device based on the sensor height sequence and the set of predicted parameter sequences.

[0132] Optionally, after determining the motion scenario of the exoskeleton device, the exoskeleton device switches to different assistance modes. For example, if the motion scenario of the exoskeleton device is an uphill steep slope, it switches to the uphill steep slope assistance mode; if the motion scenario of the exoskeleton device is an uphill gentle slope, it switches to the uphill gentle slope assistance mode; if the motion scenario of the exoskeleton device is a downhill steep slope, it switches to the downhill gentle slope assistance mode; and if the motion scenario of the exoskeleton device is on flat ground, it switches to the flat ground assistance mode.

[0133] Optionally, step 233 above can be implemented as follows: calculating the height change rate based on the sensor height sequence and the predicted parameter sequence set; determining that the motion scenario of the exoskeleton device is an uphill steep slope when the height change rate is greater than or equal to a first height change rate threshold; determining that the motion scenario of the exoskeleton device is a downhill steep slope when the height change rate is less than or equal to a second height change rate threshold; obtaining the step frequency and stride length when the height change rate is greater than the second height change rate and the height change rate is less than the first height change rate; and determining the motion scenario of the exoskeleton device based on the step frequency and the stride length.

[0134] Optionally, the first height change rate threshold and / or the second height change rate threshold are set based on expert experience, or based on prior knowledge. The first height change rate threshold is greater than the second height change rate threshold.

[0135] For details on the calculation of the altitude change rate, please refer to "Calculation of Altitude Change Rate" below.

[0136] Optionally, determining the movement scenario of the exoskeleton device based on cadence and stride information includes: determining the movement scenario of the exoskeleton device as an uphill gentle slope when the cadence increases and the change in cadence is greater than or equal to a first change threshold, and the stride increases and the change in stride is greater than or equal to a second change threshold; determining the movement scenario of the exoskeleton device as a downhill gentle slope when the cadence decreases and the change in cadence is greater than or equal to the first change threshold, and the stride decreases and the change in stride is greater than or equal to the second change threshold; and determining the movement scenario of the exoskeleton device as flat ground when the change in cadence is less than the first change threshold, and / or the change in stride is less than the second change threshold.

[0137] For example, when stride frequency and stride length gradually increase, it is identified as an uphill gentle slope exercise scenario. When stride frequency and stride length gradually decrease, it is identified as a downhill gentle slope exercise scenario. When stride frequency and stride length do not change much, it is identified as a flat ground exercise scenario.

[0138] Specifically, the determination of cadence and stride length for exoskeleton devices is detailed in the section "Determination of Cadence and / or Stride Length" below.

[0139] In summary, the method provided in this application illustrates the specific process of determining the motion scene of an exoskeleton device. By combining the sensor height sequence measured by a barometric pressure sensor with a calibrated sensor acceleration sequence, a set of predicted parameter sequences related to the motion state of the exoskeleton device is predicted. Scene recognition is then performed based on the sensor height sequence and the set of predicted parameter sequences. This approach reduces disturbances to the barometric pressure sensor caused by air pressure. Furthermore, combining the sensor height sequence and the calibrated sensor acceleration sequence for scene recognition, compared to directly using either sequence, considers more relevant factors and thus better reflects the motion scene of the exoskeleton device, improving the accuracy of motion scene recognition. Moreover, this method is more sensitive to dynamic changes in the motion scene of the exoskeleton device, resulting in better real-time motion scene recognition and enabling more timely motion feedback based on the wearer, thereby improving the user experience.

[0140] • Example of the filtering process of the second filter.

[0141] Specifically, taking a Kalman filter as the second filter and a prediction parameter sequence set including the predicted height sequence, predicted velocity sequence, and predicted acceleration sequence as an example, the entire prediction and update process based on the second filter will be explained. Figure 5 As shown.

[0142] For example, the Kalman filter is set to operate at a frequency of approximately 500 Hz and a time interval of approximately 2 ms.

[0143] Step (1) Initialize the variables. This involves initializing the relevant parameters used in the second filter. The variables to be initialized include, but are not limited to: state variable X (i.e., the prediction parameter vector mentioned above), state transition matrix F, measurement noise variance matrix R, covariance matrix P, process noise variance matrix Q, and observation matrix H.

[0144] Here, the state variable X is the prediction parameter vector. During initialization, the initial state variable can be denoted as X0, which represents the state variable at time 0.

[0145]

[0146] In the formula, H fusion V represents the initial height, i.e., the height at time 0; fusion A represents the initial velocity, i.e., the velocity at time 0; fusion This represents the initial acceleration, that is, the acceleration at time 0.

[0147] The state transition matrix F is used to describe the changes of each state variable over time.

[0148]

[0149] In the formula, dt represents a time step. The first row of the state transition matrix represents the contribution of each state variable to the altitude state during the prediction process. First, the first element 1 indicates that the altitude state remains unchanged in the next time step; the second element dt indicates that the change in altitude is approximately equal to the velocity multiplied by time; the third element 0.5 × dt × dt indicates that the change in position also includes the acceleration multiplied by the square of time, which is obtained from the integral of acceleration. The second row represents the contribution of each state variable to the velocity state during the prediction process. First, the first element 0 indicates that the altitude state has no direct effect on the velocity state; the second element 1 indicates that the velocity state remains unchanged in the next time step; the third element dt indicates that the change in velocity is approximately equal to the acceleration multiplied by time. The third row represents the contribution of each state variable to the acceleration state during the prediction process. The first and second elements 0 indicate that neither the altitude state nor the velocity state has a direct effect on the acceleration state; the third element 1 indicates that the acceleration state remains unchanged in the next time step.

[0150] The measurement noise variance R matrix is ​​used to represent the random error in the observation process. In this application, this random error is based on the variance of the observed quantities, namely the first variance Q1 (variance of the calibrated sensor acceleration sequence) and the second variance Q2 (variance of the sensor height sequence), and the wind influence factor k. qr Confirmed. For details on determining the wind force influencing factors, please refer to the section "Determination of Wind Force Influencing Factors" below.

[0151]

[0152] The covariance matrix P is used to indicate the uncertainty in the prediction process. During initialization, the initial covariance matrix can be denoted as P0, which is the covariance matrix at time 0.

[0153]

[0154] In the formula, p1 represents the initial covariance of height; p2 represents the initial covariance of velocity; and p3 represents the initial covariance of acceleration. The initial covariance of height and / or velocity and / or acceleration is set based on expert experience, prior knowledge, or by sampling random noise.

[0155] The process noise variance matrix Q is used to indicate uncaptured random dynamics. It is typically assumed to be zero-mean Gaussian white noise.

[0156] The stochastic dynamics in this application are based on the wind influence factor k. qr Sure.

[0157]

[0158] The observation matrix H is used to map state variables to the observation space, describing the relationship between state variables and observations.

[0159]

[0160] Since the observed measurement in this embodiment is the calibrated sensor acceleration (acc) H The observation matrix is ​​used to obtain the state quantity, which is the sensor height, without any conversion. However, for some scenarios, such as when the observation includes atmospheric pressure and the state quantity includes predicted height, the observation matrix is ​​used to convert the atmospheric pressure in the observation into the predicted height in the state quantity.

[0161] Step (2) State prediction and covariance prediction. That is, the state quantity at the current time is predicted based on the state quantity at the previous time step; and the covariance matrix at the current time step is predicted based on the covariance matrix at the previous time step.

[0162] Optionally, the state at the current time step is predicted based on the state variables and state transition matrix from the previous time step. The specific calculation is shown in the following equation.

[0163] K′ k+1 =FX k

[0164] In the formula, X represents the state quantity at the previous time step, i.e., the state quantity at time step k; X′ represents the predicted state quantity at the current time step, i.e., the predicted state quantity at time step k+1.

[0165] Optionally, the covariance matrix at the current time step is predicted based on the covariance matrix, state transition matrix, and process noise variance matrix of the previous time step. The specific formula is shown below.

[0166] P′ k+1 =F·P k ·F T +Q

[0167] In the formula, P′ represents the covariance matrix of the current time step obtained from the prediction, i.e., the covariance matrix of the (k+1)th time step obtained from the prediction; P represents the covariance matrix of the previous time step, i.e., the covariance matrix of the kth time step. The covariance matrix of the current time step obtained from the prediction is used to describe the uncertainty introduced into the predicted state quantity due to the influence of process noise indicated by the process noise variance matrix and the influence of the state transition process supported by the state transition matrix. This process reflects how the uncertainty of the state evolves over time and takes into account the random noise introduced during the prediction process.

[0168] For example, if the current time is time (k+1), then the previous time is time (k). In this case, the above-mentioned prediction of the current time's state quantity based on the state quantity of the previous time, and prediction of the current time's covariance matrix based on the covariance matrix of the previous time, can be described as: predicting the state quantity (first prediction parameter vector) of time (k+1) based on the state quantity (first prediction parameter vector) of time (k); and predicting the covariance matrix of time (k+1) based on the covariance matrix of time (k). Further, it can be described as predicting the state quantity (first prediction parameter vector) of time (k+1) based on the state quantity (first prediction parameter vector) of time (k) and the state transition matrix; and predicting the covariance matrix of time (k+1) based on the covariance matrix of time (k), the state transition matrix, and the process noise variance matrix.

[0169] It should be noted that since the state variables and covariance matrix here are predicted, the predicted state variables can also be called state prediction values, and the predicted covariance matrix can be called the predicted covariance matrix.

[0170] Step (3) Obtain real-time observations. Real-time observations are the observations at the current moment, or the calibrated sensor acceleration and sensor height at the current moment.

[0171] Specifically, real-time observations, such as the observation Z at time k+1. k+1 As shown below.

[0172]

[0173] Among them, Z k+1 It can also be called the observation at the current moment; Height represents the sensor height at the current moment, or the sensor height at the (k+1)th moment, which is the data (sensor height) at the (k+1)th moment in the sensor height sequence; acc H This represents the calibrated sensor acceleration at the current moment, or the sensor acceleration at the (k+1)th moment, which is the data at the (k+1)th moment in the calibrated sensor acceleration sequence (calibrated sensor acceleration). It should be noted that since this embodiment mainly focuses on the change in height of the exoskeleton device, the sensor acceleration here is the sensor acceleration in the height direction (that is, the z-axis direction of the world coordinate system). However, for other scenarios, sensor acceleration in the x-axis direction of the world coordinate system, sensor acceleration in the y-axis direction of the world coordinate system, sensor acceleration vector, etc., can also be used. This embodiment does not limit this.

[0174] Step (4) Update. Based on the state variables and covariance matrix predicted in step (2) above and the observations obtained in step (3) above, the predicted state variables and covariance matrix are updated. The updated state variables and covariance matrix will be used as the state variables and covariance of the previous time step in the next prediction process.

[0175] Specifically, the Kalman gain K is first calculated. Kalman gain K is used to dynamically adjust the state estimates during the update process to fuse the predicted state variables and observations. In other words, Kalman gain K is used to adjust the weights of the predicted state values ​​and observations during the update process. The specific calculation process for Kalman gain K is shown below.

[0176] K k+1 =P′ k+1 ·H T ·(H·P′ k+1 ·H T +R) -1

[0177] In the formula, K k+1 P′ represents the Kalman gain at time k+1. k+1Let represent the covariance matrix obtained from the prediction at time k.

[0178] After calculating the Kalman gain K, the state prediction and the predicted covariance matrix can be updated based on the Kalman gain K, resulting in updated state variables and updated covariance matrices. The update process is essentially an estimation based on actual measured observations and predicted values ​​obtained using the Kalman filter to obtain a more accurate state estimate. Therefore, the updated state variables can also be called state estimates, and the updated covariance matrix can be called estimated covariance matrices.

[0179] Specifically, the predicted state variable X is updated based on the Kalman gain K. v The calculation process for obtaining the state estimate is shown in the following formula.

[0180] X k+1 =X′ k+1 +K k+1 ·(Z k+1 -H·X′ k+1 )

[0181] In the formula, X k+1 This represents the state estimate at time k+1, or the second prediction parameter vector at time k+1, or the updated state value at the current time, etc. k+1 This represents the predicted state value at time k+1, which is also the first predicted parameter vector at time k+1, or the predicted state quantity at the current time, etc.

[0182] Specifically, the calculation process for obtaining the estimated covariance matrix by updating the predicted covariance matrix P′ based on the Kalman gain K is shown below.

[0183] P k+1 =(IK k+1 ·H)·P′ k+1

[0184] In the formula, { k+1 Let P' represent the estimated covariance matrix at time k+1, or the updated covariance matrix at the current time, etc. k+1 This represents the predicted covariance matrix at time k+1, or the predicted covariance matrix at the current time, and so on.

[0185] Optionally, the state estimate at time (k+1) calculated in step (4) will be used as the state quantity at the previous time when the state prediction at time (k+2) is predicted; the covariance matrix at time (k+1) calculated in step (4) will be used as the covariance matrix at the previous time when the prediction covariance matrix at time (k+2) is predicted. In other words, in the prediction process at time (k+2), the state quantity at time (k+1) and the covariance matrix at time (k+1) are calculated in the update process at time (k+1). Similarly, in the prediction process at time (k+1), the state quantity at time (k) and the covariance matrix at time (k) are calculated in the update process at time (k), and so on. The state quantity at time (0) and the covariance matrix at time (0) are set in the initialization process of the change quantity in step (1).

[0186] Step (5) output.

[0187] Optionally, the state estimate at the current moment can be output directly after calculation; or, the state estimate at the current moment can be output once at a preset time interval.

[0188] • Determination of wind force influencing factors.

[0189] Optionally, the measurement noise variance matrix and the process noise variance matrix are determined based on the first variance and the second variance, including: determining the acceleration sequence for the barometric pressure sensor based on the sensor height sequence; calculating the third variance based on the acceleration sequence for the barometric pressure sensor; determining the wind force influence factor due to the influence of wind on the barometric pressure sensor based on the first variance and the third variance; and determining the measurement noise variance matrix and the process noise variance matrix based on the first variance, the second variance, and the wind force influence factor.

[0190] Optionally, the measurement noise variance matrix is ​​determined based on the first variance, the second variance, and the wind influence factor; and / or, the process noise variance matrix is ​​determined based on the wind influence factor.

[0191] For example, the measurement noise variance matrix is ​​the product of the wind force influence factor and the first matrix. The first matrix is ​​a second-order square matrix, with the first and second variances as the matrix elements on the main diagonal, and all other matrix elements being 0. The process noise variance matrix is ​​the product of the wind force influence factor and the second matrix, with the second matrix being a third-order square matrix. The matrix elements on the main diagonal of the third-order square matrix are non-zero constants, typically any value from 1 to 10, and all other matrix elements being 0. Optionally, the size of the measurement noise variance matrix is ​​related to the number of observations. In this embodiment, the observations are the calibrated sensor acceleration and sensor height, so the example measurement noise variance matrix is ​​a second-order square matrix. If there is one observation, the measurement noise variance matrix is ​​a first-order square matrix; if there are three observations, the measurement noise variance matrix is ​​a third-order square matrix. The size of the process noise variance matrix is ​​related to the number of state variables. In this embodiment, the state variables are predicted height, predicted velocity and predicted acceleration. Therefore, the process noise variance matrix of the example is a third-order square matrix. If there is only one state variable, the process noise variance matrix is ​​a first-order square matrix. If there are two state variables, the process noise variance matrix is ​​a second-order square matrix.

[0192] Optionally, the first variance is the variance of the sensor acceleration sequence measured by the accelerometer; the third variance is the variance of the sensor acceleration sequence calculated by the barometric pressure sensor. Because the accelerometer and barometric pressure sensor operate on different principles, the influence of external environmental factors on the accelerometer is significantly different from that on the barometric pressure sensor. Generally, the accelerometer is less affected by external environmental factors, while the barometric pressure sensor is more susceptible. Therefore, the influence of wind force can be inferred based on the first variance of the sensor acceleration sequence measured by the accelerometer and the third variance of the sensor acceleration sequence calculated by the barometric pressure sensor. In other words, the wind force influence factor can be inferred based on the first variance and the third variance, as detailed below.

[0193] Optionally, based on the first variance and the third variance, the wind force influence factor caused by the wind's influence on the barometric pressure sensor is determined, including: determining the variance deviation caused by the wind force based on the first variance and the third variance; if the variance deviation is greater than the maximum airflow threshold, determining the variance deviation as the maximum airflow threshold; if the variance deviation is greater than the walking deviation threshold, determining the wind force influence factor as a first ratio, where the first ratio is the ratio of the first difference to the maximum airflow threshold, and the first difference is the difference between the variance deviation and the maximum airflow threshold; if the variance deviation is less than or equal to the walking deviation threshold, determining the wind force influence factor as a first constant.

[0194] Optionally, the variance deviation caused by wind force refers to the deviation between the third variance of the sensor acceleration calculated by the barometric pressure sensor and the first variance of the sensor acceleration measured by the accelerometer due to the influence of wind force on the barometric pressure sensor.

[0195] Optionally, the maximum airflow threshold is set based on expert experience; or, the maximum airflow threshold is set based on prior knowledge. For example, the maximum airflow threshold is set based on prior knowledge, such as the maximum wind chime threshold, which is set in advance by testing the variance change of the sensor acceleration sequence calculated for the barometric pressure sensor under different wind conditions in a manner similar to the test parameter sequence described above.

[0196] Optionally, the walking deviation threshold is set based on expert experience; or, the walking deviation threshold is set based on prior verified knowledge.

[0197] Optionally, the maximum wind threshold is used to indicate the magnitude of the variance skew caused by wind, and the walking deviation threshold is used to indicate the magnitude of the variance skew caused by walking.

[0198] Optionally, based on the first variance and the third variance, the variance deviation caused by wind influence is determined, including: if the first variance is outside the effective range, the variance deviation is determined to be the third variance; if the first variance is within the effective range and the third variance is greater than the first variance, the variance deviation is determined to be the absolute value of the second difference, where the second difference is the difference between the first variance and the third difference; if the first variance is within the effective range and the third difference is less than or equal to the first variance, the variance deviation is determined to be the absolute value of the third difference, where the third difference is the difference between the third difference and t times the first variance, where t is a positive integer.

[0199] Optionally, the effective range is used to indicate whether the acceleration sensor, or the calibrated sensor acceleration sequence, is reliable. If the first variance is within the effective range, the acceleration sensor, or the calibrated sensor acceleration sequence, is considered reliable; if the first variance is outside the effective range, the acceleration sensor, or the calibrated sensor acceleration sequence, is considered unreliable. This is because acceleration sensors can also have unreliable acceleration sequences due to their own device limitations or external environmental factors. Therefore, the variance bias is determined based on the first variance only when the acceleration sensor, or the calibrated sensor acceleration sequence, is reliable.

[0200] Optionally, if the third variance is greater than the first variance, it can be determined that the reliability of the sensor acceleration measured by the barometric pressure sensor is lower than that measured by the accelerometer sensor, or in other words, the accelerometer sensor has higher reliability. Therefore, the variance bias is determined as the second variance bias. If the third variance is less than or equal to the first variance, it is determined that the reliability of the accelerometer sensor is not high enough. Therefore, the variance bias is determined as the third variance bias. Here, t is usually a decimal, which means that the reliability of the accelerometer sensor is appropriately adjusted. Optionally, t is set based on expert experience or prior knowledge.

[0201] For example, the method for determining wind influence factors is as follows: Figure 6 As shown.

[0202] Step (1) Calculate the variance deviation caused by the wind force.

[0203] Optionally, the variance skewness caused by wind force can be calculated based on the first variance and the third variance.

[0204] For example, when the first variance Q1 is within the valid range and the third variance Q3 > the first variance Q1, the variance bias is determined to be |Q3-Q1|; when Q1 is within the valid range and Q3 ≤ Q1, the variance bias is determined to be |Q3-tQ1|; when Q1 is outside the valid range, i.e., Q1 is not within the valid range, the variance bias is determined to be Q3.

[0205] Step (2) Determine whether the variance deviation is greater than the maximum air volume threshold.

[0206] Optionally, after calculating the variance bias, the relationship between the variance bias and the maximum wind force threshold is first determined.

[0207] For example, if the variance deviation is greater than the maximum wind force threshold, the variance deviation is determined to be equal to the maximum wind force threshold; if the variance deviation is less than or equal to the maximum wind force threshold, the following step (3) is performed to determine the relationship between the variance deviation and the walking deviation threshold.

[0208] Step (3) Determine whether the variance bias is greater than the walking deviation threshold.

[0209] For example, when the variance skewness is greater than the walking deviation threshold, the wind force influence factor is determined. Determine the wind force influence factor k when the variance skewness is less than or equal to the walking deviation threshold. qr = The first constant, such as the first constant being 1.

[0210] • Calculation of height change rate.

[0211] Optionally, the set of predicted parameter sequences includes at least one of a predicted height sequence and a predicted acceleration sequence; based on the sensor height sequence and the set of predicted parameter sequences, the height change rate is calculated, including at least one of the following: calculating a first height change rate based on the predicted height sequence; calculating a second height change rate based on the predicted acceleration sequence; calculating a third height change rate based on the sensor height sequence; performing a smoothing operation on the third height change rate to obtain a fourth height change rate; and fusing at least two of the first height change rate, the second height change rate, the third height change rate, and the fourth height change rate based on the wind influence factor to obtain a fused height change rate.

[0212] Optionally, the first height change rate is obtained by differentiating the predicted height; the third height change rate is obtained by differentiating the sensor height; and the fourth height change rate is obtained by smoothing and filtering the third height change rate. Based on the first, second, third, and fourth height change rates, the fused height change rate is obtained.

[0213] For example, HR = p1HR1 + p2HR2 + p3HR3 + p4HR4.

[0214] In the above formula, p1, p2, p3, and p4 are four variables, affected by the wind force factor k. pr Modulation. p1=P1+k pr R P1 p2 = P2 + k pr R P2 p3 = P3 + k pr P P3 p4 = P4 + k pr R P4 P1, R P1 P2, R P2 P3, R P3 P4, R P4 These are all constants of empirical parameters. The fused altitude change rate HR characterizes the wearer's altitude change rate (positive or negative) along the z-axis in the world coordinate system. HR represents the fused altitude change rate; HR1 represents the first altitude change rate; HR2 represents the second altitude change rate; HR3 represents the third altitude change rate; and HR4 represents the fourth altitude change rate.

[0215] • Determination of stride length and / or stride frequency.

[0216] For example, cadence and stride length information are calculated based on hip joint angle values.

[0217] For example, such as Figure 7As shown, the hip joint angle value is obtained through the hip joint motor encoder (calculating the difference between the encoders of the two hip joint motors), and then converted into step frequency W and stride amplitude A through the structure of AFO (Adaptive Frequency Oscillator). AFO first defines an adaptive oscillation model x(t) = f(t), where f(t) includes the amplitude A, frequency ω, and phase of the adaptive oscillation model. Parameters such as AFO are adaptively adjusted based on feedback from the hip joint angle signal (generally by minimizing the error between AFO and the actual joint angle), and the frequency W and amplitude A are updated. The gait cycle T is determined by analyzing the peak values ​​fmax and fmin of the hip joint angle rise and fall, as well as the time of the peak rise (t.index(fmax)) and the time of the peak fall (t.index(fmin)). Finally, the amplitude A and gait frequency W for each gait cycle are determined using the gait cycle T. Here, t represents real-time (i.e., moment).

[0218] For example,

[0219] Specifically, the error(t) between AFO and the actual joint angle p(t) is shown below.

[0220] error(t) = f(t) - p(t)

[0221] The gait period T is calculated as follows.

[0222] T=2×(t.index(fmax)-t.index(fmin))

[0223] Figure 8 A flowchart illustrating an exemplary embodiment of the motion scene recognition method provided in this application is shown. The method includes:

[0224] Step 1: Obtain sensor acceleration and test acceleration; calibrate sensor acceleration based on test acceleration to obtain calibrated sensor acceleration.

[0225] The sensor acceleration is obtained from the acceleration sensor in the exoskeleton device. The acceleration test involves pre-measuring the acceleration data of different test subjects walking slowly in different motion scenarios.

[0226] Optionally, the acceleration test involves measuring the z-axis acceleration data of test subjects of different heights, weights, and BMIs as they walk slowly in different motion scenarios (such as flat ground, ascending a gentle slope, ascending a steep slope, descending a gentle slope, and descending a steep slope). Specifically, the z-axis acceleration data represents the z-axis component of acceleration in a world coordinate system. A world coordinate system such as the Northeast-Northeast World Coordinate System (NWWS) is a coordinate system with the observer's position as the origin, geographic east as the x-axis, geographic north as the y-axis, and the z-axis pointing towards the sky.

[0227] Optionally, the sensor acceleration obtained directly from the accelerometer of the exoskeleton device is usually in a body coordinate system. This body coordinate system is a three-dimensional coordinate system established based on the exoskeleton device, and it moves and rotates with the exoskeleton device. For example, the body coordinate system has the exoskeleton device's center of mass as the origin, the forward direction of the exoskeleton device as the x-axis, and the rightward direction of the exoskeleton device as the y-axis. When the sensor acceleration is in a body coordinate system, it should be transformed to the world coordinate system.

[0228] For example, the sensor acceleration is transformed from the body coordinate system to the world coordinate system based on the Direction Cosine Matrix (DCM). The components of the sensor acceleration along each coordinate axis in the body coordinate system are different from those in the world coordinate system. Since the test acceleration refers to the z-axis acceleration in the world coordinate system, the sensor acceleration and the test acceleration must be in the same coordinate system to correct the sensor acceleration.

[0229] Optionally, the DCM is calculated based on the accelerometers (accelerometer sensors), gyroscopes, and magnetometers (magnetic sensors) in the exoskeleton device. Specifically, the exoskeleton device's quadruple (used to indicate the exoskeleton device's attitude information) is calculated using the Explicit Complementary Filtering Method (MahonyAHRS). DCM is constructed based on quaternions. Taking the Northeast-Eastern Sky world coordinate system as an example... If the sensor acceleration in the body coordinate system is expressed as Sensor acceleration in world coordinate system is represented as Then we have:

[0230]

[0231] The main motion scene identification involved in this application embodiment includes flat ground, gentle slope, steep slope, gentle slope, and steep slope. The key to identifying these motion scenes lies in the z-axis acceleration of the exoskeleton device, that is, the acceleration in the vertical direction. The greater the z-axis acceleration, the greater the probability that the motion scene is a steep slope; the smaller the z-axis acceleration (the larger the negative z-axis acceleration), the greater the probability that the motion scene is a steep slope.

[0232] Optionally, the sensor acceleration is calibrated based on the test acceleration to obtain the calibrated sensor acceleration. First, the RMSE (Root Mean Square Error) between the test acceleration and the sensor acceleration is calculated. Specifically, the formula for calculating the root mean square error is shown below.

[0233]

[0234] Among them, a i It is the acceleration of the i-th sensor. It is the acceleration of the i-th test, where i and n are both positive integers.

[0235] Optionally, the similarity between the test acceleration and the sensor acceleration is determined based on RMSE. If the similarity is greater than or equal to a preset threshold, the current sensor acceleration is determined to be the calibrated sensor acceleration; if the similarity is less than the preset threshold, the sensor acceleration is input to a filter for filtering to obtain a new sensor acceleration. After obtaining the new sensor acceleration, the cutoff frequency of the filter is adjusted. The RMSE between the test acceleration and the new sensor acceleration is calculated, and the similarity is judged again until the calibrated sensor acceleration is confirmed. The cutoff frequency of the filter indicates the transition point from the passband (the frequency range that allows signals to pass) to the stopband (the frequency range that blocks signals from passing). Filters such as the Butterworth high-pass filter retain high-frequency information while eliminating low-frequency information. For sensor acceleration, passing through the Butterworth high-pass filter can retain the high-frequency changing acceleration in the sensor acceleration, i.e., the acceleration during rapid motion.

[0236] Step 2: Obtain sensor height; based on sensor height and calibrated sensor acceleration, predict the height and height change rate (predicted velocity) of the exoskeleton device.

[0237] Optionally, the sensor height is determined based on a barometer; or, the current height / altitude of the exoskeleton device is calculated based on the air pressure read from the barometer.

[0238] Optionally, the sensor height and calibrated sensor acceleration are used as observations to predict the predicted height and / or predicted velocity of the exoskeleton device. For example, the sensor height and calibrated sensor acceleration are used as observations, and the state quantities include height, velocity, and acceleration. Here, the observations can be understood as values ​​directly measured by the sensors; however, in this application, the acceleration obtained from the sensors needs to be calibrated first. The state quantities represent the values ​​to be predicted, that is, at least one of the predicted height, predicted velocity, and predicted acceleration needs to be predicted based on the sensor height and calibrated sensor acceleration.

[0239] For example, the above prediction process is implemented using a Kalman filter. That is, the sensor height and calibrated sensor acceleration are used as the observations of the Kalman filter, and the predicted height, predicted velocity, and predicted acceleration are used as the state variables of the Kalman filter. Specifically, as... Figure 2 As shown, based on prior knowledge, empirical values, or theoretical models, the variables of the Kalman filter, in addition to the state variable X mentioned above, also include the state transition matrix F, measurement noise variance R, covariance matrix P, process noise variance Q, and observation matrix H.

[0240] The state transition matrix F is used to describe the changes of each state variable over time.

[0241]

[0242] The measurement noise variance R is used to represent the random error in the observation process. In this application, this random error is based on the variance of the observed quantities (the variance of the sensor height Q2, the variance of the calibrated sensor acceleration Q1) and the first parameter k. gr Sure.

[0243]

[0244] The covariance matrix P is used to indicate the uncertainty in the prediction process.

[0245]

[0246] The process noise variance Q is used to indicate uncaptured random dynamics. It is typically assumed to be zero-mean Gaussian white noise. The random dynamics in this application are determined based on a first parameter.

[0247]

[0248] The observation matrix H is used to map state variables to the observation space, describing the relationship between state variables and observations.

[0249] After initialization, the prediction step begins, predicting the state variables and covariance at the next time step.

[0250] X′=FX

[0251] P′=F·P·F T +Q

[0252] After obtaining the predicted state variables and predicted covariance at the next time step, the predicted state variables, predicted covariance, and Kalman gain K are updated based on the observations at the next time step. The Kalman gain K is used to determine the weights between the predicted state variables and the observations during the update.

[0253] K = P′·H T ·(H·P′·H T +R) -1

[0254] X = X′ + K·(ZH·X′)

[0255] P=(IK·H)·P′

[0256] In the formula, X is the updated predicted state variable, P is the updated predicted covariance, and Z is the observation (observation matrix). Where H is the sensor height, and acc H The acceleration of the calibrated sensor.

[0257] Optionally, the final predicted height and predicted speed are obtained based on the method of predicting the predicted height and predicted speed using the Kalman filter shown above.

[0258] Step 3: Based on the predicted height, obtain the first height change rate; use the predicted speed as the second height change rate; and obtain the third height change rate based on the sensor height; based on the first height change rate, the second height change rate, and the third height change rate, obtain the fused height change rate; and perform scene recognition based on the fused height change rate.

[0259] Optionally, the first height change rate is obtained by differentiating the predicted height; the third height change rate is obtained by differentiating the sensor height; and the fourth height change rate is obtained by smoothing and filtering the third height change rate. Based on the first, second, third, and fourth height change rates, the fused height change rate is obtained.

[0260] For example, HR = p1HR1 + p2HR2 + p3HR3 + p4HR4.

[0261] In the above formula, p1, p2, p3, and p4 are four variables, subject to k. pr Modulation. p1=P1+k pr R P1 p2 = P2 + k pr R P2p3 = P3 + k pr R P3 p4 = P4 + k pr R P4 P1, R P1 P2, R P2 P3, R P3 P4, R P4 These are all constants of empirical parameters. The fused height change rate HR characterizes the wearer's height change rate (positive or negative) along the z-axis in the world coordinate system.

[0262] Optionally, if the fused height transformation rate is greater than the upper threshold, the current motion scene of the exoskeleton device is determined to be an uphill slope scene; if the fused height transformation rate is less than the lower threshold, the current motion scene of the exoskeleton device is determined to be a downhill slope scene; if the fused height transformation rate is within the range of the upper and lower thresholds, the hip joint angle value of the exoskeleton device is obtained, and the scene of the exoskeleton device is further determined based on the hip joint angle value.

[0263] For example, cadence and stride length information are calculated based on hip joint angle values; the current motion scenario is determined based on cadence and stride length information. For instance, when both cadence and stride length information increase, the current motion scenario of the exoskeleton device is determined to be an uphill gentle slope scenario; when both cadence and stride length information decrease, the current motion scenario of the exoskeleton device is determined to be a downhill gentle slope scenario; when both cadence and stride length information remain unchanged, the current motion scenario of the exoskeleton device is determined to be a flat ground scenario.

[0264] In summary, the method illustrated in this application, by obtaining in advance the acceleration data of the z-axis in the celestial direction of the world coordinate system obtained by the test subject walking slowly in different application scenarios, enables a series of sensors (accelerometer, gyroscope, magnetometer) in the IMU to obtain accurate and stable acceleration and angle data without additional tuning steps during use. This reduces the user's time and usage costs, avoids the problem that some low-cost sensors are prone to data shifts in strong motion environments, resulting in unstable results, and enhances the robustness, time cost, and stability of the algorithm.

[0265] Furthermore, by fusing barometer data and IMU data, the impact of turbulent airflow environment on the accuracy of adaptive algorithm in judging usage scenarios is significantly reduced, greatly improving the robustness and environmental adaptability of the algorithm itself.

[0266] Furthermore, the method provided in this application embodiment offers fast real-time response. Processing the IMU and barometer data using a third-order Kalman filter (approximately 500Hz) provides a higher calculation frequency than simply calculating the altitude change rate using the barometer, making it more sensitive to dynamic changes and offering stronger real-time performance. Test results from eight participants show that this algorithm can identify the wearer's application scenario and scenario transitions within hundreds of milliseconds, with an average recognition time of 304–776ms for flat ground, uphill, downhill, and various application scenario transitions. The gait cycle completion rate for recognition is between 87% and 218%. Specifically, the method provided in this application embodiment offers high recognition accuracy. Through the fusion processing of data from the IMU, barometer, and encoder, test results from eight participants show a 100% recognition rate for flat ground, uphill, downhill, and various application scenario transitions.

[0267] Furthermore, the core logic of the method provided in this application embodiment lies in using the acceleration information of different subjects tested in advance as a reference, reducing the tuning steps required by general low-cost inertial navigation systems, and saving algorithm time and usage costs; at the same time, it achieves the fusion and adjustment of IMU and barometer data through third-order Kalman filtering, combining the advantages of barometer data being more stable in an environment without airflow influence and IMU (accelerometer, gyroscope, magnetometer) being less affected by the external environment and having high dynamic sensitivity, which greatly improves the robustness, stability and accuracy of the adaptive identification algorithm; through a high computing frequency (500Hz), it achieves sensitive perception of dynamic changes, which greatly improves the real-time performance of the adaptive identification algorithm; furthermore, by fusing and tuning the output z-axis axial velocity, and combining the hip joint angle information obtained by the encoder, the gait cycle and the step frequency and stride length under each cycle are obtained through the AFO model, which achieves a more refined judgment of the magnitude of the incline and slope; finally, the auxiliary torque corresponding to the determined application scenario is output, making the assistance more suitable for the current usage scenario and optimizing the quality of assistance.

[0268] In summary, the adaptive algorithm provided in this application combines reference dataset information, data collection from inertial navigation, barometers, and encoders, and integrates information from various sensors. By combining the advantages of each sensor, it achieves more robust, real-time, and accurate adaptive recognition for application scenarios, greatly improving the assist quality of active exoskeleton devices.

[0269] Figure 9 This diagram illustrates a structural block diagram of a motion scene recognition device provided in an exemplary embodiment of this application. The motion scene recognition device has the functionality to implement the motion scene recognition method example described above. This functionality can be implemented in hardware or by hardware executing corresponding software. The device can be the server described above, or it can be located within a server. Figure 9 As shown, the device may include: an acquisition module 410, a calibration module 420, and an identification module 430.

[0270] The acquisition module 410 is used to acquire a test parameter sequence and a sensor parameter sequence. The test parameter sequence includes motion parameters of different test objects under different motion scenarios, and the sensor parameter sequence is a set of motion parameters obtained from sensors.

[0271] Calibration module 420 is used to calibrate the sensor parameter sequence based on the test parameter sequence to obtain the calibrated sensor parameter sequence;

[0272] The identification module 430 is used to determine the motion scene of the exoskeleton device based on the calibrated sensor parameter sequence.

[0273] In some embodiments, the calibration module 420 is further configured to input the sensor parameter sequence into a first filter to obtain a sensor parameter sequence output by the first filter; and based on the test parameter sequence, determine that the sensor parameter sequence output by the first filter is the calibrated sensor parameter sequence.

[0274] In some embodiments, the calibration module 420 is further configured to calculate the similarity between the test parameter sequence and the sensor parameter sequence after the (i-1)th filtering, where i is an integer greater than 1; if the similarity is greater than or equal to a similarity threshold, determine the sensor parameter sequence after the (i-1)th filtering as the calibrated sensor parameter sequence; if the similarity is less than the similarity threshold, update the cutoff frequency of the first filter; input the sensor parameter sequence into the updated first filter to obtain the sensor parameter sequence after the i-th filtering; repeat at least two of the above three steps until the calibrated sensor parameter sequence is determined.

[0275] In some embodiments, the exoskeleton device includes an accelerometer and a barometric pressure sensor, and the sensor parameter sequence is a sensor acceleration sequence; the identification module 430 includes an acquisition submodule, a prediction submodule, and an identification submodule.

[0276] An acquisition submodule is used to acquire a sensor altitude sequence, which is obtained from the barometric pressure sensor;

[0277] The prediction submodule is used to predict a set of prediction parameter sequences based on the calibrated sensor acceleration sequence and the sensor height sequence, wherein the set of prediction parameter sequences includes at least one of a prediction height sequence, a prediction velocity sequence and a prediction acceleration sequence.

[0278] The identification submodule is used to determine the motion scene of the exoskeleton device based on the sensor height sequence and the set of prediction parameter sequences.

[0279] In some embodiments, the prediction submodule is further configured to calculate a first variance of the calibrated sensor acceleration sequence; and calculate a second variance of the sensor height sequence; based on the first variance and the second variance, determine a measurement noise variance matrix and a process noise variance matrix, wherein the measurement noise variance matrix is ​​used to simulate the measurement noise caused by sensor error in the motion parameters in the prediction parameter sequence set, and the process noise variance matrix is ​​used to indicate the noise generated during prediction; based on the measurement noise variance matrix, the process noise variance matrix, and the prediction parameter vector at time k, a second filter is used to predict the prediction parameter vector at time k+1, where k is a non-negative integer.

[0280] In some embodiments, the prediction submodule is further configured to: determine an acceleration sequence for the barometric pressure sensor based on the sensor height sequence; calculate a third variance based on the acceleration sequence for the barometric pressure sensor; determine a wind force influence factor for the influence of wind on the barometric pressure sensor based on the first variance and the third variance; and determine the measurement noise variance matrix and the process noise variance matrix based on the first variance, the second variance, and the wind force influence factor.

[0281] In some embodiments, the prediction submodule is further configured to determine the variance skew caused by the wind force based on the first variance and the third variance; if the variance skew is greater than the maximum wind volume threshold, determine the variance skew as the maximum wind volume threshold; if the variance skew is greater than the walking deviation threshold, determine the wind force influence factor as a first ratio, the first ratio being the ratio of the first difference to the maximum wind volume threshold, the first difference being the difference between the variance skew and the maximum wind volume threshold; if the variance skew is less than or equal to the walking deviation threshold, determine the wind force influence factor as a first constant.

[0282] In some embodiments, the prediction submodule is further configured to: determine the variance bias as the third variance when the first variance is outside the effective range; determine the variance bias as the absolute value of a second difference when the first variance is within the effective range and the third difference is greater than the first variance, wherein the second difference is the difference between the first variance and the third difference; and determine the variance bias as the absolute value of a third difference when the first variance is within the effective range and the third difference is less than or equal to the first variance, wherein the third difference is the difference between the third difference and t times the first variance, where t is a positive integer.

[0283] In some embodiments, the identification submodule is further configured to: calculate a height change rate based on the sensor height sequence and the set of predicted parameter sequences; determine that the motion scenario of the exoskeleton device is an uphill steep slope if the height change rate is greater than or equal to a first height change rate threshold; determine that the motion scenario of the exoskeleton device is a downhill steep slope if the height change rate is less than or equal to a second height change rate threshold; acquire stride frequency and stride length if the height change rate is greater than the second height change rate and less than the first height change rate; and determine the motion scenario of the exoskeleton device based on the stride frequency and stride length.

[0284] In some embodiments, the identification submodule is further configured to: determine that the movement scenario of the exoskeleton device is an uphill gentle slope when the step frequency increases and the change in step frequency is greater than or equal to a first change threshold, and the stride length increases and the change in stride length is greater than or equal to a second change threshold; determine that the movement scenario of the exoskeleton device is a downhill gentle slope when the step frequency decreases and the change in step frequency is greater than or equal to the first change threshold, and the stride length decreases and the change in stride length is greater than or equal to the second change threshold; and determine that the movement scenario of the exoskeleton device is flat ground when the change in step frequency is less than the first change threshold, and / or the change in stride length is less than the second change threshold.

[0285] In some embodiments, the set of predicted parameter sequences includes at least one of the predicted height sequence and the predicted acceleration sequence; the identification submodule is further configured to calculate a first height change rate based on the predicted height sequence; calculate a second height change rate based on the predicted acceleration sequence; calculate a third height change rate based on the sensor height sequence; perform a smoothing operation on the third height change rate to obtain a fourth height change rate; and fuse at least two of the first height change rate, the second height change rate, the third height change rate, and the fourth height change rate based on the wind influence factor to obtain a fused height change rate.

[0286] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0287] Figure 10A structural block diagram of a computer device provided in an exemplary embodiment of this application is shown.

[0288] The computer device 800 includes a central processing unit (CPU) 801, a system memory 804 including random access memory (RAM) 802 and read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the CPU 801. The computer device 800 also includes a basic input / output system (I / O system) 806 to facilitate information transfer between various devices within the server, and a mass storage device 807 for storing the operating system 813, application programs 814, and other program modules 815.

[0289] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 for user input, such as a mouse or keyboard. Both the display 808 and the input device 809 are connected to the central processing unit 801 via an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include the input / output controller 810 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, printer, or other types of output devices.

[0290] The mass storage device 807 is connected to the central processing unit 801 via a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable storage media provide non-volatile storage for the computer device 800. That is, the mass storage device 807 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0291] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 804 and mass storage device 807 described above can be collectively referred to as memory.

[0292] The memory stores one or more programs, which are configured to be executed by one or more central processing units 801. The one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 801 executes the one or more programs to implement the methods provided by the above method embodiments.

[0293] According to various embodiments of this application, the computer device 800 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 800 can be connected to a network 812 via a network interface unit 811 connected to the system bus 805, or the network interface unit 811 can be used to connect to other types of networks or remote computer device systems (not shown).

[0294] The memory further includes one or more programs stored in the memory, and the one or more programs include steps performed by a computer device in the methods provided in the embodiments of this application.

[0295] In an exemplary embodiment, this application provides an exoskeleton device, which includes sensors and is used to implement the motion scene recognition method provided in the above-described method embodiments.

[0296] In an exemplary embodiment, this application provides a chip that includes programmable logic circuits and / or program instructions. When the chip is run on a computer device, it is used to implement the motion scene recognition method provided in the above-described method embodiments.

[0297] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the motion scene recognition method described above.

[0298] In an exemplary embodiment, a computer program product is also provided, which, when executed by a processor, is used to implement the motion scene recognition method described above.

[0299] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0300] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A motion scene recognition method, characterized in that, The method is performed by an exoskeleton device, the exoskeleton device including sensors, and the method includes: The test parameter sequence and sensor parameter sequence are obtained. The test parameter sequence includes motion parameters of different test objects under different motion scenarios, and the sensor parameter sequence is a set of motion parameters obtained from the sensors. Based on the test parameter sequence, the sensor parameter sequence is calibrated to obtain the calibrated sensor parameter sequence; Based on the calibrated sensor parameter sequence, the motion scenario of the exoskeleton device is determined.

2. The method according to claim 1, characterized in that, The step of calibrating the sensor parameter sequence based on the test parameter sequence to obtain the calibrated sensor parameter sequence includes: The sensor parameter sequence is input into the first filter to obtain the sensor parameter sequence output by the first filter; Based on the test parameter sequence, the sensor parameter sequence output by the first filter is determined to be the calibrated sensor parameter sequence.

3. The method according to claim 2, characterized in that, The step of determining the sensor parameter sequence output by the first filter as the calibrated sensor parameter sequence based on the test parameter sequence includes: Calculate the similarity between the test parameter sequence and the sensor parameter sequence after the (i-1)th filtering, where i is an integer greater than 1; If the similarity is greater than or equal to the similarity threshold, the sensor parameter sequence after the (i-1)th filtering is determined as the calibrated sensor parameter sequence. If the similarity is less than the similarity threshold, the cutoff frequency of the first filter is updated; the sensor parameter sequence is input into the updated first filter to obtain the sensor parameter sequence after the i-th filtering. Repeat at least two of the above three steps until the calibrated sensor parameter sequence is determined.

4. The method according to any one of claims 1 to 3, characterized in that, The exoskeleton device includes an accelerometer and a pressure sensor, and the sensor parameter sequence is a sensor acceleration sequence. Determining the motion scenario of the exoskeleton device based on the calibrated sensor parameter sequence includes: Acquire a sensor altitude sequence, which is obtained from the barometric pressure sensor; Based on the calibrated sensor acceleration sequence and sensor height sequence, a set of prediction parameter sequences is predicted, which includes at least one of a prediction height sequence, a prediction velocity sequence, and a prediction acceleration sequence. Based on the sensor height sequence and the set of predicted parameter sequences, the motion scene in which the exoskeleton device is located is determined.

5. The method according to claim 4, characterized in that, The set of predicted parameter sequences, obtained based on the calibrated sensor acceleration sequence and the sensor height sequence, includes: Calculate the first variance of the calibrated sensor acceleration sequence; and calculate the second variance of the sensor height sequence; Based on the first variance and the second variance, a measurement noise variance matrix and a process noise variance matrix are determined. The measurement noise variance matrix is ​​used to simulate the measurement noise caused by sensor errors in the motion parameters in the predicted parameter sequence set. The process noise variance matrix is ​​used to indicate the noise generated during prediction. Based on the measurement noise variance matrix, the process noise variance matrix, and the prediction parameter vector at time k, the prediction parameter vector at time k+1 is obtained using the second filter, where k is a non-negative integer.

6. The method according to claim 5, characterized in that, The step of determining the measurement noise variance matrix and the process noise variance matrix based on the first variance and the second variance includes: Based on the sensor altitude sequence, an acceleration sequence for the barometric pressure sensor is determined; Based on the acceleration sequence for the barometric pressure sensor, calculate the third difference; Based on the first variance and the third variance, the wind force influence factor that affects the air pressure sensor due to wind force is determined; Based on the first variance, the second variance, and the wind force influence factor, the measurement noise variance matrix and the process noise variance matrix are determined.

7. The method according to claim 6, characterized in that, The step of determining the wind force influence factor, based on the first variance and the third variance, which determines the impact of wind force on the barometric pressure sensor, includes: Based on the first variance and the third variance, determine the variance skew caused by the wind force. If the variance deviation is greater than the maximum airflow threshold, the variance deviation is determined to be the maximum airflow threshold. If the variance deviation is greater than the walking deviation threshold, the wind force influence factor is determined to be the first ratio, which is the ratio of the first difference to the maximum wind volume threshold. If the variance deviation is less than or equal to the walking deviation threshold, the wind force influence factor is determined to be the first constant.

8. The method according to claim 7, characterized in that, The step of determining the variance skewness caused by the wind force based on the first variance and the third variance includes: If the first variance is outside the valid range, the variance bias is determined to be the third variance; If the first variance is within the effective range and the third variance is greater than the first variance, the variance bias is determined to be the absolute value of the second difference, where the second difference is the difference between the first variance and the third difference. If the first variance is within the effective range and the third variance is less than or equal to the first variance, the variance bias is determined to be the absolute value of the third difference, where the third difference is the difference between the third difference and t times the first variance, and t is a positive integer.

9. The method according to any one of claims 6 to 8, characterized in that, The step of determining the motion scene of the exoskeleton device based on the sensor height sequence and the predicted parameter sequence set includes: The rate of change of altitude is calculated based on the sensor altitude sequence and the set of predicted parameter sequences; If the height change rate is greater than or equal to the first height change rate threshold, the motion scenario of the exoskeleton device is determined to be an uphill steep slope; If the height change rate is less than or equal to the second height change rate threshold, the motion scenario of the exoskeleton device is determined to be a steep downhill slope; When the height change rate is greater than the second height change rate and less than the first height change rate, the step frequency and stride length are obtained; and the motion scene of the exoskeleton device is determined based on the step frequency and stride length.

10. The method according to claim 9, characterized in that, Determining the motion scenario of the exoskeleton device based on the cadence information and stride information includes: When the step frequency increases and the change in step frequency is greater than or equal to a first change threshold, and the stride length increases and the change in stride length is greater than or equal to a second change threshold, the motion scenario of the exoskeleton device is determined to be an uphill gentle slope. If the step frequency decreases and the change in step frequency is greater than or equal to the first change threshold, and the stride length decreases and the change in stride length is greater than or equal to the second change threshold, the motion scenario of the exoskeleton device is determined to be a gentle downhill slope. If the change in step frequency is less than the first change threshold, and / or the change in stride length is less than the second change threshold, the motion scenario of the exoskeleton device is determined to be flat ground.

11. The method according to claim 9 or 10, characterized in that, The set of predicted parameter sequences includes at least one of the predicted height sequence and the predicted acceleration sequence; The calculation of the altitude change rate based on the sensor altitude sequence and the set of predicted parameter sequences includes at least one of the following: Based on the predicted height sequence, a first rate of change of height is calculated; and based on the predicted acceleration sequence, a second rate of change of height is calculated. Based on the sensor height sequence, the third height change rate is calculated; A smoothing operation is performed on the third rate of change of altitude to obtain a fourth rate of change of altitude; Based on the wind influence factor, the fused height change rate is obtained by fusing at least two of the first height change rate, the second height change rate, the third height change rate, and the fourth height change rate.

12. A motion scene recognition device, characterized in that, The device includes: The acquisition module is used to acquire test parameter sequences and sensor parameter sequences. The test parameter sequences include motion parameters of different test objects under different motion scenarios, and the sensor parameter sequences are a set of motion parameters obtained from sensors. A calibration module is used to calibrate the sensor parameter sequence based on the test parameter sequence to obtain a calibrated sensor parameter sequence. The identification module is used to determine the motion scene of the exoskeleton device based on the calibrated sensor parameter sequence.

13. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the motion scene recognition method as described in any one of claims 1 to 11.

14. An exoskeleton device, characterized in that, The exoskeleton device is used to perform the motion scene recognition method as described in any one of claims 1 to 11.

15. A computer storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the motion scene recognition method as described in any one of claims 1 to 11.

16. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform the motion scene recognition method as described in any one of claims 1 to 11.

Images