Waist Movement Trajectory Prediction Method, Device, Equipment and Wearable Walking Aid Equipment

Abstract

This application is applicable to the field of robot control technology, and provides a method, device, equipment and wearable walking assistance equipment for predicting the waist movement trajectory. The method includes: obtaining the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; extracting features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; embedding the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory; inputting the optimized historical waist movement trajectory into a waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking on the second terrain, and generating a target waist movement trajectory corresponding to the wearer when walking on the second terrain, realizing fast and effective prediction of the waist movement trajectory of the wearer under multi-terrain walking.

Description

Technical Field

[0001] This application belongs to the technical field of robot control, and particularly relates to a method, device, equipment, and wearable walking assistance equipment for predicting the waist movement trajectory. Background Art

[0002] Wearable walking assistance equipment such as exoskeletons and powered prostheses use wearable mechanical structures and high-tech means such as sensors, control systems, and power devices to provide functions such as assisting walking, load-bearing, and rehabilitation training for the wearer, thereby enhancing or restoring the wearer's motor ability.

[0003] Since the movement of the human body's center of mass comprehensively reflects the coordinated movement of the lower limb joints, it not only represents the coordination and stability of walking but also reflects the movement intention of the human body during walking; at the same time, the human body's center of mass is located at the waist position. Therefore, the waist movement trajectory can reflect the movement of the human body's center of mass.

[0004] Currently, advanced wearable walking assistance equipment has begun to consider the movement of the wearer's center of mass, so that the wearable walking assistance equipment can provide more natural assistance to the wearer, especially during the transition between various movements. However, currently, most researchers measure the waist movement trajectory of the wearer through two types of methods: external and wearable, and lack the prediction of the waist movement trajectory, resulting in the wearable walking assistance equipment being unable to effectively assist the wearer in walking and unable to obtain the movement intention of the wearer during walking. Summary of the Invention

[0005] The embodiments of this application provide a method, device, equipment, and wearable walking assistance equipment for predicting the waist movement trajectory, which can solve the problem in the prior art that only the waist movement trajectory of the wearer wearing the wearable walking assistance equipment is measured and the waist movement trajectory of the wearer is lacking in prediction, resulting in the inability to effectively guide the wearable walking assistance equipment to assist the wearer in walking and the inability to obtain the movement intention of the wearer during walking.

[0006] In the first aspect, the embodiments of this application provide a method for predicting the waist movement trajectory, and the method includes:

[0007] Obtain the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and obtain the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;

[0008] Extract features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;

[0009] Embed the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory;

[0010] Input the optimized historical waist movement trajectory into the waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking on the second terrain, and generate the target waist movement trajectory corresponding to the wearer when walking on the second terrain.

[0011] In a possible implementation manner of the first aspect, the extracting the terrain feature data corresponding to the second terrain from the environmental point cloud data of the second terrain includes:

[0012] Perform two-dimensional projection on all the point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data;

[0013] Extract the terrain features in the binary image through a preset terrain feature extraction model to obtain the initial terrain feature data corresponding to the second terrain;

[0014] Compensate the initial terrain feature data to obtain the terrain feature data.

[0015] In a possible implementation manner of the first aspect, the compensating the initial terrain feature data to obtain the terrain feature data includes:

[0016] Encode the terrain feature data corresponding to the current moment to obtain a first binary image;

[0017] Obtain the retrospective moment corresponding to the retrospective distance through a visual inertial odometry system; wherein, the retrospective distance is used to represent the missing terrain feature data in the first binary image;

[0018] Encode the missing terrain feature data corresponding to the retrospective moment to obtain a second binary image;

[0019] Extract the pixels corresponding to the retrospective distance in the second binary image according to the retrospective distance to obtain missing pixels;

[0020] Splice the missing pixels into the first binary image to obtain an optimized first binary image;

[0021] Based on the optimized first binary image, obtain the terrain feature data.

[0022] In a possible implementation manner of the first aspect, the historical waist movement trajectory includes: the X-axis historical waist movement trajectory, the Y-axis historical waist movement trajectory, and the Z-axis historical waist movement trajectory;

[0023] The embedding the terrain feature data into the historical waist movement trajectory to generate the optimized historical waist movement trajectory corresponding to the historical waist movement trajectory includes:

[0024] Taking the historical waist movement trajectory of the X-axis as a reference, interpolation processing is respectively performed on the historical waist movement trajectory of the Y-axis and the historical waist movement trajectory of the Z-axis to obtain the historical waist movement trajectory of the X-Y axis and the historical waist movement trajectory of the X-Z axis;

[0025] Filtering processing is respectively performed on the historical waist movement trajectory of the X-Y axis and the historical waist movement trajectory of the X-Z axis to obtain the optimized historical waist movement trajectory of the X-Y axis and the optimized historical waist movement trajectory of the X-Z axis;

[0026] Adding the terrain feature data to the end of the optimized historical waist movement trajectory of the X-Y axis and the optimized historical waist movement trajectory of the X-Z axis respectively to generate the optimized historical waist movement trajectory corresponding to the historical waist movement trajectory.

[0027] In a possible implementation manner of the first aspect, the obtaining of the environmental point cloud data of the second terrain that the wearer will walk on at a future moment includes:

[0028] Obtaining the initial environmental point cloud data of the second terrain that the wearer will walk on at a future moment through a visual sensor fixed on a predetermined body part of the wearer;

[0029] Performing preprocessing on the initial environmental point cloud data to obtain the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; wherein, the preprocessing includes at least one of the following: direct filtering, rotation processing.

[0030] In a possible implementation manner of the first aspect, the obtaining of the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment includes:

[0031] Obtaining the walking motion information data and the camera pose data of the wearer when walking on the first terrain through an inertial measurement unit connected to the visual sensor;

[0032] Calculating the camera motion trajectory corresponding to the visual sensor according to the walking motion information data and the camera pose data to obtain the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment.

[0033] In a possible implementation manner of the first aspect, the waist movement trajectory prediction model is obtained through the following method:

[0034] Obtaining a training set; wherein, the training set includes a plurality of sample waist movement trajectories and prediction results of the plurality of sample waist movement trajectories;

[0035] Constructing an initial waist movement trajectory prediction model;

[0036] Train the initial lumbar motion trajectory prediction model using multiple of the sample lumbar motion trajectories and the prediction results of the multiple sample lumbar motion trajectories to obtain the lumbar motion trajectory prediction model.

[0037] In a second aspect, an embodiment of the present application provides a lumbar motion trajectory prediction device, the device includes:

[0038] An acquisition module, configured to acquire the historical lumbar motion trajectory of the wearer when walking on the first terrain at a historical moment, and acquire the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;

[0039] An encoding module, configured to perform feature extraction on the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;

[0040] An optimization module, configured to embed the terrain feature data into the historical lumbar motion trajectory to generate an optimized historical lumbar motion trajectory corresponding to the historical lumbar motion trajectory;

[0041] A prediction module, configured to input the optimized historical lumbar motion trajectory into the lumbar motion trajectory prediction model, predict the lumbar motion trajectory of the wearer when walking on the second terrain, and generate a target lumbar motion trajectory corresponding to the wearer when walking on the second terrain.

[0042] In a third aspect, an embodiment of the present application provides a lumbar motion trajectory prediction device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the lumbar motion trajectory prediction method described in any one of the above.

[0043] In a fourth aspect, an embodiment of the present application provides a wearable walking assistance device, including a controller, and the controller is configured to execute the lumbar motion trajectory prediction method described in any one of the above.

[0044] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the lumbar motion trajectory prediction method described in any one of the above.

[0045] In a sixth aspect, an embodiment of the present application provides a computer program product, and when the computer program product runs on a terminal device, it causes the terminal device to execute a lumbar motion trajectory prediction method described in any one of the above.

[0046] The beneficial effects of the embodiments of the present application compared with the prior art are:

[0047] The waist movement trajectory prediction method provided by the embodiment of the present application obtains the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and obtains the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; extracts features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; then, embeds the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory; finally, inputs the optimized historical waist movement trajectory into the waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking on the second terrain, and generates a target waist movement trajectory corresponding to the wearer when walking on the second terrain. The present application solves the problem in the prior art that only the waist movement trajectory of the wearer wearing a wearable walking assistance device is measured, but the prediction of the waist movement trajectory of the wearer is lacking, resulting in the inability to effectively guide the wearable walking assistance device to assist the wearer in walking and the inability to obtain the movement intention of the wearer when walking. Thus, it realizes the fast and effective prediction of the waist movement trajectory of the wearer wearing a wearable walking assistance device under daily multi-terrain walking. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0049] Figure 1 is a schematic flowchart of a waist movement trajectory prediction method provided by an embodiment of the present application;

[0050] Figure 2 is a schematic structural diagram of a preset terrain feature extraction model and a waist movement trajectory prediction model provided by an embodiment of the present application.

[0051] Figure 3 is a schematic flowchart of a waist movement trajectory prediction method provided by another embodiment of the present application;

[0052] Figure 4 is a schematic diagram of a terrain feature compensation method provided by an embodiment of the present application;

[0053] Figure 5 is a schematic structural diagram of a waist movement trajectory prediction device provided by an embodiment of the present application;

[0054] Figure 6 is a schematic structural diagram of a waist movement trajectory prediction device provided by an embodiment of the present application;

[0055] Figure 7 It is a schematic structural diagram of a wearable walking assistance device provided by an embodiment of the present application. Detailed implementation manners

[0056] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0057] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0058] It should also be understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0059] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" according to the context.

[0060] In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0061] References to "an embodiment" or "some embodiments" in the description of this application mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification are not necessarily all referring to the same embodiment, but rather mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0062] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for predicting the waist movement trajectory provided by an embodiment of this application. The method for predicting the waist movement trajectory includes:

[0063] S11. Obtain the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and obtain the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;

[0064] S12. Extract features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;

[0065] S13. Embed the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory;

[0066] S14. Input the optimized historical waist movement trajectory into the waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking on the second terrain, and generate a target waist movement trajectory corresponding to the wearer when walking on the second terrain.

[0067] It should be noted that in this embodiment, the execution subject may be a terminal device such as a server, and no specific limitation is made thereto.

[0068] In step S11, the wearer refers to a user who walks while wearing a wearable walking assistance device. A wearable walking assistance device refers to a device that can be worn on the human body to assist or enhance walking ability, such as a lower limb exoskeleton robot, a smart walking assistance robot, etc. The first terrain refers to the terrain that the wearer has walked on. The waist movement trajectory generated by the wearer when walking on the first terrain has been recorded, that is, the historical waist movement trajectory has been recorded. The historical waist movement trajectory refers to the waist movement trajectory or path data of the wearer when walking on the first terrain at past historical moments. Usually, the historical waist movement trajectory can be recorded by an inertial measurement unit (such as a gyroscope, an accelerometer) to record the movement information of the wearer's waist in three-dimensional space, such as position, speed, etc. The historical waist movement trajectory provides the movement patterns and habits of the wearer when walking on similar terrains. The second terrain refers to the terrain that the wearer will walk on in front during the walking process. The environmental point cloud data is the three-dimensional space information of the second terrain obtained by a visual sensor (such as a depth camera). The environmental point cloud data is usually represented by point clouds. Each point cloud contains its coordinate information in three-dimensional space, and these point clouds constitute the three-dimensional model of the second terrain. The environmental point cloud data of the second terrain is used to analyze and predict the waist movement trajectory of the wearer on the second terrain.

[0069] In step S12, the terrain feature data is the key terrain feature information extracted from the environmental point cloud data of the second terrain, such as slope, terrain undulation, obstacle position, etc. The terrain feature data is used to describe the physical characteristics and geometric structure of the second terrain, facilitating subsequent analysis and prediction.

[0070] In step S13, the optimized historical waist movement trajectory is an optimized waist trajectory obtained by combining the terrain feature data with the historical waist movement trajectory, and it is an adjustment and optimization of the historical waist movement trajectory. Since different terrains may affect the walking manner and habits of the wearer, by combining the terrain feature data with the historical waist movement trajectory, the prediction of the waist movement trajectory can be made more accurate and reliable.

[0071] In step S14, the waist movement trajectory prediction model is a deep learning model that is pre-constructed and trained. It is used to predict the waist movement trajectory of the wearer when walking on the second terrain, that is, the target waist movement trajectory, according to the input optimized historical waist movement trajectory. This waist movement trajectory prediction model learns the relationship between terrain features and waist movement trajectories by training a large amount of data. The target waist movement trajectory can help the wearer understand and adapt to the changes in the second terrain in advance, so as to walk more safely and comfortably. At the same time, it can also be used to optimize the design and control strategy of the wearable walking assistance device to provide better assistance effects.

[0072] It can be understood that the waist movement trajectory prediction method provided by the embodiments of the present application obtains the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and obtains the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; extracts features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; then, embeds the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory; finally, inputs the optimized historical waist movement trajectory into the waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking on the second terrain, and generates a target waist movement trajectory corresponding to the wearer when walking on the second terrain. The present application solves the problem in the prior art that only the waist movement trajectory of the wearer wearing a wearable walking assistance device is measured and the prediction of the wearer's waist movement trajectory is lacking, resulting in the inability to effectively guide the wearable walking assistance device to assist the wearer in walking and the inability to obtain the movement intention of the wearer when walking, thereby realizing the rapid and effective prediction of the waist movement trajectory of the wearer wearing a wearable walking assistance device under daily multi-terrain walking.

[0073] In a possible implementation manner, extracting features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain includes:

[0074] Performing two-dimensional projection on all the point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data;

[0075] Extracting the terrain features in the binary image through a preset terrain feature extraction model to obtain initial terrain feature data corresponding to the second terrain;

[0076] Compensating the initial terrain feature data to obtain the terrain feature data.

[0077] It should be noted that a binary image is an image representation method, and each pixel point in the image has only two possible values, usually 0 and 1 (or black and white). In a binary image, all pixels are divided into two categories, usually used to represent the target and background in the image. Since the environmental point cloud data is three-dimensional and contains the coordinates of each point cloud in three-dimensional space, performing two-dimensional projection on the environmental point cloud data to obtain a binary image is used to simplify and highlight the terrain features of the second terrain ahead, facilitating subsequent extraction of the terrain features.

[0078] The preset terrain feature extraction model is a pre-constructed image processing model that has been trained to identify and analyze the terrain features in a binary image. As Figure 2 shown, Figure 2It is a schematic structural diagram of a preset terrain feature extraction model and a waist movement trajectory prediction model provided by an embodiment of the present application. The preset terrain feature extraction model extracts relevant terrain features from a binary image, classifies their types, marks labels, etc., such as flat ground, going up stairs, going up a slope, going down stairs, going down a slope, etc., to form initial terrain feature data related to the second terrain.

[0079] The initial terrain feature data is the raw terrain feature data directly extracted from the binary image without being processed, which may contain some noise or errors and needs to be further processed or optimized.

[0080] Such as Figure 3 shown, Figure 3 It is a schematic flowchart of a waist movement trajectory prediction method provided by another embodiment of the present application. In the process of obtaining environmental point cloud data, due to the limitation of the visual sensor's perspective, while the visual sensor obtains environmental information in the distance ahead, that is, while obtaining the environmental point cloud data of the second terrain, it will inevitably lose the environmental information within a certain range in front of the visual sensor, which causes interference to the calculation of the terrain feature data of the walking environment. Therefore, it is necessary to compensate the current environmental feature information to obtain terrain feature data with complete information, that is, to obtain terrain feature data by compensating the initial terrain feature data. Among them, the terrain feature data is the data obtained after optimizing and processing the initial terrain feature data through compensation, etc. These data are more accurate and reliable and can comprehensively reflect various features of the second terrain. Compensation is a process of optimizing and processing the initial terrain feature data, such as filtering, interpolation, denoising, etc., to eliminate or reduce the errors or noise in it to improve the accuracy and reliability of the terrain feature data.

[0081] In a possible implementation manner, compensating the initial terrain feature data to obtain terrain feature data includes:

[0082] S21. Encode the terrain feature data corresponding to the current moment to obtain a first binary image;

[0083] S22. Obtain the backtracking moment corresponding to the backtracking distance through the visual inertial odometry system; where the backtracking distance is used to represent the missing terrain feature data in the first binary image;

[0084] S23. Encode the missing terrain feature data corresponding to the backtracking moment to obtain a second binary image;

[0085] S24. Extract the pixels corresponding to the backtracking distance in the second binary image according to the backtracking distance to obtain missing pixels;

[0086] S25. Splice the missing pixels into the first binary image to obtain an optimized first binary image;

[0087] S26. Based on the optimized first binary image, obtain terrain feature data.

[0088] It should be noted that in step S21, the first binary image is an image obtained by encoding the terrain feature data at the current moment. The first binary image can simplify the representation of the terrain feature data and facilitate subsequent data processing. As Figure 4 shown, Figure 4 is a schematic diagram of a terrain feature compensation method provided by an embodiment of the present application. As Figure 4 shown, image t represents the first binary image at the current moment t.

[0089] In step S22, the Visual-Inertial Odometry (VIO) system is an odometry calculation system that combines a visual sensor (such as a depth camera) and an inertial measurement unit (such as a gyroscope and an accelerometer), and can be used to estimate the motion trajectory and pose of a device. The visual sensor can provide rich environmental information, but is easily affected by factors such as illumination and occlusion; the inertial measurement unit can provide high-frequency inertial data, but the error will gradually increase over time. Therefore, fusing the visual sensor and the inertial measurement unit can complement their advantages to improve the robustness and accuracy of the VIO system.

[0090] In this embodiment, the VIO system is used to determine the backtracking moment corresponding to the backtracking distance, that is, according to the historical motion trajectory of the device and the environmental point cloud data, backtrack to a certain moment in the past. As Figure 4 shown, d represents the backtracking distance, t - Δt represents the backtracking moment, and the backtracking distance d corresponds to the missing terrain feature data in the first binary image image t . Among them, the backtracking distance is the spatial distance from the current moment to the backtracking moment, and is used to characterize the range or degree of the missing terrain feature data in the first binary image. The backtracking moment is a past time point determined by the VIO system, and the terrain feature data corresponding to this time point will be used to supplement the missing terrain feature data in the first binary image at the current moment.

[0091] In step S23, the missing terrain feature data corresponding to the backtracking moment is encoded to obtain a second binary image. Among them, the second binary image is an image obtained by finding the terrain feature data corresponding to the backtracking moment through a backtracking mechanism and encoding the terrain feature data corresponding to the backtracking moment. The second binary image is used to supplement the missing terrain feature data in the first binary image. As Figure 4 shown, imaget-Δt represents the second binary image at the backtracking moment t-Δt.

[0092] It should be noted that since there is a linear relationship between the pixels of the binary image and the walking distance, the backtracking distance can be calculated according to the missing pixel width. In step S24, according to the backtracking distance, the pixels corresponding to the backtracking distance are extracted from the second binary image to obtain the missing pixels. The missing pixels refer to the pixels corresponding to the missing terrain feature data of the first binary image extracted from the second binary image.

[0093] In steps S25 and S26, the extracted missing pixels are spliced into the first binary image to obtain an optimized first binary image; thus, based on the optimized first binary image, terrain feature data is obtained. As Figure 4 shown, result represents the optimized first binary image. The optimized first binary image contains the missing pixels of the missing terrain feature data. Compared with the first binary image, it contains more complete and accurate terrain feature data. So far, the compensation process for the initial terrain feature data is completed, and terrain feature data with complete information is obtained, thereby improving the accuracy of terrain feature extraction and classification.

[0094] In a possible implementation, the historical waist movement trajectory includes: the X-axis historical waist movement trajectory, the Y-axis historical waist movement trajectory, and the Z-axis historical waist movement trajectory;

[0095] Embedding the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory, including:

[0096] Taking the X-axis historical waist movement trajectory as a reference, interpolation processing is respectively performed on the Y-axis historical waist movement trajectory and the Z-axis historical waist movement trajectory to obtain the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory;

[0097] Filtering processing is respectively performed on the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory to obtain the optimized X-Y axis historical waist movement trajectory and the optimized X-Z axis historical waist movement trajectory;

[0098] Adding the terrain feature data to the ends of the optimized X-Y axis historical waist movement trajectory and the optimized X-Z axis historical waist movement trajectory respectively to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory.

[0099] It should be noted that in the world coordinate system, the historical waist movement trajectory includes movement trajectories in three axis directions, namely, the X-axis historical waist movement trajectory, the Y-axis historical waist movement trajectory, and the Z-axis historical waist movement trajectory. Among them, the X-axis historical waist movement trajectory represents the distance that the wearer's waist moves forward or backward during walking; the Y-axis historical waist movement trajectory represents the distance that the wearer's waist tilts laterally during walking; the Z-axis historical waist movement trajectory represents the distance that the wearer's waist height changes during walking.

[0100] In this embodiment, first, taking the X-axis historical waist movement trajectory as a reference, interpolate the Y-axis historical waist movement trajectory to obtain the X-Y axis historical waist movement trajectory; and, taking the X-axis historical waist movement trajectory as a reference, interpolate the YZ-axis historical waist movement trajectory to obtain the X-Z axis historical waist movement trajectory. Among them, the interpolation process is a method of estimating unknown data points through known data points. That is, according to each point on the X-axis, find the corresponding points on the Y-axis and Z-axis, so as to obtain the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory, that is, convert the three-dimensional movement trajectory into two two-dimensional movement trajectories, making the movement trajectories of the three axes synchronous in time, which is convenient for subsequent analysis.

[0101] Secondly, after obtaining the two movement trajectories of the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory, in order to eliminate the burrs in the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory, it is necessary to perform filtering processing on these two movement trajectories of the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory, so as to obtain the optimized X-Y axis historical waist movement trajectory and the optimized X-Z axis historical waist movement trajectory. Among them, the filtering process is a signal processing used to remove noise or unwanted frequency components in the signal. The purpose of performing filtering processing on these two movement trajectories of the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory is to smooth the movement trajectory, reduce noise, burrs, etc. caused by external interference, and make the historical waist movement trajectory more accurate and reliable.

[0102] Then, after obtaining the optimized X-Y axis historical waist movement trajectory and the optimized X-Z axis historical waist movement trajectory, the terrain feature data extracted by the preset terrain feature extraction model are respectively added to the end of the optimized X-Y axis historical waist movement trajectory and the optimized X-Z axis historical waist movement trajectory, that is, the data related to the terrain features are combined with the historical waist movement trajectory to generate an optimized historical waist movement trajectory containing terrain information. Among them, the optimized historical waist movement trajectory is a more comprehensive and accurate movement trajectory that has undergone interpolation and filtering processing and incorporates terrain feature data. It not only reflects the historical movement of the wearer's waist but also takes into account the influence of the terrain, making it more suitable for motion analysis and optimization under complex terrain conditions.

[0103] In a possible implementation, obtaining the environmental point cloud data of the second terrain that the wearer will walk on at a future moment includes:

[0104] Obtaining the initial environmental point cloud data of the second terrain that the wearer will walk on at a future moment through a visual sensor fixed on a predetermined body part of the wearer;

[0105] Preprocessing the initial environmental point cloud data to obtain the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; where the preprocessing includes at least one of the following: straight-through filtering, rotation processing.

[0106] It should be noted that a visual sensor is a sensor that uses optical elements and imaging devices to obtain two-dimensional image information and three-dimensional depth information of objects in an external scene, such as a depth camera. Usually, technologies such as infrared light or lasers are used to measure the distance between the sensor and each point in the scene and generate point cloud data. Visual sensors are widely used in fields such as robot navigation and three-dimensional reconstruction. Among them, in this embodiment, the visual sensor is fixed on a predetermined body part of the wearer, and the predetermined body part can be the waist of the wearer. The initial environmental point cloud data is the three-dimensional point cloud data of the second terrain directly obtained by the visual sensor without being processed. These data contain the three-dimensional coordinate information of all visible points in the scene and may also contain noise, redundant data, or distortion, etc. The environmental point cloud data is the point cloud data with improved quality obtained after preprocessing the initial environmental point cloud data.

[0107] Preprocessing is a series of data processing operations performed on the initial environmental point cloud data, aiming to improve the data quality of the initial environmental point cloud data, remove noise, reduce redundancy, etc., facilitate the analysis and processing of subsequent data, and improve the efficiency and accuracy of data processing. Among them, preprocessing mainly includes through filtering and rotation processing. Through filtering, by setting a certain spatial range, only the point cloud data within the range is retained, and the point cloud data outside the range is eliminated. Through through filtering, background noise, unnecessary elements, or abnormal data caused by camera errors can be effectively removed. Rotation processing is to transform the point cloud data from the initial camera coordinate system to the world coordinate system by rotating the point cloud data, so as to make it easier to identify and extract ground feature data. Rotation processing can help eliminate the deformation of environmental point cloud data caused by changes in angle or wearer posture.

[0108] It can be understood that the initial environmental point cloud data of the second terrain that the wearer will walk on in the future is obtained by fixing the visual sensor on the predetermined body part of the wearer; and the initial environmental point cloud data is pre-processed by straight-through filtering, rotation processing, etc. to improve the data quality, and the final environmental point cloud data is obtained, so as to facilitate the subsequent data analysis and processing and improve the efficiency and accuracy of data processing.

[0109] In a possible implementation, obtaining the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment includes:

[0110] Acquire walking motion information data and camera position data of the wearer when walking on the first terrain through an inertial measurement unit connected to the visual sensor;

[0111] According to the walking motion information data and the camera posture data, the camera motion trajectory corresponding to the visual sensor is calculated to obtain the historical waist motion trajectory of the wearer when walking on the first terrain at the historical moment.

[0112] It should be noted that an Inertial Measurement Unit (IMU) is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object, usually including a three-axis accelerometer and a three-axis gyroscope. The inertial measurement unit can measure the angular velocity and acceleration of an object in three-dimensional space and calculate the attitude of the object from this, and is commonly used in fields such as motion tracking and navigation. The walking motion information data is the data related to motion generated by the wearer during walking obtained through the inertial measurement unit, such as walking frequency, step length, acceleration change, speed, etc., and is commonly used to analyze the wearer's motion patterns and habits. The camera pose data refers to the position information and attitude information of the visual sensor in three-dimensional space. Among them, the position information describes the coordinates of the visual sensor in three-dimensional space, and the attitude information describes the orientation and rotation state of the visual sensor. The camera pose data can describe how the visual sensor moves relative to the surrounding environment, that is, it can describe how the wearer moves relative to the surrounding environment.

[0113] The camera motion trajectory refers to the motion trajectory formed by the visual sensor moving along a certain path in three-dimensional space over a period of time. The world coordinate system takes the initial point as the origin and does not change in position and attitude with the movement of a person. Since the visual sensor is fixedly connected to the preset fixed position of the human body, i.e., the waist position, therefore, the position of the visual sensor can be approximately regarded as the waist position of the human body. Further, the position change of the visual sensor coordinate system relative to the world coordinate system can be approximately regarded as the waist movement trajectory of the human body. That is, the camera motion trajectory can be used as the waist movement trajectory of the wearer.

[0114] It should be noted that, as Figure 3 shown, Figure 3 is a schematic flowchart of a method for predicting the waist movement trajectory provided by another embodiment of the present application. As Figure 3 shown, through a series of function processing, pre-integration, bundle adjustment, etc. on the motion information stream and picture information stream obtained by the inertial measurement unit and the depth camera, the historical waist movement trajectory is obtained. Among them, Figure 3 in, represents the historical waist movement trajectory, Indicates the optimized historical waist movement trajectory. The process of calculating the camera movement trajectory corresponding to the visual sensor is also the process of measuring the historical waist movement trajectory of the wearer when walking on the first terrain at historical moments. In this embodiment, in order to accurately obtain the historical waist movement trajectory, a visual inertial odometry system is used for measurement. Among them, the visual information comes from the visual sensor, and the motion information comes from the inertial measurement unit (IMU). The visual sensor can make up for the inevitable systematic deviation generated by the inertial measurement unit, and the inertial measurement unit can make up for the low adaptability of the visual sensor in the case of rapid movement or drastic changes in the motion state. The process is as follows:

[0115] First, define the state of the visual inertial odometry system (VIO system). The state of the VIO system can be described by the attitude, position, velocity of the IMU, and the IMU bias, as shown in Equation (1).

[0116] S i =[R i ,p i ,v i ,b i (1)

[0117] In Equation (1), S i represents the state of the VIO system; (R i ,p i ) represents the camera pose data, R i represents the position information of the camera, p i represents the attitude information of the camera, (R i ,p i ) belongs to the special Euclidean group SE(3); v i represents the velocity of the camera, b i represents the IMU bias, where, represents the bias of the gyroscope, represents the bias of the accelerometer,

[0118] Since the acquisition frequency of the IMU is higher than that of the visual sensor, in order to use the measurement data of the IMU to estimate the state of the VIO system, it is necessary to pre-integrate the measurement data of the IMU between two consecutive key frames i and i + 1, and then obtain the change values of rotation, velocity, and displacement between the two key frames; according to the change values obtained by pre-integration, calculate the rotation residual term, velocity residual term, and displacement residual term; further, the inertial residual term can be obtained as shown in Equation (2).

[0119]

[0120] In formula (2), represents the inertial residual term; represents the rotational residual term; represents the velocity residual term; represents the displacement residual term; R i represents the position information of the camera at key frame i; represents R i the transpose of; R i+1 represents the position information of the camera at key frame i + 1; ΔR i，i+1 represents the rotational change value; represents ΔR i，i+1 the transpose of; v i represents the velocity of the camera at key frame i; v i+1 represents the velocity of the camera at key frame i + 1; Δv i，i+1 represents the velocity change value; p i represents the attitude information of the camera at key frame i; p i+1 represents the attitude information of the camera at key frame i + 1; Δp i，i+1 represents the displacement change value; Log represents the logarithmic mapping on SO(3); used to map the Lie group to the vector space; g represents the gravitational acceleration; Δt i，i+1 represents the time interval between two adjacent key frames i and i + 1; p j represents the attitude information of the camera at key frame j.

[0121] Then, the state of the VIO system is estimated using the visual sensor data. ORB feature extraction is performed on the images collected by the visual sensor, and the obtained ORB feature points are also called ORB landmarks; the three-dimensional coordinates of each ORB landmark are calculated according to the parallax principle; according to the projection of the landmark in key frame i and the position x of ORB landmark j j between the reprojection error r ij the visual residual term is obtained, as shown in formula (3).

[0122]

[0123] In formula (3), r ij represents the visual residual term, u i，j represents the observation of ORB landmark j in image frame i, x j represents the position of ORB landmark j, Π represents the projection function, T CB represents the transformation between the IMU coordinate system and the visual sensor coordinate system, T CB ∈SE(3), T i =[R i , p i ∈SE(3), Represents the mapping from the SE(3) group to three-dimensional space.

[0124] Combined with the inertial residual term and the visual residual term r ij , a least-squares problem based on key frames is used to optimize the state of the VIO system. Given k + 1 key frames, the state estimates at each key frame l ORB landmarks that can appear in these key frames, and the corresponding coordinates X = {x0...x l-1} of the ORB landmarks are used to optimize the state of the VIO system. The optimization problem can be shown as in Equation (4).

[0125]

[0126] In Equation (4), represents the inertial residual term, r ij represents the visual residual term, K j is the set of key frames of the ORB landmark j, l represents the number of ORB landmarks j, and ρ Hub represents the Huber kernel of the loss function. For the reprojection error, the Huber kernel is used to reduce the influence of incorrect matches.

[0127] Finally, the optimized state estimate value is obtained. Among them, the historical waist movement trajectory τ can be represented by the state p of the VIO system i as shown in Equation (5).

[0128] τ = [p i (5)

[0129] It can be understood that the visual-inertial odometry system used can be robustly used for measuring the historical waist movement trajectory under daily walking terrains.

[0130] In a possible implementation, the waist movement trajectory prediction model is obtained in the following way:

[0131] Obtain a training set; where the training set includes multiple sample waist movement trajectories and the prediction results of multiple sample waist movement trajectories;

[0132] Construct an initial waist movement trajectory prediction model;

[0133] Use multiple sample waist movement trajectories and the prediction results of multiple sample waist movement trajectories to train the initial waist movement trajectory prediction model to obtain the waist movement trajectory prediction model.

[0134] It should be noted that in this embodiment, before using the waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking while wearing the wearable walking assistance device, it is necessary to train the constructed initial waist movement trajectory prediction model. As Figure 2 shown, Figure 2 FIG. Figure 2 is a schematic structural diagram of a preset terrain feature extraction model and a waist movement trajectory prediction model provided in an embodiment of the present application. The initial waist movement trajectory prediction model is a newly constructed deep learning model that has not undergone model training. The training set is a subset of data used to train the model, and this data contains known input features and expected output labels, that is, the sample waist movement trajectory and the prediction result of the sample waist movement trajectory. The sample waist movement trajectory is the waist movement trajectory measured when the wearer walks while wearing the wearable walking assistance device. The prediction result of the sample waist movement trajectory is the prediction result output by the model.

[0135] It can be understood that by training the initial waist movement trajectory prediction model, the model can accurately predict new data and accurately predict the waist movement trajectory of the wearer to achieve guiding the assistance of the wearable walking assistance device.

[0136] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0137] Corresponding to the waist movement trajectory prediction method in the above embodiment, Figure 5 FIG. Figure 5 shows a schematic structural diagram of a waist movement trajectory prediction device provided in an embodiment of the present application. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown.

[0138] Referring to Figure 5 , the waist movement trajectory prediction device 3 in this embodiment includes:

[0139] An acquisition module 31, configured to acquire the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and acquire the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;

[0140] An encoding module 32, configured to perform feature extraction on the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;

[0141] An optimization module 33, configured to embed the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory;

[0142] The prediction module 34 is configured to input the optimized historical waist movement trajectory into the waist movement trajectory prediction model, predict the waist movement trajectory of the wearer when walking on the second terrain, and generate the target waist movement trajectory corresponding to the wearer when walking on the second terrain.

[0143] It can be understood that for the waist movement trajectory prediction device 3 provided in the embodiments of the present application, first, the acquisition module 31 acquires the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and acquires the environmental point cloud data of the second terrain that the wearer will walk on in the future; then, the encoding module 32 extracts features from the environmental point cloud data of the second terrain to obtain the terrain feature data corresponding to the second terrain; the optimization module 33 embeds the terrain feature data into the historical waist movement trajectory to generate the optimized historical waist movement trajectory corresponding to the historical waist movement trajectory; finally, the prediction module 34 inputs the optimized historical waist movement trajectory into the waist movement trajectory prediction model, predicts the waist movement trajectory of the wearer when walking on the second terrain, and generates the target waist movement trajectory corresponding to the wearer when walking on the second terrain. The present application solves the problem in the prior art that only the waist movement trajectory of the wearer wearing a wearable walking assistance device is measured, but the prediction of the waist movement trajectory of the wearer is lacking, resulting in the inability to effectively guide the wearable walking assistance device to assist the wearer in walking and the inability to obtain the movement intention of the wearer when walking. Thus, it realizes the rapid and effective prediction of the waist movement trajectory of the wearer wearing a wearable walking assistance device under daily multi-terrain walking.

[0144] It should be noted that for the information interaction, execution process, etc. between the modules in the above-mentioned waist movement trajectory prediction device 3, since it is based on the same concept as the method embodiment of the present application, the specific functions and the technical effects brought by them can be specifically referred to in the method embodiment part, and will not be elaborated here.

[0145] Further, the encoding module 32 includes:

[0146] The binarization sub-module is configured to perform two-dimensional projection on all the point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data;

[0147] The terrain feature extraction sub-module is configured to extract the terrain features in the binary image through a preset terrain feature extraction model to obtain the initial terrain feature data corresponding to the second terrain;

[0148] The terrain feature compensation sub-module is configured to compensate the initial terrain feature data to obtain the terrain feature data.

[0149] Further, the terrain feature compensation sub-module includes:

[0150] The first binarization unit is used to encode the terrain feature data corresponding to the current moment to obtain a first binary image;

[0151] The backtracking unit is used to obtain the backtracking moment corresponding to the backtracking distance through the visual inertial odometry system; wherein, the backtracking distance is used to represent the missing terrain feature data in the first binary image;

[0152] The second binarization unit is used to encode the missing terrain feature data corresponding to the backtracking moment to obtain a second binary image;

[0153] The missing pixel extraction unit is used to extract the pixels corresponding to the backtracking distance in the second binary image according to the backtracking distance to obtain missing pixels;

[0154] The optimization unit is used to splice the missing pixels into the first binary image to obtain an optimized first binary image;

[0155] The terrain feature data determination unit is used to obtain terrain feature data based on the optimized first binary image.

[0156] Further, the optimization module 33 includes:

[0157] The interpolation processing unit is used to use the X-axis historical waist movement trajectory as a reference to perform interpolation processing on the Y-axis historical waist movement trajectory and the Z-axis historical waist movement trajectory respectively to obtain the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory;

[0158] The filtering processing unit is used to perform filtering processing on the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory respectively to obtain an optimized X-Y axis historical waist movement trajectory and an optimized X-Z axis historical waist movement trajectory;

[0159] The feature embedding unit is used to add the terrain feature data to the ends of the optimized X-Y axis historical waist movement trajectory and the optimized X-Z axis historical waist movement trajectory respectively to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory.

[0160] Further, the acquisition module 31 includes:

[0161] The initial environmental point cloud data acquisition unit is used to acquire the initial environmental point cloud data of the second terrain that the wearer will walk on in the future through a visual sensor fixed on a predetermined body part of the wearer;

[0162] The environmental point cloud data acquisition unit is used to preprocess the initial environmental point cloud data to obtain the environmental point cloud data of the second terrain that the wearer will walk on in the future; wherein, the preprocessing includes at least one of the following: passing through filtering, rotation processing.

[0163] Further, the acquisition module 31 further includes:

[0164] A motion information and camera pose acquisition unit, configured to acquire the walking motion information data and camera pose data of the wearer when walking on the first terrain through an inertial measurement unit connected to the vision sensor;

[0165] A historical waist motion trajectory acquisition unit, configured to calculate the camera motion trajectory corresponding to the vision sensor according to the walking motion information data and camera pose data, and obtain the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment.

[0166] Further, the waist motion trajectory prediction device 3 includes:

[0167] A training set acquisition module, configured to acquire a training set; wherein, the training set includes multiple sample waist motion trajectories and prediction results of multiple sample waist motion trajectories;

[0168] An initial prediction model construction module, configured to construct an initial waist motion trajectory prediction model;

[0169] A prediction model training module, configured to train the initial waist motion trajectory prediction model using multiple sample waist motion trajectories and prediction results of multiple sample waist motion trajectories to obtain a waist motion trajectory prediction model.

[0170] The embodiment of the present application further provides a waist motion trajectory prediction device, as Figure 6 shown, Figure 6 is a schematic structural diagram of a waist motion trajectory prediction device provided by an embodiment of the present application. Referring to Figure 6 , the waist motion trajectory prediction device 4 of this embodiment includes: a memory 41, a processor 42, and a computer program stored in the memory 41 and executable on the processor 42. When the processor 42 executes the computer program, the steps in any of the above-mentioned waist motion trajectory prediction method embodiments are implemented.

[0171] The embodiment of the present application further provides a wearable walking assistance device, as Figure 7 shown, Figure 7 is a schematic structural diagram of a wearable walking assistance device provided by an embodiment of the present application. Referring to Figure 7 , the wearable walking assistance device 5 includes: a controller 51, and the controller 51 is configured to execute the steps in any of the above-mentioned waist motion trajectory prediction method embodiments.

[0172] The embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0173] The embodiments of the present application also provide a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can be enabled to execute the steps in the above-mentioned method embodiments.

[0174] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned method embodiments of the present application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0175] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0176] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0177] In the embodiments provided in the present application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0178] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for predicting the waist movement trajectory, characterized in that Including: Obtain the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and obtain the environmental point cloud data of the second terrain that the wearer will walk on at a future moment through a visual sensor fixed to the waist of the wearer; wherein, the historical waist movement trajectory is used to characterize the position and velocity movement information of the wearer's waist in three-dimensional space recorded by an inertial measurement unit when the wearer walks on the first terrain at a past historical moment; Extract features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; Embed the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory; Input the optimized historical waist movement trajectory into a waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking on the second terrain, and generate a target waist movement trajectory corresponding to the wearer when walking on the second terrain.

2. The waist movement trajectory prediction method according to claim 1, characterized in that The extracting features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain includes: Perform two-dimensional projection on all point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data; Extract terrain features in the binary image through a preset terrain feature extraction model to obtain initial terrain feature data corresponding to the second terrain; Compensate the initial terrain feature data to obtain the terrain feature data.

3. The waist movement trajectory prediction method according to claim 2, characterized in that, The compensating the initial terrain feature data to obtain the terrain feature data includes: Encode the terrain feature data corresponding to the current moment to obtain a first binary image; Obtain a retrospective moment corresponding to a retrospective distance through a visual inertial odometry system; wherein, the retrospective distance is used to characterize the missing terrain feature data in the first binary image; Encode the missing terrain feature data corresponding to the retrospective moment to obtain a second binary image; Extract pixels corresponding to the retrospective distance in the second binary image according to the retrospective distance to obtain missing pixels; Splice the missing pixels into the first binary image to obtain an optimized first binary image; Based on the optimized first binary image, obtain the terrain feature data.

4. The lumbar motion trajectory prediction method according to any one of claims 1-3, characterized in that, The historical waist movement trajectory includes: an X-axis historical waist movement trajectory, a Y-axis historical waist movement trajectory, and a Z-axis historical waist movement trajectory; The embedding the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory includes: Taking the X-axis historical waist movement trajectory as a reference, perform interpolation processing on the Y-axis historical waist movement trajectory and the Z-axis historical waist movement trajectory respectively to obtain an X-Y axis historical waist movement trajectory and an X-Z axis historical waist movement trajectory; Perform filtering processing on the X-Y axis historical waist movement trajectory and the X-Z axis historical waist movement trajectory respectively to obtain an optimized X-Y axis historical waist movement trajectory and an optimized X-Z axis historical waist movement trajectory; Add the terrain feature data to the end of the optimized X-Y axis historical waist movement trajectory and the optimized X-Z axis historical waist movement trajectory respectively to generate the optimized historical waist movement trajectory corresponding to the historical waist movement trajectory.

5. The waist movement trajectory prediction method according to claim 1, wherein The environmental point cloud data of the second terrain that the wearer will walk on at a future moment is obtained by a visual sensor fixed to the waist of the wearer, including: Obtain the initial environmental point cloud data of the second terrain that the wearer will walk on at a future moment through a visual sensor fixed to the waist of the wearer; Preprocess the initial environmental point cloud data to obtain the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; wherein, the preprocessing includes at least one of the following: straight-through filtering, rotation processing.

6. The waist movement trajectory prediction method according to claim 5, wherein The obtaining of the historical waist movement trajectory of the wearer walking on the first terrain at a historical moment includes: Obtain the walking motion information data and camera pose data of the wearer walking on the first terrain through an inertial measurement unit connected to the visual sensor; Calculate the camera motion trajectory corresponding to the visual sensor according to the walking motion information data and the camera pose data to obtain the historical waist movement trajectory of the wearer walking on the first terrain at a historical moment.

7. The waist movement trajectory prediction method according to claim 1, wherein The waist movement trajectory prediction model is obtained by the following method: Obtain a training set; wherein, the training set includes multiple sample waist movement trajectories and prediction results of multiple sample waist movement trajectories; Construct an initial waist movement trajectory prediction model; Train the initial waist movement trajectory prediction model with multiple sample waist movement trajectories and prediction results of multiple sample waist movement trajectories to obtain the waist movement trajectory prediction model.

8. A lumbar motion trajectory prediction device, characterized in that Including: An acquisition module, configured to acquire the historical waist movement trajectory of the wearer walking on the first terrain at a historical moment, and the environmental point cloud data of the second terrain that the wearer will walk on at a future moment through a visual sensor fixed to the waist of the wearer; wherein, the historical waist movement trajectory is used to represent the position and velocity motion information of the wearer's waist in three-dimensional space recorded by the inertial measurement unit when the wearer walked on the first terrain in the past historical moment. An encoding module, configured to extract features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; An optimization module, configured to embed the terrain feature data into the historical waist movement trajectory to generate an optimized historical waist movement trajectory corresponding to the historical waist movement trajectory; A prediction module, configured to input the optimized historical waist movement trajectory into a waist movement trajectory prediction model to predict the waist movement trajectory of the wearer when walking on the second terrain, and generate a target waist movement trajectory corresponding to the wearer when walking on the second terrain.

9. A waist movement trajectory prediction device, characterized in that, Comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method according to any one of claims 1 to 7.

10. A wearable walking assistance device, characterized in that, Comprising: A controller configured to execute the method according to any one of claims 1 to 7.

Images