Waist motion trail prediction method, device and equipment and wearable walking aid equipment
By extracting topographic features and embedding historical waist motion trajectories for prediction, the problem of unpredictable waist motion trajectory in the prior art is solved, and the rapid and effective prediction and assisted walking effect of the wearer's waist motion trajectory is achieved.
Patent Information
- Application Number
- CN202411205275.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-29
AI Technical Summary
Existing wearable walking devices cannot effectively predict waist movement trajectory, resulting in the inability to accurately assist the wearer in walking and the inability to obtain the intention to walk.
By obtaining the wearer's historical waist motion trajectory and environmental point cloud data of the future terrain, terrain feature data are extracted, and the optimization trajectory is embedded in the historical waist motion trajectory to generate the optimization trajectory, and input the waist motion trajectory prediction model for prediction to generate the target waist motion trajectory.
It realizes rapid and effective prediction of the wearer's waist movement trajectory in multi-terrain environments, improves the auxiliary effect of wearable walking equipment, and obtains walking movement intentions.
Smart Images

Figure CN119091505A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of robot control technology, and in particular relates to a waist motion trajectory prediction method, device, equipment and wearable walking aid. Background Art
[0002] Wearable walking aids such as exoskeletons and powered prostheses use wearable mechanical structures and high-tech means such as sensors, control systems, and power devices to provide the wearer with functions such as assisted walking, weight-bearing, and rehabilitation training, thereby enhancing or restoring the body's motor ability.
[0003] Since the movement of the human body's center of mass comprehensively reflects the coordinated movement of the joints of the lower limbs, it not only represents the coordination and stability of walking, but also reflects the movement intention of the human body's walking. At the same time, the human body's center of mass is at the waist position, so the waist movement trajectory can reflect the movement of the human body's center of mass.
[0004] At present, advanced wearable walking aids have begun to consider the movement of the wearer's center of mass, so that wearable walking aids can provide more natural assistance to the wearer, especially when transitioning between various movements. However, most researchers currently measure the wearer's waist movement trajectory through two methods: external and wearable. However, there is a lack of prediction of the waist movement trajectory, which results in the wearable walking aid being unable to effectively assist the wearer in walking and unable to obtain the wearer's walking movement intention. Summary of the invention
[0005] The embodiments of the present application provide a waist motion trajectory prediction method, device, equipment and wearable walking aid device, which can solve the problem that in the prior art, only the waist motion trajectory of a wearer wearing a wearable walking aid device is measured but the waist motion trajectory of the wearer is not predicted, resulting in the inability to effectively guide the wearable walking aid device to assist the wearer in walking and the inability to obtain the wearer's walking movement intention.
[0006] In a first aspect, an embodiment of the present application provides a waist motion trajectory prediction method, the method comprising:
[0007] Acquire the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment, and acquire the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;
[0008] Performing feature extraction on the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;
[0009] Embedding the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory;
[0010] The optimized historical waist motion trajectory is input into a waist motion trajectory prediction model, the waist motion trajectory of the wearer when walking on the second terrain is predicted, and a target waist motion trajectory corresponding to the wearer when walking on the second terrain is generated.
[0011] In a possible implementation manner of the first aspect, extracting features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain includes:
[0012] Performing two-dimensional projection on all point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data;
[0013] Extracting terrain features from the binary image using a preset terrain feature extraction model to obtain initial terrain feature data corresponding to the second terrain;
[0014] The initial terrain feature data is compensated to obtain the terrain feature data.
[0015] In a possible implementation manner of the first aspect, compensating the initial terrain feature data to obtain the terrain feature data includes:
[0016] Encoding the terrain feature data corresponding to the current moment to obtain a first binary image;
[0017] Obtaining a backtracking time corresponding to a backtracking distance through a visual inertial odometry system; wherein the backtracking distance is used to characterize the terrain feature data missing in the first binary image;
[0018] Encoding the missing terrain feature data corresponding to the backtracking time to obtain a second binary image;
[0019] According to the backtracking distance, extract pixels corresponding to the backtracking distance in the second binary image to obtain missing pixels;
[0020] splicing the missing pixels into the first binary image to obtain an optimized first binary image;
[0021] The terrain feature data is obtained based on the optimized first binary image.
[0022] In a possible implementation of the first aspect, the historical waist motion trajectory includes: an X-axis historical waist motion trajectory, a Y-axis historical waist motion trajectory, and a Z-axis historical waist motion trajectory;
[0023] The embedding of the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory includes:
[0024] Taking the X-axis historical waist motion trajectory as a reference, interpolation processing is performed on the Y-axis historical waist motion trajectory and the Z-axis historical waist motion trajectory respectively to obtain the XY-axis historical waist motion trajectory and the XZ-axis historical waist motion trajectory;
[0025] Performing filtering processing on the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory respectively to obtain an optimized XY axis historical waist motion trajectory and an optimized XZ axis historical waist motion trajectory;
[0026] The terrain feature data are added to the end of the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory respectively to generate the optimized historical waist motion trajectory corresponding to the historical waist motion trajectory.
[0027] In a possible implementation manner of the first aspect, obtaining environmental point cloud data of a second terrain on which the wearer will walk at a future moment includes:
[0028] Acquiring initial environmental point cloud data of the second terrain that the wearer will walk on at a future moment through a visual sensor fixed to a predetermined body part of the wearer;
[0029] The initial environmental point cloud data is preprocessed to obtain the environmental point cloud data of the second terrain on which the wearer will walk at a future moment; wherein the preprocessing includes at least one of the following: through filtering and rotation processing.
[0030] In a possible implementation of the first aspect, obtaining a historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment includes:
[0031] Acquiring walking motion information data and camera position data of the wearer when walking on the first terrain through an inertial measurement unit connected to the visual sensor;
[0032] The camera motion trajectory corresponding to the visual sensor is calculated according to the walking motion information data and the camera posture data to obtain the historical waist motion trajectory of the wearer when walking on the first terrain at the historical moment.
[0033] In a possible implementation manner of the first aspect, the waist motion trajectory prediction model is obtained in the following manner:
[0034] Acquire a training set; wherein the training set includes a plurality of sample waist motion trajectories and a plurality of prediction results of the sample waist motion trajectories;
[0035] Construct an initial waist motion trajectory prediction model;
[0036] The initial waist motion trajectory prediction model is trained using a plurality of the sample waist motion trajectories and a plurality of prediction results of the sample waist motion trajectories to obtain the waist motion trajectory prediction model.
[0037] In a second aspect, an embodiment of the present application provides a waist motion trajectory prediction device, the device comprising:
[0038] An acquisition module, used to acquire the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment, and to acquire the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;
[0039] An encoding module, used for performing feature extraction on the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;
[0040] An optimization module, used for embedding the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory;
[0041] A prediction module is used to input the optimized historical waist motion trajectory into a waist motion trajectory prediction model, predict the waist motion trajectory of the wearer when he is about to walk on the second terrain, and generate a target waist motion trajectory corresponding to the wearer when he is about to walk on the second terrain.
[0042] In a third aspect, an embodiment of the present application provides a waist motion trajectory prediction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the waist motion trajectory prediction methods described above when executing the computer program.
[0043] In a fourth aspect, an embodiment of the present application provides a wearable walking aid device, comprising a controller, wherein the controller is used to execute any of the waist motion trajectory prediction methods described above.
[0044] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the waist motion trajectory prediction method described in any one of the above items is implemented.
[0045] In a sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any of the waist motion trajectory prediction methods described above.
[0046] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0047] The waist motion trajectory prediction method provided by the embodiment of the present application obtains the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment, and obtains the environmental point cloud data of the second terrain that the wearer will walk on in the future; extracts features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; then, embeds the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory; finally, inputs the optimized historical waist motion trajectory into the waist motion trajectory prediction model, predicts the waist motion trajectory of the wearer when he is about to walk on the second terrain, and generates a target waist motion trajectory corresponding to the wearer when he is about to walk on the second terrain. The present application solves the problem that the waist motion trajectory of the wearer wearing the wearable walking aid device is only measured in the prior art, but the waist motion trajectory of the wearer is not predicted, resulting in the inability to effectively guide the wearable walking aid device to assist the wearer in walking and the inability to obtain the wearer's walking intention, thereby realizing the rapid and effective prediction of the waist motion trajectory of the wearer wearing the wearable walking aid device under daily multi-terrain walking. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0049] Figure 1 It is a flowchart of a waist motion trajectory prediction method provided by an embodiment of the present application;
[0050] Figure 2 It is a structural schematic diagram of a preset terrain feature extraction model and a waist motion trajectory prediction model provided in one embodiment of the present application.
[0051] Figure 3 is a flow chart of a waist motion trajectory prediction method provided by another embodiment of the present application;
[0052] Figure 4 is a schematic diagram of a terrain feature compensation method provided in an embodiment of the present application;
[0053] Figure 5 It is a structural schematic diagram of a waist motion trajectory prediction device provided by an embodiment of the present application;
[0054] Figure 6 It is a structural schematic diagram of a waist motion trajectory prediction device provided by an embodiment of the present application;
[0055] Figure 7 It is a schematic diagram of the structure of a wearable walking aid device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0056] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0057] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0058] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0059] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0060] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0061] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0062] See also Figure 1 , Figure 1 : is a flow chart of a waist motion trajectory prediction method provided by an embodiment of the present application. The waist motion trajectory prediction method includes:
[0063] S11, obtaining the historical waist movement trajectory of the wearer when walking on the first terrain at a historical moment, and obtaining the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;
[0064] S12, extracting features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;
[0065] S13, embedding the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory;
[0066] S14, inputting the optimized historical waist motion trajectory into the waist motion trajectory prediction model, predicting the waist motion trajectory of the wearer when he is about to walk on the second terrain, and generating a target waist motion trajectory corresponding to the wearer when he is about to walk on the second terrain.
[0067] It should be noted that, in this embodiment, the execution subject may be a terminal device such as a server, and no specific limitation is made to this.
[0068] In step S11, the wearer refers to a user who walks while wearing a wearable walking aid. A wearable walking aid refers to a device that can be worn on the human body to assist or enhance walking ability, such as a lower limb exoskeleton robot, an intelligent walking aid robot, etc. The first terrain refers to the terrain that the wearer has walked on, and the waist motion trajectory generated by the wearer when walking on the first terrain has been recorded, that is, the historical waist motion trajectory has been recorded. The historical waist motion trajectory refers to the waist motion trajectory or path data of the wearer when walking on the first terrain at a historical moment in the past. Usually, the historical waist motion trajectory can record the position, speed and other motion information of the wearer's waist in three-dimensional space through an inertial measurement unit (such as a gyroscope, an accelerometer). The historical waist motion trajectory provides the wearer's movement pattern and habits when walking on similar terrain. The second terrain refers to the terrain that the wearer will walk in front of during the walking process. The environmental point cloud data is the three-dimensional space information of the second terrain obtained by a visual sensor (such as a depth camera). The environmental point cloud data is usually represented by a point cloud, each point cloud contains its coordinate information in three-dimensional space, and these point clouds constitute a three-dimensional model of the second terrain. The environmental point cloud data of the second terrain is used to analyze and predict the waist movement trajectory of the wearer on the second terrain.
[0069] In step S12, the terrain feature data is the key feature information of the terrain extracted from the environmental point cloud data of the second terrain, such as slope, terrain undulation, obstacle position, etc. The terrain feature data is used to describe the physical characteristics and geometric structure of the second terrain, which is convenient for subsequent analysis and prediction.
[0070] In step S13, optimizing the historical waist motion trajectory is to combine the terrain feature data with the historical waist motion trajectory to obtain an optimized waist trajectory, which is an adjustment and optimization of the historical waist motion trajectory. Since different terrains may affect the wearer's walking style and habits, by combining the terrain feature data with the historical waist motion trajectory, the prediction of the waist motion trajectory can be made more accurate and reliable.
[0071] In step S14, the waist motion trajectory prediction model is a deep learning model that is pre-built and trained, and is used to predict the waist motion trajectory of the wearer when walking on the second terrain, that is, the target waist motion trajectory, based on the input optimized historical waist motion trajectory. The waist motion trajectory prediction model learns the relationship between terrain features and waist motion trajectories by training a large amount of data. The target waist motion trajectory can help the wearer understand and adapt to the changes in the second terrain in advance, so that he can walk more safely and comfortably. At the same time, it can also be used to optimize the design and control strategy of wearable walking aids to provide better assistance effects.
[0072] It can be understood that the waist motion trajectory prediction method provided in the embodiment of the present application obtains the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment, and obtains the environmental point cloud data of the second terrain that the wearer will walk on in the future; extracts features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; then, embeds the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory; finally, inputs the optimized historical waist motion trajectory into the waist motion trajectory prediction model, predicts the waist motion trajectory of the wearer when he is about to walk on the second terrain, and generates a target waist motion trajectory corresponding to the wearer when he is about to walk on the second terrain. The present application solves the problem that the waist motion trajectory of the wearer wearing the wearable walking aid device is only measured in the prior art, but the waist motion trajectory of the wearer is not predicted, resulting in the inability to effectively guide the wearable walking aid device to assist the wearer in walking and the inability to obtain the wearer's walking intention, thereby realizing the rapid and effective prediction of the waist motion trajectory of the wearer wearing the wearable walking aid device under daily multi-terrain walking.
[0073] In a possible implementation, feature extraction is performed on the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain, including:
[0074] Perform two-dimensional projection on all point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data;
[0075] The terrain features in the binary image are extracted by using a preset terrain feature extraction model to obtain initial terrain feature data corresponding to the second terrain;
[0076] The initial terrain feature data is compensated to obtain terrain feature data.
[0077] It should be noted that a binary image is an image representation method in which each pixel in the image has only two possible values, usually 0 and 1 (or black and white). In a binary image, all pixels are divided into two categories, usually used to represent the target and background in the image. Since the environmental point cloud data is three-dimensional and contains the coordinates of each point cloud in three-dimensional space, the environmental point cloud data is projected two-dimensionally to obtain a binary image, which is used to simplify and highlight the terrain features of the second terrain in front, facilitating the subsequent extraction of terrain features.
[0078] The preset terrain feature extraction model is a pre-built image processing model that has been trained to identify and analyze terrain features in binary images. Figure 2 As shown, Figure 2The present invention provides a schematic diagram of a preset terrain feature extraction model and a waist motion trajectory prediction model provided in an embodiment of the present invention. The preset terrain feature extraction model is used to extract relevant terrain features from the binary image, and its types are classified and labeled, such as flat ground, stairs, slope, stairs, slope, etc., to form initial terrain feature data related to the second terrain.
[0079] The initial terrain feature data is the original terrain feature data extracted directly from the binary image and has not been processed. It may contain some noise or errors and needs to be further processed or optimized.
[0080] like Figure 3 As shown, Figure 3 It is a flow chart of a waist motion trajectory prediction method provided by another embodiment of the present application. In the process of acquiring environmental point cloud data, due to the limitation of the visual sensor's viewing angle, the visual sensor will inevitably lose environmental information within a certain range in front of the visual sensor while acquiring environmental information far ahead, that is, acquiring environmental point cloud data of the second terrain. This interferes with the calculation of the terrain feature data of the walking environment. Therefore, it is necessary to compensate the current environmental feature information to obtain terrain feature data containing complete information, that is, to obtain terrain feature data by compensating the initial terrain feature data. Among them, the terrain feature data is the data obtained after the initial terrain feature data is optimized by compensation, etc. These data are more accurate and reliable, and can fully reflect the various characteristics of the second terrain. Compensation is the process of optimizing the initial terrain feature data, such as filtering, interpolation, denoising, etc., to eliminate or reduce the error or noise therein, so as to improve the accuracy and reliability of the terrain feature data.
[0081] In a possible implementation, compensating the initial terrain feature data to obtain the terrain feature data includes:
[0082] S21, encoding the terrain feature data corresponding to the current moment to obtain a first binary image;
[0083] S22, obtaining a backtracking time corresponding to the backtracking distance through a visual inertial odometry system; wherein the backtracking distance is used to characterize the terrain feature data missing in the first binary image;
[0084] S23, encoding the missing terrain feature data corresponding to the retrospective time to obtain a second binary image;
[0085] S24, extracting pixels corresponding to the tracing distance in the second binary image according to the tracing distance to obtain missing pixels;
[0086] S25, splicing the missing pixels into the first binary image to obtain an optimized first binary image;
[0087] S26. Obtaining terrain feature data based on optimizing the first binary image.
[0088] It should be noted that, in step S21, the first binary image is an image obtained by encoding the terrain feature data at the current moment. The first binary image can simplify the representation of the terrain feature data, facilitating subsequent data processing. Figure 4 As shown, Figure 4 FIG. 1 is a schematic diagram of a terrain feature compensation method provided by an embodiment of the present application. Figure 4 As shown in the image t Represents the first binary image at the current time t.
[0089] In step S22, the Visual-Inertial Odometry (VIO) system is a mileage calculation system that combines visual sensors (such as depth cameras) and inertial measurement units (such as gyroscopes and accelerometers) to estimate the motion trajectory and posture of the device. Visual sensors can provide rich environmental information, but are easily affected by factors such as lighting and occlusion; inertial measurement units can provide high-frequency inertial data, but the accumulated error will gradually increase over time. Therefore, the fusion of visual sensors and inertial measurement units can complement each other's advantages to improve the robustness and accuracy of the VIO system.
[0090] In this embodiment, the backtracking time corresponding to the backtracking distance is determined by the information of the VIO system, that is, based on the historical motion trajectory of the device and the environmental point cloud data, backtracking to a certain time point in the past. Figure 4 As shown in , d represents the backtracking distance, t-Δt represents the backtracking time, and the backtracking distance d corresponds to the first binary image image t The backtracking distance is the spatial distance from the current moment to the backtracking moment, which is used to characterize the range or degree of terrain feature data that may be missing in the first binary image. The backtracking moment is a time point in the past determined by the VIO system, and the terrain feature data corresponding to the time point will be used to supplement the terrain feature data missing in the first binary image at the current moment.
[0091] In step S23, the missing terrain feature data corresponding to the backtracking time is encoded to obtain a second binary image. The second binary image is an image obtained by finding the terrain feature data corresponding to the backtracking time through the backtracking mechanism and encoding the terrain feature data corresponding to the backtracking time. The second binary image is used to supplement the terrain feature data that may be missing in the first binary image. Figure 4 As shown in the imaget-Δt Represents the second binary image at the retrospective time t-Δt.
[0092] It should be noted that, since the pixels of the binary image have a linear relationship with the walking distance, the backtracking distance can be calculated according to the missing pixel width. In step S24, according to the backtracking distance, the pixels corresponding to the backtracking distance are extracted in the second binary image to obtain the missing pixels. The missing pixels refer to the pixels extracted from the second binary image corresponding to the terrain feature data missing from the first binary image.
[0093] In steps S25 and S26, the extracted missing pixels are spliced into the first binary image to obtain an optimized first binary image; thus, terrain feature data is obtained based on the optimized first binary image. Figure 4 As shown in , result represents the optimization of the first binary image. The optimized first binary image contains missing pixels of the missing terrain feature data, and contains more complete and accurate terrain feature data compared with the first binary image. So far, the compensation process for the initial terrain feature data is completed, and terrain feature data with complete information is obtained, thereby improving the accuracy of terrain feature extraction and classification.
[0094] In a possible implementation, the historical waist motion trajectory includes: an X-axis historical waist motion trajectory, a Y-axis historical waist motion trajectory, and a Z-axis historical waist motion trajectory;
[0095] The terrain feature data is embedded into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory, including:
[0096] Taking the X-axis historical waist motion trajectory as a reference, interpolation processing is performed on the Y-axis historical waist motion trajectory and the Z-axis historical waist motion trajectory to obtain the XY-axis historical waist motion trajectory and the XZ-axis historical waist motion trajectory;
[0097] Perform filtering processing on the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory respectively to obtain the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory;
[0098] The terrain feature data are added to the end of the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory respectively to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory.
[0099] It should be noted that, in the world coordinate system, the historical waist motion trajectory includes motion trajectories in three axis directions, namely, the X-axis historical waist motion trajectory, the Y-axis historical waist motion trajectory, and the Z-axis historical waist motion trajectory. Among them, the X-axis historical waist motion trajectory indicates the distance the wearer's waist moves forward or backward during walking; the Y-axis historical waist motion trajectory indicates the distance the wearer's waist tilts during walking; and the Z-axis historical waist motion trajectory indicates the distance the wearer's waist height changes during walking.
[0100] In this embodiment, first, the historical waist motion trajectory of the X-axis is used as a reference to interpolate the historical waist motion trajectory of the Y-axis to obtain the historical waist motion trajectory of the XY-axis; and, the historical waist motion trajectory of the X-axis is used as a reference to interpolate the historical waist motion trajectory of the YZ-axis to obtain the historical waist motion trajectory of the XZ-axis. Among them, interpolation processing is a method of estimating unknown data points through known data points. That is, according to each point on the X-axis, the corresponding point is found on the Y-axis and the Z-axis, so as to obtain the historical waist motion trajectory of the XY-axis and the historical waist motion trajectory of the XZ-axis, that is, the three-dimensional running trajectory is converted into two two-dimensional motion trajectories, so that the motion trajectories of the three axes are synchronized in time, which is convenient for subsequent analysis.
[0101] Secondly, after obtaining the two motion trajectories of the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory, in order to eliminate the burrs in the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory, it is necessary to filter the two motion trajectories of the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory, so as to obtain the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory. Among them, filtering is a signal processing used to remove noise or unnecessary frequency components in the signal. The purpose of filtering the two motion trajectories of the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory is to smooth the motion trajectory, reduce noise, burrs, etc. caused by external interference, and make the historical waist motion trajectory more accurate and reliable.
[0102] Then, after obtaining the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory, the terrain feature data extracted by the preset terrain feature extraction model are added to the end of the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory, respectively, that is, the terrain feature-related data is combined with the historical waist motion trajectory to generate an optimized historical waist motion trajectory containing terrain information. Among them, the optimized historical waist motion trajectory is a more comprehensive and accurate motion trajectory that has been processed by interpolation and filtering and incorporates terrain feature data. It not only reflects the historical movement of the wearer's waist, but also takes into account the influence of the terrain, making it more suitable for motion analysis and optimization under complex terrain conditions.
[0103] In a possible implementation, obtaining environmental point cloud data of a second terrain on which the wearer will walk at a future moment includes:
[0104] Acquire initial environmental point cloud data of a second terrain that the wearer will walk on at a future moment through a visual sensor fixed to a predetermined body part of the wearer;
[0105] The initial environmental point cloud data is preprocessed to obtain environmental point cloud data of a second terrain that the wearer will walk on in the future; wherein the preprocessing includes at least one of the following: through filtering and rotation processing.
[0106] It should be noted that a visual sensor is a sensor that uses optical elements and imaging devices to obtain two-dimensional image information and three-dimensional depth information of objects in an external scene, such as a depth camera. Infrared light or laser technology is usually used to measure the distance between the sensor and each point in the scene, and generate point cloud data. Visual sensors are widely used in robot navigation, three-dimensional reconstruction and other fields. Among them, in this embodiment, the visual sensor is fixed to a predetermined body part of the wearer, and the predetermined body part can be the waist of the wearer. The initial environmental point cloud data is the unprocessed three-dimensional point cloud data about the second terrain directly obtained by the visual sensor. These data contain the three-dimensional coordinate information of all visible points in the scene, and may also contain noise, redundant data or deformation. The environmental point cloud data is the point cloud data with improved quality obtained after preprocessing the initial environmental point cloud data.
[0107] Preprocessing is a series of data processing operations performed on the initial environmental point cloud data, aiming to improve the data quality of the initial environmental point cloud data, remove noise, reduce redundancy, etc., facilitate the analysis and processing of subsequent data, and improve the efficiency and accuracy of data processing. Among them, preprocessing mainly includes through filtering and rotation processing. Through filtering, by setting a certain spatial range, only the point cloud data within the range is retained, and the point cloud data outside the range is eliminated. Through through filtering, background noise, unnecessary elements, or abnormal data caused by camera errors can be effectively removed. Rotation processing is to transform the point cloud data from the initial camera coordinate system to the world coordinate system by rotating the point cloud data, so as to make it easier to identify and extract ground feature data. Rotation processing can help eliminate the deformation of environmental point cloud data caused by changes in angle or wearer posture.
[0108] It can be understood that the initial environmental point cloud data of the second terrain that the wearer will walk on in the future is obtained by fixing the visual sensor on the predetermined body part of the wearer; and the initial environmental point cloud data is pre-processed by straight-through filtering, rotation processing, etc. to improve the data quality, and the final environmental point cloud data is obtained, so as to facilitate the subsequent data analysis and processing and improve the efficiency and accuracy of data processing.
[0109] In a possible implementation, obtaining the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment includes:
[0110] Acquire walking motion information data and camera position data of the wearer when walking on the first terrain through an inertial measurement unit connected to the visual sensor;
[0111] According to the walking motion information data and the camera posture data, the camera motion trajectory corresponding to the visual sensor is calculated to obtain the historical waist motion trajectory of the wearer when walking on the first terrain at the historical moment.
[0112] It should be noted that an inertial measurement unit (IMU) is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object, usually including a three-axis accelerometer and a three-axis gyroscope. The inertial measurement unit can measure the angular velocity and acceleration of an object in three-dimensional space, and use this to calculate the attitude of the object, and is often used in the fields of motion tracking and navigation. Walking motion information data is the motion-related data generated by the wearer during walking obtained by the inertial measurement unit, such as step frequency, step length, acceleration change, speed, etc., which is often used to analyze the wearer's movement patterns and habits. Camera pose data refers to the position information and pose information of the visual sensor in three-dimensional space, where the position information describes the coordinates of the visual sensor in three-dimensional space, and the pose information describes the orientation and rotation state of the visual sensor. Camera pose data can describe how the visual sensor moves relative to the surrounding environment, that is, it can describe how the wearer moves relative to the surrounding environment.
[0113] The camera motion trajectory refers to the motion trajectory formed by the visual sensor moving in a three-dimensional space along a certain path within a period of time. The world coordinate system takes the initial point as the origin, and the position and posture will not change with the movement of the person. Since the visual sensor is fixed to the preset fixed position of the human body, that is, the waist position, the position of the visual sensor can be approximately regarded as the waist position of the human body. Furthermore, the position change of the visual sensor coordinate system relative to the world coordinate system can be approximated as the waist motion trajectory of the human body. In other words, the camera motion trajectory can be regarded as the waist motion trajectory of the wearer.
[0114] It should be noted that if Figure 3 As shown, Figure 3 FIG. 2 is a flow chart of a waist motion trajectory prediction method provided by another embodiment of the present application. Figure 3 As shown in , the historical waist motion trajectory is obtained by performing a series of function processing, pre-integration, bundle adjustment and other processing on the motion information stream and image information stream obtained by the inertial measurement unit and the depth camera. Figure 3 middle, Indicates the historical waist movement trajectory, Represents the optimization of the historical waist motion trajectory. The process of calculating the camera motion trajectory corresponding to the visual sensor is the process of measuring the historical waist motion trajectory of the wearer when walking on the first terrain at the historical moment. In this embodiment, in order to accurately obtain the historical waist motion trajectory, a visual inertial mileage system is used for measurement. Among them, the visual information comes from the visual sensor, and the motion information comes from the inertial measurement unit (IMU). The visual sensor can make up for the inevitable system deviation of the inertial measurement unit, and the inertial measurement unit can make up for the low adaptability of the visual sensor in the case of rapid movement or drastic changes in motion state. The specific process is as follows:
[0115] First, the state of the visual inertial odometry system (VIO system) is defined. The state of the VIO system can be described by the attitude, position, velocity and IMU deviation of the IMU, as shown in formula (1).
[0116] S i =[R i , p i , v i , b i ] (1)
[0117] In formula (1), S i Indicates the status of the VIO system; (R i , p i ) represents the camera pose data, R i Indicates the camera's position information, p i Represents the camera’s attitude information, (R i , p i ) belongs to the special Euclidean group SE(3); v i represents the speed of the camera, b i represents the IMU bias, in, represents the deviation of the gyroscope, represents the bias of the accelerometer,
[0118] Since the acquisition frequency of IMU is higher than that of visual sensor, in order to use the measurement data of IMU to estimate the state of VIO system, it is necessary to pre-integrate the measurement data of IMU between consecutive key frames i and i+1, and then obtain the change values of rotation, speed and displacement between two key frames; according to the change values obtained by pre-integration, calculate the rotation residual term, speed residual term and displacement residual term; further, the inertial residual term can be obtained As shown in formula (2).
[0119]
[0120] In formula (2), represents the inertia residual term; represents the rotation residual term; represents the velocity residual term; represents the displacement residual term; R i Represents the position information of the camera at key frame i; Represents R i The transpose of R i+1 Indicates the position information of the camera at key frame i+1; ΔR i,i+1 Indicates the rotation change value; Denotes ΔR i,i+1 The transpose of v i Indicates the speed of the camera at key frame i; v i+1 Indicates the speed of the camera at key frame i+1; Δv i,i+1 Indicates the speed change value; p i represents the camera posture information at key frame i; p i+1 Indicates the camera's posture information at key frame i+1; Δp i,i+1 Represents the displacement change value; Log represents the logarithmic mapping on SO(3); used to map the Lie group to the vector space; g represents the gravitational acceleration; Δt i,i+1 represents the time interval between two adjacent key frames i and i+1; p j Represents the camera's posture information at key frame j.
[0121] Then, the state of the VIO system is estimated using the visual sensor data. ORB feature extraction is performed on the image captured by the visual sensor to obtain ORB feature points, also known as ORB landmark points; the three-dimensional coordinates of each ORB landmark point are calculated based on the parallax principle; based on the landmark point projection in key frame i and the position x of ORB landmark point j j The reprojection error between ij The visual residual term is obtained, as shown in formula (3).
[0122]
[0123] In formula (3), r ij represents the visual residual term, u i,j represents the observation of ORB landmark point j in image frame i, x j represents the position of ORB landmark point j, Π represents the projection function, T CB Represents the transformation between the IMU coordinate system and the visual sensor coordinate system, T CB ∈SE(3),T i =[R i , p i ]∈SE(3), Represents the mapping of the SE(3) group to three-dimensional space.
[0124] Combined with the inertia residual and the visual residual term r ij , using the least squares problem based on keyframes to optimize the state of the VIO system. Given k+1 keyframes, the state estimation under each keyframe There are l ORB landmarks that can appear in these key frames and the coordinates X={x0...x l-1}, to optimize the state of the VIO system, the optimization problem can be shown as formula (4).
[0125]
[0126] In formula (4), represents the inertia residual term, r ij represents the visual residual term, K j is the key frame set of ORB landmark point j, l represents the number of ORB landmark point j, ρ Hub Represents the loss function Huber kernel. For the reprojection error, the Huber kernel is used to reduce the impact of false matching.
[0127] Finally, the optimized state estimation value is obtained. Among them, the historical waist motion trajectory τ can be obtained by the state p of the VIO system i It is expressed as shown in formula (5).
[0128] τ=[p i ] (5)
[0129] It can be understood that the visual inertial odometry system used can be robustly used to measure the historical waist motion trajectory under daily walking terrain.
[0130] In a possible implementation, the waist motion trajectory prediction model is obtained by:
[0131] Obtaining a training set; wherein the training set includes a plurality of sample waist motion trajectories and a plurality of prediction results of the sample waist motion trajectories;
[0132] Construct an initial waist motion trajectory prediction model;
[0133] An initial waist motion trajectory prediction model is trained using a plurality of sample waist motion trajectories and prediction results of a plurality of sample waist motion trajectories to obtain a waist motion trajectory prediction model.
[0134] It should be noted that, in this embodiment, before using the waist motion trajectory prediction model to predict the waist motion trajectory of the wearer wearing the wearable walking aid when walking, it is necessary to train the constructed initial waist motion trajectory prediction model. Figure 2 As shown, Figure 2 It is a structural schematic diagram of a preset terrain feature extraction model and a waist motion trajectory prediction model provided in an embodiment of the present application. The initial waist motion trajectory prediction model is a newly constructed deep learning model that has not been trained. The training set is a data subset used to train the model. These data contain known input features and expected output labels, namely, sample waist motion trajectories and predicted results of sample waist motion trajectories. The sample waist motion trajectory is a waist motion trajectory measured when the wearer is walking while wearing a wearable walking aid. The predicted result of the sample waist motion trajectory is the predicted result of the model output.
[0135] It can be understood that by training the initial waist motion trajectory prediction model, the model can accurately predict new data and accurately predict the wearer's waist motion trajectory to achieve the assistance of guiding the wearable walking aid device.
[0136] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0137] Corresponding to a waist motion trajectory prediction method in the above embodiment, Figure 5 A schematic structural diagram of a waist motion trajectory prediction device provided by an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.
[0138] Reference Figure 5 , the waist motion trajectory prediction device 3 of this embodiment includes:
[0139] An acquisition module 31 is used to acquire the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment, and to acquire the environmental point cloud data of the second terrain that the wearer will walk on at a future moment;
[0140] The encoding module 32 is used to extract features from the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain;
[0141] An optimization module 33, used for embedding the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory;
[0142] The prediction module 34 is used to input the optimized historical waist motion trajectory into the waist motion trajectory prediction model, predict the waist motion trajectory of the wearer when he is about to walk on the second terrain, and generate a target waist motion trajectory corresponding to the wearer when he is about to walk on the second terrain.
[0143] It can be understood that the waist motion trajectory prediction device 3 provided in the embodiment of the present application first obtains the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment through the acquisition module 31, and obtains the environmental point cloud data of the second terrain that the wearer will walk at a future moment; then, the environmental point cloud data of the second terrain is subjected to feature extraction through the encoding module 32 to obtain the terrain feature data corresponding to the second terrain; the terrain feature data is embedded into the historical waist motion trajectory through the optimization module 33 to generate the optimized historical waist motion trajectory corresponding to the historical waist motion trajectory; finally, the optimized historical waist motion trajectory is input into the waist motion trajectory prediction model through the prediction module 34 to predict the waist motion trajectory of the wearer when he is about to walk on the second terrain, and the target waist motion trajectory corresponding to the wearer when he is about to walk on the second terrain is generated. The present application solves the problem that the prior art only measures the waist motion trajectory of a wearer wearing a wearable walking aid device but lacks prediction of the wearer's waist motion trajectory, resulting in the inability to effectively guide the wearable walking aid device to assist the wearer in walking and the inability to obtain the wearer's walking movement intention, thereby achieving rapid and effective prediction of the waist motion trajectory of a wearer wearing a wearable walking aid device in daily multi-terrain walking.
[0144] It should be noted that the information interaction, execution process and other contents between the modules in the above-mentioned waist motion trajectory prediction device 3 are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0145] Furthermore, the encoding module 32 includes:
[0146] The binarization submodule is used to perform two-dimensional projection on all point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data;
[0147] A terrain feature extraction submodule is used to extract terrain features in the binary image by using a preset terrain feature extraction model to obtain initial terrain feature data corresponding to the second terrain;
[0148] The terrain feature compensation submodule is used to compensate the initial terrain feature data to obtain terrain feature data.
[0149] Furthermore, the terrain feature compensation submodule includes:
[0150] A first binarization unit is used to encode the terrain feature data corresponding to the current moment to obtain a first binary image;
[0151] A backtracking unit, used to obtain a backtracking time corresponding to a backtracking distance through a visual inertial odometry system; wherein the backtracking distance is used to characterize the terrain feature data missing in the first binary image;
[0152] A second binarization unit is used to encode the missing terrain feature data corresponding to the backtracking time to obtain a second binary image;
[0153] A missing pixel extraction unit is used to extract pixels corresponding to the tracing distance in the second binary image according to the tracing distance to obtain the missing pixels;
[0154] An optimization unit, used for splicing the missing pixels into the first binary image to obtain an optimized first binary image;
[0155] The terrain feature data determining unit is used to obtain terrain feature data based on optimizing the first binary image.
[0156] Furthermore, the optimization module 33 includes:
[0157] An interpolation processing unit is used to use the X-axis historical waist motion trajectory as a reference, and perform interpolation processing on the Y-axis historical waist motion trajectory and the Z-axis historical waist motion trajectory respectively to obtain the XY-axis historical waist motion trajectory and the XZ-axis historical waist motion trajectory;
[0158] A filtering processing unit is used to filter the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory respectively to obtain an optimized XY axis historical waist motion trajectory and an optimized XZ axis historical waist motion trajectory;
[0159] The feature embedding unit is used to add the terrain feature data to the end of the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory, respectively, to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory.
[0160] Furthermore, the acquisition module 31 includes:
[0161] An initial environment point cloud data acquisition unit, used to acquire initial environment point cloud data of a second terrain that the wearer will walk on at a future moment through a visual sensor fixed to a predetermined body part of the wearer;
[0162] The environmental point cloud data acquisition unit is used to pre-process the initial environmental point cloud data to obtain environmental point cloud data of a second terrain that the wearer will walk on in the future; wherein the pre-processing includes at least one of the following: through filtering and rotation processing.
[0163] Furthermore, the acquisition module 31 further includes:
[0164] A motion information and camera posture acquisition unit, used to acquire walking motion information data and camera posture data of the wearer when walking on the first terrain through an inertial measurement unit connected to the visual sensor;
[0165] The historical waist motion trajectory acquisition unit is used to calculate the camera motion trajectory corresponding to the visual sensor according to the walking motion information data and the camera posture data, and obtain the historical waist motion trajectory of the wearer when walking on the first terrain at the historical moment.
[0166] Furthermore, the waist motion trajectory prediction device 3 comprises:
[0167] A training set acquisition module, used to acquire a training set; wherein the training set includes a plurality of sample waist motion trajectories and a plurality of prediction results of the sample waist motion trajectories;
[0168] An initial prediction model building module is used to build an initial waist motion trajectory prediction model;
[0169] The prediction model training module is used to train an initial waist motion trajectory prediction model using multiple sample waist motion trajectories and prediction results of multiple sample waist motion trajectories to obtain a waist motion trajectory prediction model.
[0170] The present application also provides a waist motion trajectory prediction device, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of a waist motion trajectory prediction device provided by an embodiment of the present application. Figure 6 The waist motion trajectory prediction device 4 of this embodiment includes: a memory 41, a processor 42, and a computer program stored in the memory 41 and executable on the processor 42. When the processor 42 executes the computer program, the steps in any one of the above waist motion trajectory prediction method embodiments are implemented.
[0171] The present application also provides a wearable walking aid device, such as Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of a wearable walking aid device provided in one embodiment of the present application. Figure 7 The wearable walking aid device 5 includes: a controller 51, and the controller 51 is used to execute the steps in any of the above-mentioned waist motion trajectory prediction method embodiments.
[0172] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0173] The embodiment of the present application also provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0174] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0175] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0176] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0177] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0179] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A waist motion trajectory prediction method, It is characterized in that include: Acquire the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment, and acquire the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; Performing feature extraction on the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; Embedding the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory; The optimized historical waist motion trajectory is input into a waist motion trajectory prediction model, the waist motion trajectory of the wearer when walking on the second terrain is predicted, and a target waist motion trajectory corresponding to the wearer when walking on the second terrain is generated.
2. The waist motion trajectory prediction method according to claim 1, It is characterized in that The extracting features of the environment point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain includes: Performing two-dimensional projection on all point clouds in the environmental point cloud data to obtain a binary image corresponding to the environmental point cloud data; Extracting terrain features from the binary image using a preset terrain feature extraction model to obtain initial terrain feature data corresponding to the second terrain; The initial terrain feature data is compensated to obtain the terrain feature data.
3. The waist motion trajectory prediction method according to claim 2, It is characterized in that The compensating the initial terrain feature data to obtain the terrain feature data includes: Encoding the terrain feature data corresponding to the current moment to obtain a first binary image; Obtaining a backtracking time corresponding to a backtracking distance through a visual inertial odometry system; wherein the backtracking distance is used to characterize the terrain feature data missing in the first binary image; Encoding the missing terrain feature data corresponding to the backtracking time to obtain a second binary image; According to the backtracking distance, extract pixels corresponding to the backtracking distance in the second binary image to obtain missing pixels; splicing the missing pixels into the first binary image to obtain an optimized first binary image; The terrain feature data is obtained based on the optimized first binary image.
4. The waist motion trajectory prediction method according to any one of claims 1 to 3, It is characterized in that The historical waist motion trajectory includes: an X-axis historical waist motion trajectory, a Y-axis historical waist motion trajectory and a Z-axis historical waist motion trajectory; The embedding of the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory includes: Taking the X-axis historical waist motion trajectory as a reference, interpolation processing is performed on the Y-axis historical waist motion trajectory and the Z-axis historical waist motion trajectory respectively to obtain the XY-axis historical waist motion trajectory and the XZ-axis historical waist motion trajectory; Performing filtering processing on the XY axis historical waist motion trajectory and the XZ axis historical waist motion trajectory respectively to obtain an optimized XY axis historical waist motion trajectory and an optimized XZ axis historical waist motion trajectory; The terrain feature data are added to the end of the optimized XY axis historical waist motion trajectory and the optimized XZ axis historical waist motion trajectory respectively to generate the optimized historical waist motion trajectory corresponding to the historical waist motion trajectory.
5. The waist motion trajectory prediction method according to claim 1, It is characterized in that The step of obtaining the environmental point cloud data of the second terrain on which the wearer will walk in the future includes: Acquiring initial environmental point cloud data of the second terrain that the wearer will walk on at a future moment through a visual sensor fixed to a predetermined body part of the wearer; The initial environmental point cloud data is preprocessed to obtain the environmental point cloud data of the second terrain on which the wearer will walk at a future moment; wherein the preprocessing includes at least one of the following: through filtering and rotation processing.
6. The waist motion trajectory prediction method according to claim 5, It is characterized in that The obtaining of the historical waist movement trajectory of the wearer when walking on the first terrain at the historical moment includes: Acquiring walking motion information data and camera position data of the wearer when walking on the first terrain through an inertial measurement unit connected to the visual sensor; The camera motion trajectory corresponding to the visual sensor is calculated according to the walking motion information data and the camera posture data to obtain the historical waist motion trajectory of the wearer when walking on the first terrain at the historical moment.
7. The waist motion trajectory prediction method according to claim 1, It is characterized in that The waist motion trajectory prediction model is obtained by the following method: Acquire a training set; wherein the training set includes a plurality of sample waist motion trajectories and a plurality of prediction results of the sample waist motion trajectories; Construct an initial waist motion trajectory prediction model; The initial waist motion trajectory prediction model is trained using a plurality of the sample waist motion trajectories and a plurality of prediction results of the sample waist motion trajectories to obtain the waist motion trajectory prediction model.
8. A waist motion trajectory prediction device, It is characterized in that include: An acquisition module, used to acquire the historical waist motion trajectory of the wearer when walking on the first terrain at a historical moment, and to acquire the environmental point cloud data of the second terrain that the wearer will walk on at a future moment; An encoding module, used for performing feature extraction on the environmental point cloud data of the second terrain to obtain terrain feature data corresponding to the second terrain; An optimization module, used for embedding the terrain feature data into the historical waist motion trajectory to generate an optimized historical waist motion trajectory corresponding to the historical waist motion trajectory; A prediction module is used to input the optimized historical waist motion trajectory into a waist motion trajectory prediction model, predict the waist motion trajectory of the wearer when he is about to walk on the second terrain, and generate a target waist motion trajectory corresponding to the wearer when he is about to walk on the second terrain.
9. A waist motion trajectory prediction device, It is characterized in that The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.
10. A wearable walking aid, It is characterized in that include: A controller, wherein the controller is used to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Trajectory prediction method and device and computer readable storage medium
CN112070808A
Intelligent agent trajectory prediction method, system and device and storage medium
CN114022847A
Multi-information fusion pedestrian trajectory prediction device and method in crowded space
CN115861383A
Trajectory prediction method and device, storage medium and vehicle
CN116503700A
Prosthetic limb motion prediction method and device and terminal equipment
CN117218152A
Cited By
Moving target trajectory prediction method and related equipment
CN120144975A
