Human lower limb motion environment recognition method, system, device and medium

Abstract

The application discloses a human lower limb motion environment recognition method, system, device and medium, the method comprises the following steps: according to the first environment topography point cloud data returned by the depth camera and the Euler angle data returned by the IMU, the first environment topography point cloud data is converted to the world coordinate system; after conversion, for the first environment topography point cloud data, the point cloud data outside the preset target range is removed, and the second environment topography point cloud data is obtained; the second environment topography point cloud data is projected onto the sagittal plane of human anatomy, and a binary image is obtained; the binary image is subjected to morphological processing and feature extraction, and a plurality of features are obtained; based on the plurality of features, an SVM classifier is used to realize online identification of human front flat ground, obstacles, ascending stairs, descending stairs, ascending slopes and descending slopes. By implementing the method, the complexity of the algorithm can be reduced, and the portability, wearability and applicability of the system can be improved.

Description

TECHNICAL FIELD

[0001] The application relates to a human lower limb motion environment recognition method, system, device and medium, and belongs to the field of data recognition. BACKGROUND

[0002] A lower limb exoskeleton is a human lower limb motion assisting device including a mechanical system, a sensing system, a control system and a power system, which aims to assist a user in lower limb motion in daily life and reduce energy consumption of lower limb motion. In the process of human lower limb motion, the motion mode of the lower limb of the user is different in different terrains, such as walking in a flat terrain and climbing stairs in an ascending stair terrain. For different motion environment terrains, the exoskeleton needs to switch different control modes to achieve better coupling with the human lower limb motion.

[0003] A traditional method is to switch the control mode by sensing the change of the human motion state, which has the disadvantage of hysteresis. Therefore, direct recognition of the environment terrain is considered to reduce or even eliminate the hysteresis, so as to achieve better closed-loop control of the exoskeleton. SUMMARY

[0004] Therefore, the application provides a human lower limb motion environment recognition method, system, mobile terminal device and readable storage medium, which can realize online recognition of six terrains in the process of human lower limb motion, including flat ground, obstacles, ascending stairs, descending stairs, ascending slopes and descending slopes in front, while reducing the complexity of the algorithm and improving the portability, wearability and applicability of the system.

[0005] The first aspect of the embodiment of the application discloses a human lower limb motion environment recognition method, which is executed by a mobile terminal device, the mobile terminal device is connected with a lower limb exoskeleton sensing device, the lower limb exoskeleton sensing device includes a depth camera and an IMU arranged on the waist, and the method comprises the following steps:

[0006] According to the first environment terrain point cloud data returned by the depth camera and the Euler angle data returned by the IMU, the first environment terrain point cloud data is converted to a world coordinate system;

[0007] After the conversion, the point cloud data outside a preset target range is removed for the first environment terrain point cloud data, and second environment terrain point cloud data is obtained;

[0008] The second environment terrain point cloud data is projected onto a sagittal plane of human anatomy to obtain a binary image;

[0009] The binary image is subjected to morphological processing and feature extraction to obtain a plurality of features;

[0010] Based on the plurality of features, online recognition of a flat ground, an obstacle, an ascending stair, a descending stair, an ascending slope and a descending slope in front of the human body is realized using an SVM classifier.

[0011] As a preferred embodiment, the first environment terrain point cloud data returned by the depth camera and the Euler angle data returned by the IMU are converted to a world coordinate system, and the first environment terrain point cloud data is converted to a world coordinate system as follows:

[0012]

[0013] wherein P w is the point cloud data in the world coordinate system; P IMU is the point cloud data in the IMU coordinate system; a rotation matrix representing the attitude relationship between the camera coordinate system and the IMU coordinate system, a rotation matrix representing the attitude relationship between the IMU coordinate system and the world coordinate system, as follows:

[0014]

[0015]

[0016] wherein β and γ are the yaw angle and the roll angle returned by the IMU.

[0017] As a preferred embodiment, point cloud data outside a preset target range is removed by a pass-through filter.

[0018] The range of the pass-through filter is: the region with a z-axis range of [-2.4m, 0.8m], a y-axis range of [0.2m, 1.75m] and an x-axis range of [-0.17m, 0.23m].

[0019] As a preferred embodiment, the projection rule is as follows:

[0020]

[0021]

[0022] wherein 155×320 represents the size of the projected binary image.

[0023] As a preferred embodiment, the morphological processing of the binary image includes:

[0024] For the binary image, an opening operation is performed using a 3×3 structure element, and a closing operation is performed using a 5×5 structure element.

[0025] As a preferred embodiment, the plurality of features include 20-dimensional features, and the 20-dimensional features include:

[0026] circumscribed rectangle width W rect , circumscribed rectangle height H rect , circumscribed rectangle top-left corner y-axis coordinate minimum circumscribed rectangle width W minRect , minimum circumscribed rectangle height H minRect , minimum circumscribed rectangle top-left corner y-axis coordinate minimum circumscribed rectangle tilt angle image centroid y-axis coordinate y centroid , corner point fitted line slope k; corner point fitted line mean absolute deviation MAD, horizontal line proportion R HL , vertical line proportion R VL minimum circumscribed rectangle pixel density p minrect ; four sub-regional pixel densities p r1 , p r2 , p r3 , p r4 ; horizontal sub-regional pixel density ratio p H , vertical sub-regional pixel density ratio p V , diagonal sub-regional pixel density ratio p D .

[0027] As a preferred embodiment, the SVM classifier is a multivariate SVM classifier, the number of classification target categories is 6, and the kernel function is a radial basis function.

[0028] The second aspect of the embodiment of the application discloses a human lower limb motion environment recognition system, which is applied to a mobile terminal device, the mobile terminal device is connected with a lower limb exoskeleton sensing device, the lower limb exoskeleton sensing device comprises a depth camera and an IMU arranged at a waist, and the system comprises:

[0029] a conversion unit, configured to convert first environment terrain point cloud data returned by the depth camera and Euler angle data returned by the IMU to a world coordinate system;

[0030] a removal unit, configured to remove point cloud data outside a preset target range from the first environment terrain point cloud data after conversion to obtain second environment terrain point cloud data;

[0031] a projection unit, configured to project the second environment terrain point cloud data onto a sagittal plane of human anatomy to obtain a binary image;

[0032] a processing and extraction unit, configured to perform morphological processing and feature extraction on the binary image to obtain a plurality of features;

[0033] An online recognition unit is configured to implement online recognition of a flat ground, an obstacle, an ascending stair, a descending stair, an ascending slope and a descending slope in front of a human body based on the plurality of features and using an SVM classifier.

[0034] A third aspect of the embodiments of the present application discloses a mobile terminal device, comprising a processor and a memory for storing a program executable by the processor, and when the processor executes the program stored in the memory, any one of the human lower limb motion environment recognition methods disclosed by the embodiments of the present application is implemented.

[0035] A fourth aspect of the embodiments of the present application discloses a readable storage medium, storing a program, and when the program is executed by a processor, any one of the human lower limb motion environment recognition methods disclosed by the embodiments of the present application is implemented.

[0036] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0037] The embodiments of the present application build an exoskeleton perception system software and hardware platform based on an Android mobile phone, which improves the portability, wearability and applicability of the system compared with the traditional exoskeleton system using a computer as an upper computer. The embodiments of the present application apply a visual sensor to exoskeleton perception research, perform terrain recognition by projecting environmental depth information to a binary graph, extracting binary graph features and using an SVM classifier for classification and recognition, and reduce the complexity of the algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the drawings shown.

[0039] Figure 1 A general assembly schematic diagram of a lower limb exoskeleton perception system device provided by the embodiments of the present application.

[0040] Figure 2 A component schematic diagram of a lower limb exoskeleton perception system device provided by the embodiments of the present application.

[0041] Figure 3 A communication schematic diagram of a sensor and an upper computer provided by the embodiments of the present application.

[0042] Figure 4 A layout schematic diagram of a sensor and a depth camera provided by the embodiments of the present application.

[0043] Figure 5A flowchart of a human lower limb motion environment recognition method provided by an embodiment of the present application.

[0044] Figure 6 A flowchart of a human lower limb motion environment recognition method provided by an embodiment of the present application.

[0045] Figure 7 A cropping diagram of a region of interest provided by an embodiment of the present application.

[0046] Figure 8 A structural diagram of a human lower limb motion environment recognition system provided by an embodiment of the present application.

[0047] Figure 9 A structural diagram of a mobile terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0049] In the specification and claims of the present application, the objects distinguished by "first" and "second" are usually a category, and the number of objects is not limited, for example, the first object can be one or multiple. In addition, "and" in the specification and claims means at least one of the connected objects, and the character " / " generally represents an "or" relationship between the front and rear associated objects.

[0050] It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0051] The embodiments of the present application disclose a human lower limb motion environment recognition method, and a lower limb exoskeleton perception system device applying the method. By running the method of the embodiments on the device, online recognition of six terrains, i.e., flat ground, obstacle, ascending stairs, descending stairs, ascending slope, and descending slope in front of the human lower limb motion process, can be achieved.

[0052] The assembly diagram of the lower limb exoskeleton perception system device is as shown in Figure 1As shown in Figure 2 As shown in the drawings, the lower limb exoskeleton perception system device comprises an Android smartphone 1, a data line 2, a shell 3, a binocular camera module 4, a shell 5, a waistband 6, a base 7, an IMU module 8, and a bottom shell 9. Among them, the binocular camera module coordinate system and the IMU arranged coordinate system are as shown in Figure 1

[0053] The Android smartphone 1 is a smartphone with a running system of Android 10.0 or above, a Bluetooth connection function, and a Type-C data interface, which serves as the upper computer in the lower limb exoskeleton perception system.

[0054] The binocular camera module 4 is composed of two cameras and corresponding peripheral circuits, and the baseline distance (the distance between the optical centers of the two cameras) is 60 mm. The module is connected with the upper computer through the data line 2, communicates according to the UVC protocol, and returns the images acquired by the two cameras. The resolution of the image returned by a single camera is 640x480, and the return frame rate is 60 fps.

[0055] The IMU module 8 comprises a nine-axis inertial measurement unit with model JY901, a Bluetooth module with model HC-05, a power supply and peripheral circuits. The module sends the nine-axis data acquired by the IMU to the upper computer through Bluetooth, and the return frequency is 100 Hz.

[0056] In the Android smartphone 1, the communication with the binocular camera module 4 and the IMU module 8 is realized by constructing an application program, and the sensor data is acquired, as shown in Figure 3 The upper computer application program is constructed by multi-threading, in which the UVC thread realizes the acquisition of image data collected by the binocular camera module 4 through the data line, and the Bluetooth thread acquires the data returned by the IMU module 8.

[0057] In the UVC thread, the application program communicates with the binocular camera module 4 through the UVC protocol to acquire the binocular images returned by the binocular camera module 4.

[0058] As shown in Figure 4 The lower limb exoskeleton perception system device is worn by the user at the waist position, the main body part of the device (i.e. the shell 3 connection part) is located in front of the waist, and the camera optical axis of the binocular camera module 4 forms an angle of about 50° with the vertical direction downward.

[0059] The method of the embodiment is realized by constructing an application program on an Android phone for online running.

[0060] As shown in Figure 5 and Figure 6 As shown in Figure 5 and Figure 6 ​is a flowchart of a human lower limb motion environment recognition method provided by an embodiment of the present application. The human lower limb motion environment recognition method can include the following steps:

[0061] S1, a user correctly wears a lower limb exoskeleton sensing system device, connects the device with an Android smartphone 1 through a data line 2, opens an installed application of the method in the Android smartphone 1, runs the method through the program, and the user performs normal lower limb walking motion.

[0062] S2, the method acquires an environment binocular image photographed by a binocular camera module. In order to reduce the calculation amount of subsequent binocular matching, the image is cropped in a region of interest, as shown in Figure 7 , a monocular image with an original size of 640x480 is cropped to obtain a region of interest image with a size of 257x480. The SGBM algorithm is used to perform binocular matching on the cropped binocular image, and point cloud data P cam of the environment terrain in the camera coordinate system is acquired.

[0063] S3, the acquired point cloud data in the camera coordinate system is converted to the world coordinate system. The world coordinate system is defined as shown in Figure 6 , the y-axis is parallel to the sagittal axis of human anatomy, and the z-axis is parallel to the vertical axis of human anatomy. Therefore, the point cloud conversion is performed by the following formula:

[0064]

[0065] wherein P w is the point cloud data in the world coordinate system; P IMU is the point cloud data in the IMU coordinate system; is a rotation matrix representing the attitude relationship between the camera coordinate system and the IMU coordinate system, the relative pose of the camera and the IMU is fixed, and therefore is a fixed value; represents the attitude relationship between the IMU and the world coordinate, and is obtained from the Euler angles returned by the IMU, i.e.

[0066]

[0067]

[0068] wherein β and γ are the yaw angle and roll angle returned by the IMU.

[0069] S4, the point cloud data P wPass-through filtering is performed. The purpose of pass-through filtering is to remove point cloud data outside the target area. The range of pass-through filtering is set as follows: the z-axis range is [-2.4m, 0.8m], the y-axis range is [0.2m, 1.75m], and the x-axis range is [-0.17m, 0.23m]. That is, the filtered point cloud data is:

[0070] P′ w = {(x i ,y i ,z i )|i = 1,..., n; -0.17m < x i < 0.23m, 0.2m < y i < 1.75m, -2.4m < z i < 0.8m}

[0071] S5, project the point cloud onto the sagittal plane of human anatomy (i.e. the plane formed by the y-axis and z-axis of the world coordinate system) to obtain a binary image. Create a projection binary image with a size of 155x320. The point cloud is projected onto the binary image according to the following rules:

[0072]

[0073]

[0074] where r and c are the pixel coordinates of the binary image; (x i ,y i ,z i ) is the three-dimensional coordinate of the point cloud data, with the unit of meter (m).

[0075] S6, perform morphological processing on the binary image generated by projection. This includes using a 3x3 structure element to perform an opening operation on the binary image, and using a 5x5 structure element to perform a closing operation.

[0076] S7, perform feature extraction on the binary image. A total of 20-dimensional feature vectors are extracted:

[0077]

[0078] The circumscribed rectangle refers to the rectangle that envelopes the binary image, which is composed of horizontal and vertical lines; the minimum circumscribed rectangle refers to the rectangle with the smallest area that envelopes the binary image, which may have an inclination angle.

[0079] The image centroid refers to the center position of the image area, i.e. the average of all pixel coordinates.

[0080] The corner point fitting straight line is obtained by using the Shi-Tomasi corner point detection algorithm to extract the corner points of the binary image, and then fitting the corner points to obtain a straight line.

[0081] The corner point fitting straight line average absolute deviation calculation formula of the feature 10 is as follows:

[0082]

[0083] Wherein, (v x ,ν y ) is the unit vector of the corner point fitting straight line, (x0,y0) is the coordinate of a point on the straight line, (x i ,y i ) is the coordinate of the binary image pixel point, and n is the number of binary image pixel points.

[0084] The feature 11 horizontal line ratio, first uses a 1x11 structure element to open the binary image, and then the number of image pixel points in the processed binary image is n HL , the number of image pixel points in the binary image before processing is n binary , and the horizontal line ratio R HL The calculation formula is as follows:

[0085]

[0086] The feature 12 vertical line ratio, first uses an 11x1 structure element to open the binary image, and then the number of image pixel points in the processed binary image is n VL , the number of image pixel points in the binary image before processing is n binary , and the vertical line ratio R VL The calculation formula is as follows:

[0087]

[0088] The feature 13 minimum circumscribed rectangle pixel density ρ minrect is the ratio of the number of binary image pixels to the area of the minimum circumscribed rectangle, and the calculation formula is as follows:

[0089]

[0090] The feature 14-17 circumscribed rectangle sub-region pixel density is to divide the circumscribed rectangle into four sub-regions by a vertical line and a horizontal line passing through the image centroid, and respectively count the pixel density of the sub-region, that is:

[0091]

[0092]

[0093]

[0094]

[0095] wherein n r1 , n r2 , n r3 , n r4 are the pixel number of the binary image in the four divided sub-regions respectively, and n binary is the pixel number of the binary image.

[0096] The horizontal sub-region pixel density ratio P H of the feature 18 is calculated as follows:

[0097]

[0098] The vertical sub-region pixel density ratio P H of the feature 19 is calculated as follows:

[0099]

[0100] The diagonal sub-region pixel density ratio P H of the feature 20 is calculated as follows:

[0101]

[0102] After obtaining the 20-dimensional feature vector, the features are normalized using linear normalization method.

[0103] S8, using support vector machine (SVM) to realize classification of the input feature vector. The SVM classifier is a multivariate SVM classifier, the number of classification target categories is 6, and the kernel function is radial basis function.

[0104] Those skilled in the art can understand that all or part of the steps in the method for implementing the above embodiments can be instructed by a program to relevant hardware, and the corresponding program can be stored in a computer readable storage medium.

[0105] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all of the shown operations must be performed to achieve the desired result. On the contrary, the depicted steps can change the order of execution. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps.

[0106] As Figure 8 shown, Figure 8This is a schematic diagram of a human lower limb motion environment recognition system provided in an embodiment of this application. The human lower limb motion environment recognition system is applied to a mobile terminal device, which is connected to a lower limb exoskeleton sensing device. The lower limb exoskeleton sensing device includes a depth camera and an IMU mounted on the waist. The human lower limb motion environment recognition system may include:

[0107] The conversion unit 801 is used to convert the first environmental terrain point cloud data back from the depth camera to the world coordinate system based on the first environmental terrain point cloud data back from the depth camera and the Euler angle data back from the IMU.

[0108] The removal unit 802 is used to remove point cloud data outside the preset target range from the first environmental terrain point cloud data after conversion, so as to obtain the second environmental terrain point cloud data.

[0109] Projection unit 803 is used to project the second environmental terrain point cloud data onto the sagittal plane of human anatomy to obtain a binary image;

[0110] The processing and extraction unit 804 is used to perform morphological processing and feature extraction on the binary image to obtain multiple features;

[0111] The online recognition unit 805 is used to recognize flat ground, obstacles, ascending stairs, descending stairs, ascending ramps, and descending ramps in front of a human body based on the multiple features using an SVM classifier.

[0112] like Figure 9 As shown, Figure 9 This is a schematic diagram of the structure of a mobile terminal device provided in an embodiment of this application. The mobile terminal device may include a processor 902, a memory, an input device 903, an output device 904, and a network interface 905 connected via a system bus 901. The processor 902 provides computing and control capabilities. The memory includes a non-volatile storage medium 906 and internal memory 907. The non-volatile storage medium 906 stores an operating system, computer programs, and a database. The internal memory 907 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 906. When the computer program is executed by the processor 902, it implements any of the human lower limb movement environment recognition methods described in the above embodiments.

[0113] This application discloses a storage medium. The storage medium is a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements any of the human lower limb movement environment recognition methods described in the above embodiments.

[0114] It should be noted that the computer readable storage medium in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0115] In the embodiments of the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the embodiments of the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable programs. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium that can send, propagate or transmit programs for use by or in connection with an instruction execution system, apparatus or device. The computer programs contained in the computer readable storage medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination of the above.

[0116] The computer readable storage medium described above can be written in one or more programming languages or a combination of the above for executing the computer programs of the embodiments of the present application, including object oriented programming languages such as Java, Python, C++, and conventional procedural programming languages such as C language or similar programming languages. The program can be executed entirely on a user computer, partially on a user computer, as a separate software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet by using an Internet service provider).

[0117] In summary, the embodiment of the application builds an exoskeleton sensing system software and hardware platform based on an Android mobile phone, which improves the portability, wearability and applicability of the system compared with the traditional exoskeleton system using a computer as an upper computer.

[0118] The above merely describes preferred embodiments of the present application patent, but the protection scope of the present application patent is not limited thereto, and any person skilled in the art can make equivalent replacements or changes to the technical scheme and inventive concept of the present application patent within the scope disclosed by the present application patent, which shall all fall within the protection scope of the present application patent.

Claims

1. A human lower limb motion environment recognition method, characterized by, The method is executed by a mobile terminal device connected with a lower extremity exoskeleton perception device, the lower extremity exoskeleton perception device comprises a depth camera and an IMU arranged at a waist, and the method comprises the following steps: According to the first environment terrain point cloud data returned by the depth camera and the Euler angle data returned by the IMU, the first environment terrain point cloud data is converted to a world coordinate system; the first environment terrain point cloud data is converted to the world coordinate system according to the first environment terrain point cloud data returned by the depth camera and the Euler angle data returned by the IMU, and the conversion is as follows: P w is the point cloud data in the world coordinate system; P IMU is the point cloud data in the IMU coordinate system; is a rotation matrix representing the attitude relationship between the camera coordinate system and the IMU coordinate system, is a rotation matrix representing the attitude relationship between the IMU coordinate system and the world coordinate system, as follows: Wherein, β and γ are the yaw angle and the roll angle returned by the IMU; After conversion, the point cloud data outside the preset target range is removed for the first environment terrain point cloud data, and second environment terrain point cloud data is obtained; The second environment terrain point cloud data is projected onto a sagittal plane of human anatomy to obtain a binary image; the projection rule is as follows: wherein 155x320 represents the size of the projection binary image, r and c are pixel coordinates of the binary image, (x i ,y i ,z i ) is the three-dimensional coordinate of the point cloud data, with the unit of meters; The binary image is morphologically processed and feature extracted to obtain a plurality of features; the plurality of features comprises a 20-dimensional feature, and the 20-dimensional feature comprises: circumscribed rectangle width W rect , circumscribed rectangle height H rect , circumscribed rectangle top-left corner y-axis coordinate minimum circumscribed rectangle width W minRect , minimum circumscribed rectangle height H minRect , minimum circumscribed rectangle top-left corner y-axis coordinate minimum circumscribed rectangle angle of inclination image centroid y-axis coordinate y centroid , corner point fitted straight line slope k; corner point fitted straight line mean absolute deviation MAD, horizontal line proportion R HL , vertical line proportion R VL , minimum circumscribed rectangle pixel density p minrect ; four sub-regions pixel density p r1 , p r2 , p r3 , p r4 ; horizontal sub-region pixel density ratio p H , vertical sub-region pixel density ratio p V , diagonal sub-region pixel density ratio p D ; Based on the plurality of features, an SVM classifier is used to realize online identification of a flat ground, an obstacle, an ascending stair, a descending stair, an ascending slope and a descending slope in front of a human body.

2. The method of claim 1, wherein, The point cloud data outside the preset target range is removed through a pass-through filter; The range of the pass-through filter is: the region with a z-axis range of [-2.4m, 0.8m], a y-axis range of [0.2m, 1.75m] and an x-axis range of [-0.17m, 0.23m].

3. The method of claim 1, wherein, The morphological processing of the binary image comprises: For the binary image, a 3*3 structure element is used for open operation, and a 5*5 structure element is used for close operation.

4. The method of claim 1, wherein, The SVM classifier is a multi-element SVM classifier, the number of classification target categories is 6, and the kernel function is a radial basis function.

5. A human lower limb motion environment recognition system characterized by comprising: The method of any one of claims 1-4 is applied to a mobile terminal device connected with a lower extremity exoskeleton perception device, the lower extremity exoskeleton perception device comprises a depth camera and an IMU arranged at a waist, and the system comprises: A conversion unit is configured to convert, according to the first environment terrain point cloud data returned by the depth camera and the Euler angle data returned by the IMU, the first environment terrain point cloud data to a world coordinate system; A removal unit is configured to remove, after conversion, the point cloud data outside the preset target range for the first environment terrain point cloud data, to obtain second environment terrain point cloud data; A projection unit is configured to project the second environment terrain point cloud data onto a sagittal plane of human anatomy to obtain a binary image; A processing and extraction unit is configured to morphologically process and feature extract the binary image to obtain a plurality of features; An online identification unit is configured to use an SVM classifier to realize online identification of a flat ground, an obstacle, an ascending stair, a descending stair, an ascending slope and a descending slope in front of a human body based on the plurality of features.

6. A mobile terminal device comprising a processor and a memory for storing processor-executable programs, characterized in that The processor executes the program stored in the memory to realize the method of any one of claims 1-4.

7. A readable storage medium, storing a program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-4.

Images