An intelligent assisted travel system based on multi-sensor information fusion and health detection

Abstract

The application relates to an intelligent auxiliary travel system based on multi-sensor information fusion and health detection, which comprises a wheelchair trolley, an automatic driving controller, a UWB base station and a health detector. The automatic driving controller comprises a multi-sensor module, a VCU module and a multi-sensor information fusion module. The multi-sensor module is connected with the multi-sensor information fusion module, and the multi-sensor information fusion module is connected with the VCU module. The multi-sensor information fusion module is used for carrying out fusion processing on data information of the multi-sensor by adopting a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm, so that fused multi-sensor data information is obtained. The application not only provides an intelligent auxiliary travel tool with human-machine co-driving function, improves safety and reliability, but also increases the automatic following capacity of the intelligent auxiliary travel tool, and improves the demand of intelligent following and safety monitoring of accompanying and nursing personnel.

Description

Technical Field

[0001] This invention relates to the field of intelligent assisted travel technology, and in particular to an intelligent assisted travel system based on multi-sensor information fusion and health detection. Background Technology

[0002] With the rapid development of artificial intelligence, the application of smart terminal products is becoming increasingly widespread. For people with mobility impairments, improving their quality of life through artificial intelligence will be of great significance. In complex scenarios within restricted areas such as residential areas, scenic spots, and parks, the use of intelligent driving wheelchair-type assistive mobility tools will bring great convenience to people with mobility impairments. Existing electric wheelchair-type assistive mobility tools simply respond to control commands from joysticks or buttons, performing basic driving, steering, and braking operations. However, in complex driving environments such as narrow alleys, elevators, and uneven roads, users must rely entirely on their own driving skills to solve various problems encountered during the journey, or additional assistance from other personnel is required. This results in users of assistive mobility tools not being able to independently cope with complex driving environments, leading to a poor user experience.

[0003] In the prior art, a patent (application number: 201811398670.8) discloses an assisted driving wheelchair, which consists of a mobile phone service module, a wireless communication module, a backend server module, a database module, and a physical wheelchair robot module. Combined with autonomous driving technology, this advanced assisted driving wheelchair can provide users with fully autonomous driving services in simple scenarios, as well as assisted driving functions in complex environments. Simultaneously, the wheelchair has physiological indicator detection, voice interaction, and positioning functions. Users and caregivers on the mobile phone can monitor their physiological health status in real time, such as blood pressure, body temperature, and electrocardiogram, to allow for timely responses. Users can also issue commands to the wheelchair and obtain necessary information through voice interaction; the positioning function allows caregivers to locate the wheelchair user's position in real time. However, it only has autonomous driving capabilities and lacks manual driving capabilities. In terms of safety and reliability, it is not as good as a human-machine co-driving system, and it lacks automatic following capabilities, making it unfriendly to companions and caregivers. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the present invention provides an intelligent assisted travel system based on multi-sensor information fusion and health detection. It not only provides an intelligent assisted travel tool with human-machine co-driving function to improve safety and reliability, but also increases the automatic following capability of the intelligent assisted travel tool to meet the needs of companions and caregivers for intelligent following and safety monitoring.

[0005] To achieve the above and other related objectives, the present invention provides the following technical solution:

[0006] An intelligent assisted mobility system based on multi-sensor information fusion and health detection includes a wheelchair trolley, an autonomous driving controller, a UWB base station, and a health detector.

[0007] The autonomous driving controller includes a multi-sensor module, a VCU module, and a multi-sensor information fusion module. The multi-sensor module is connected to the multi-sensor information fusion module, and the multi-sensor information fusion module is connected to the VCU module. The multi-sensor information fusion module is used to fuse the data information of the multiple sensors using a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm to obtain fused multi-sensor data information.

[0008] The UWB base station includes a first UWB base station module, a second UWB base station module, a third UWB base station module, a UWB tag module, and a tracking control algorithm module. The first UWB base station module is connected to the UWB tag module, the second UWB base station module is connected to the UWB tag module, and the third UWB base station module is connected to the UWB tag module. The tracking control algorithm module uses a single-target path tracking control algorithm to optimize and adjust the path of the wheelchair trolley.

[0009] Furthermore, the multi-sensor information fusion module is used to perform fusion processing on the data information from multiple sensors using a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm, including:

[0010] M1. Acquire data from multiple sensors, integrate and process the data using a genetic algorithm, and establish a fitness function G. i ,

[0011] , ,

[0012] Among them, α j Let F be the fitness factor, F be the probability function, f be the fitness function, and x be the fitness factor. j Given data from multiple sensors, where N is the sample size and i and j are constant parameters, we obtain the integrated data from the multiple sensors.

[0013] M2. Based on the integrated multi-sensor data, a membership function H is used for fuzzy logic reasoning.

[0014] ,

[0015] Where u represents the integrated multi-sensor data, v represents the fuzzy set data from the multi-sensor dataset, and H... A→B(u,v) is the confidence function, which yields the fuzzy matrix data information from multiple sensors;

[0016] M3. Input the fuzzy matrix data information of the multi-sensor into the neural network algorithm to fuse the data and obtain the fused multi-sensor data information.

[0017] Furthermore, in step M2, the confidence function H A→B (u,v) is,

[0018] ,

[0019] Where a, b, and c are constant parameters, u is the integrated multi-sensor data information, and v is the fuzzy set data information of the multi-sensor system.

[0020] Furthermore, in step M3, the step of inputting the fuzzy matrix data information from the multiple sensors into the neural network algorithm for data fusion includes:

[0021] M31. Based on the fuzzy matrix data information from the multiple sensors, establish the activation function Q of the neural network.

[0022] ,

[0023] ,

[0024] Where, q i For the fuzzy matrix data information of the i-th multi-sensor, β i Here, θ represents the corresponding weight coefficient, and θ is the activation factor.

[0025] M32. Based on the activation function Q of the neural network, the fused multi-sensor data information is obtained.

[0026] Furthermore, the tracking control algorithm module employs a single-target path tracking control algorithm to optimize and adjust the path of the wheelchair vehicle, including:

[0027] L1. Obtain the distance data between the first UWB base station and the UWB tag module, the distance data between the second UWB base station and the UWB tag module, and the distance data between the third UWB base station and the UWB tag module, and establish a multimodal path planning function P.

[0028] ,

[0029] Where ω1 is the weighting coefficient of the first UWB base station, ω2 is the weighting coefficient of the second UWB base station, ω3 is the weighting coefficient of the third UWB base station, and l 1i This refers to the distance data between the first UWB base station and the UWB tag module.2i For the distance data between the second UWB base station and the UWB tag module, l 3i ρ1 represents the distance data between the third UWB base station and the UWB tag module, ρ2 represents the relationship function between the first UWB base station and the second UWB base station, and ρ2 represents the relationship function between the second UWB base station and the third UWB base station.

[0030] L2. Based on the multimodal path planning function P, data information for multiple planned paths is obtained;

[0031] L3. Based on the data information of the multiple planned paths, establish a path evaluation and optimization function V.

[0032] ,

[0033] Where n is the total number of samples, P0 is the mean of the multiple planned paths, and P j Given the data information for the j-th planned path, we obtain the optimized path data information for the wheelchair vehicle.

[0034] Furthermore, the wheelchair trolley includes a vehicle body, a power battery, a drive motor, a motor controller, a rocker arm, and a display screen. The power battery provides electrical energy to the wheelchair trolley. The wheelchair trolley relies on the drive motor and the motor controller to achieve driving, steering, and braking. The rocker arm controls the vehicle's forward and backward movement, acceleration and deceleration, and left and right steering by moving it forward, backward, left, and right. The display screen displays the wheelchair trolley's real-time speed, gear, and battery level information.

[0035] Furthermore, the health detector monitors the user's basic health information, such as blood pressure, body temperature, and respiratory rate, making it convenient for medical staff and caregivers to understand the user's basic health status in a timely manner.

[0036] Furthermore, the multi-sensor module includes a lidar, a front-view camera, a rear-view camera, and an IMU. The lidar is used to acquire point cloud data of the surrounding environment with distance and angle information, and the IMU acquires the attitude and acceleration data of the wheelchair vehicle in real time.

[0037] Furthermore, the system also includes an MCU, which is connected to the UWB base station and is used to receive path optimization data information output by the UWB base station and control the rotation of the left and right motors of the wheelchair trolley to achieve automatic following.

[0038] Furthermore, the system also includes a voice interaction module, which is used for voice wake-up, voice search, and reading audiobooks, providing a voice companion experience.

[0039] The present invention has the following positive effects:

[0040] 1. This invention uses a multi-sensor information fusion module to fuse data from multiple sensors using a neural network algorithm coupled with genetic algorithm and fuzzy logic reasoning algorithm, thereby obtaining fused multi-sensor data information to control the wheelchair trolley. This not only improves the safety and reliability of use, but also makes it applicable to complex and ever-changing scenarios.

[0041] 2. This invention optimizes and adjusts the path of the wheelchair trolley by using a single-target path tracking control algorithm module through a tracking control algorithm module. This not only frees up the hands of caregivers and greatly meets the needs of intelligent following and safety monitoring for users in medical, home-based elderly care and other scenarios, but also improves the user experience. Attached Figure Description

[0042] Figure 1 This is a system framework diagram (I) of the present invention;

[0043] Figure 2 This is a system framework diagram (II) of the present invention;

[0044] Figure 3 This is a system framework diagram (III) of the present invention;

[0045] Figure 4 This is a system framework diagram (IV) of the present invention;

[0046] Figure 5 This is a circuit connection diagram of the present invention;

[0047] Figure 6 This is a circuit connection diagram of the automatic driving controller of the present invention.

[0048] The numbers in the diagram are as follows: 1-Wheelchair trolley, 2-Joystick, 3-Rearview camera, 4-Front-view camera, 5-LiDAR, 6-Health detector, 7-Display screen, 8-Voice interaction module, 9-UWB base station, 10-IMU, 11-Autopilot controller, 12-VCU module, 13-Drive motor, 14-Motor controller, 15-Battery. Detailed Implementation

[0049] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0050] Example 1: As Figure 1 or Figure 2or Figure 3 or Figure 4 As shown, an intelligent assisted travel system based on multi-sensor information fusion and health detection includes a wheelchair trolley 1, an autonomous driving controller 11, a UWB base station 9, and a health detector 6.

[0051] The autonomous driving controller 11 includes a multi-sensor module, a VCU module 12, and a multi-sensor information fusion module. The multi-sensor module is connected to the multi-sensor information fusion module, and the multi-sensor information fusion module is connected to the VCU module 12. The multi-sensor information fusion module is used to fuse the data information of the multiple sensors using a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm to obtain fused multi-sensor data information.

[0052] The UWB base station 9 includes a first UWB base station module, a second UWB base station module, a third UWB base station module, a UWB tag module, and a tracking control algorithm module. The first UWB base station module is connected to the UWB tag module, the second UWB base station module is connected to the UWB tag module, and the third UWB base station module is connected to the UWB tag module. The tracking control algorithm module uses a single-target path tracking control algorithm to optimize and adjust the path of the wheelchair trolley.

[0053] In this embodiment, the multi-sensor information fusion module is used to perform fusion processing on the data information from multiple sensors using a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm, including:

[0054] M1. Acquire data from multiple sensors, integrate and process the data using a genetic algorithm, and establish a fitness function G. i ,

[0055] , ,

[0056] Among them, α j Let F be the fitness factor, F be the probability function, f be the fitness function, and x be the fitness factor. j Given data from multiple sensors, where N is the sample size and i and j are constant parameters, we obtain the integrated data from the multiple sensors.

[0057] M2. Based on the integrated multi-sensor data, a membership function H is used for fuzzy logic reasoning.

[0058] ,

[0059] Where u represents the integrated multi-sensor data, v represents the fuzzy set data from the multi-sensor dataset, and H... A→B(u,v) is the confidence function, which yields the fuzzy matrix data information from multiple sensors;

[0060] M3. Input the fuzzy matrix data information of the multi-sensor into the neural network algorithm to fuse the data and obtain the fused multi-sensor data information.

[0061] In this embodiment, in step M2, the confidence function H A→B (u,v) is,

[0062] ,

[0063] Where a, b, and c are constant parameters, u is the integrated multi-sensor data information, and v is the fuzzy set data information of the multi-sensor system.

[0064] In this embodiment, step M3, which involves inputting the fuzzy matrix data information from the multiple sensors into a neural network algorithm for data fusion, includes:

[0065] M31. Based on the fuzzy matrix data information from the multiple sensors, establish the activation function Q of the neural network.

[0066] ,

[0067] ,

[0068] Where, q i For the fuzzy matrix data information of the i-th multi-sensor, β i Here, θ represents the corresponding weight coefficient, and θ is the activation factor.

[0069] M32. Based on the activation function Q of the neural network, the fused multi-sensor data information is obtained.

[0070] Example 2: Based on the intelligent assisted travel system based on multi-sensor information fusion and health detection in Example 1, the present invention will be further described and explained below.

[0071] like Figure 1 or Figure 2 or Figure 3 or Figure 4 As shown, an intelligent assisted travel system based on multi-sensor information fusion and health detection includes a wheelchair trolley 1, an autonomous driving controller 11, a UWB base station 9, and a health detector 6.

[0072] like Figure 6As shown, the autonomous driving controller 11 includes a multi-sensor module, a VCU module 12, and a multi-sensor information fusion module. The multi-sensor module is connected to the multi-sensor information fusion module, and the multi-sensor information fusion module is connected to the VCU module. The multi-sensor information fusion module is used to fuse the data information of the multiple sensors using a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm to obtain fused multi-sensor data information.

[0073] The UWB base station 9 includes a first UWB base station module, a second UWB base station module, a third UWB base station module, a UWB tag module, and a tracking control algorithm module. The first UWB base station module is connected to the UWB tag module, the second UWB base station module is connected to the UWB tag module, and the third UWB base station module is connected to the UWB tag module. The tracking control algorithm module uses a single-target path tracking control algorithm to optimize and adjust the path of the wheelchair trolley.

[0074] In this embodiment, the tracking control algorithm module uses a single-target path tracking control algorithm to optimize and adjust the path of the wheelchair vehicle, including:

[0075] L1. Obtain the distance data between the first UWB base station and the UWB tag module, the distance data between the second UWB base station and the UWB tag module, and the distance data between the third UWB base station and the UWB tag module, and establish a multimodal path planning function P.

[0076] ,

[0077] Where ω1 is the weighting coefficient of the first UWB base station, ω2 is the weighting coefficient of the second UWB base station, ω3 is the weighting coefficient of the third UWB base station, and l 1i This refers to the distance data between the first UWB base station and the UWB tag module. 2i For the distance data between the second UWB base station and the UWB tag module, l 3i ρ1 represents the distance data between the third UWB base station and the UWB tag module, ρ2 represents the relationship function between the first UWB base station and the second UWB base station, and ρ2 represents the relationship function between the second UWB base station and the third UWB base station.

[0078] L2. Based on the multimodal path planning function P, data information for multiple planned paths is obtained;

[0079] L3. Based on the data information of the multiple planned paths, establish a path evaluation and optimization function V.

[0080] ,

[0081] Where n is the total number of samples, P0 is the mean of the multiple planned paths, and P j Given the data information for the j-th planned path, we obtain the optimized path data information for the wheelchair vehicle.

[0082] In this embodiment, as Figure 5 As shown, the wheelchair trolley includes a body, a power battery 15, a drive motor 13, a motor controller 14, a rocker arm 2, and a display screen 7. The power battery 15 provides power to the wheelchair trolley. The wheelchair trolley 1 relies on the drive motor 13 and the motor controller 14 to achieve driving, steering, and braking. The rocker arm 2 controls the vehicle's forward and backward movement, acceleration and deceleration, and left and right steering by moving it forward, backward, left, and right. The display screen 7 displays the wheelchair trolley's real-time speed, gear, and battery level information.

[0083] In this embodiment, the health detector 6 monitors the user's basic health information such as blood pressure, body temperature, and respiratory rate, making it convenient for medical staff and caregivers to understand the user's basic health status in a timely manner.

[0084] In this embodiment, the multi-sensor module includes a lidar 5, a front-view camera 4, a rear-view camera 3, and an IMU 10. The lidar 5 is used to acquire point cloud data of the surrounding environment with distance and angle information, and the IMU 10 acquires the attitude and acceleration data of the wheelchair trolley 1 in real time.

[0085] In this embodiment, the system also includes an MCU, which is connected to the UWB base station 9 and is used to receive path optimization data information output by the UWB base station 9, and control the rotation of the left and right motors of the wheelchair trolley 1 to achieve automatic following.

[0086] In this embodiment, the system further includes a voice interaction module, which is used for voice wake-up, voice search, and reading audiobooks, providing a voice companion experience.

[0087] In summary, this invention not only provides intelligent assisted travel tools with human-machine co-driving capabilities, improving safety and reliability, but also enhances the automatic following ability of intelligent assisted travel tools, improving the intelligent following and safety monitoring needs of companions and caregivers.

[0088] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An intelligent assisted travel system based on multi-sensor information fusion and health detection, comprising a wheelchair trolley, an autonomous driving controller, a UWB base station, and a health detector, characterized in that: The autonomous driving controller includes a multi-sensor module, a VCU module, and a multi-sensor information fusion module. The multi-sensor module is connected to the multi-sensor information fusion module, and the multi-sensor information fusion module is connected to the VCU module. The multi-sensor information fusion module is used to fuse the data information from multiple sensors using a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm to obtain fused multi-sensor data information. The fused multi-sensor data information is then sent to the motor controller via the VCU to control the wheelchair trolley. The UWB base station includes a first UWB base station module, a second UWB base station module, a third UWB base station module, a UWB tag module, and a tracking control algorithm module. The first UWB base station module is connected to the UWB tag module, the second UWB base station module is connected to the UWB tag module, and the third UWB base station module is connected to the UWB tag module. The tracking control algorithm module uses a single-target path tracking control algorithm to optimize and adjust the path of the wheelchair trolley. The multi-sensor information fusion module is used to fuse data from multiple sensors using a neural network algorithm coupled with a genetic algorithm and a fuzzy logic reasoning algorithm, including: M1. Acquire data from multiple sensors, integrate and process the data using a genetic algorithm, and establish a fitness function G. i , , , Among them, α j Let F be the fitness factor, F be the probability function, f be the fitness function, and x be the fitness factor. j Given data from multiple sensors, where N is the sample size and i and j are constant parameters, we obtain the integrated data from the multiple sensors. M2. Based on the integrated multi-sensor data, a membership function H is used for fuzzy logic reasoning. ， Where u represents the integrated multi-sensor data, v represents the fuzzy set data from the multi-sensor dataset, and H... A→B (u,v) is the confidence function, which yields the fuzzy matrix data information from multiple sensors; M3. Input the fuzzy matrix data information of the multi-sensor into the neural network algorithm to fuse the data and obtain the fused multi-sensor data information.

2. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that, In step M2, the confidence function H A→B (u,v) is, ， Where a, b, and c are constant parameters, u is the integrated multi-sensor data information, and v is the fuzzy set data information of the multi-sensor system.

3. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that, In step M3, the step of inputting the fuzzy matrix data information from the multi-sensor neural network algorithm for data fusion includes: M31. Based on the fuzzy matrix data information from the multiple sensors, establish the activation function Q of the neural network. ， ， Where, q i For the fuzzy matrix data information of the i-th multi-sensor, β i Here, θ represents the corresponding weight coefficient, and θ is the activation factor. M32. Based on the activation function Q of the neural network, the fused multi-sensor data information is obtained.

4. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that, The tracking control algorithm module uses a single-target path tracking control algorithm to optimize and adjust the path of the wheelchair vehicle, including: L1. Obtain the distance data between the first UWB base station and the UWB tag module, the distance data between the second UWB base station and the UWB tag module, and the distance data between the third UWB base station and the UWB tag module, and establish a multimodal path planning function P. ， Where ω1 is the weighting coefficient of the first UWB base station, ω2 is the weighting coefficient of the second UWB base station, ω3 is the weighting coefficient of the third UWB base station, and l 1i This refers to the distance data between the first UWB base station and the UWB tag module. 2i For the distance data between the second UWB base station and the UWB tag module, l 3i ρ1 represents the distance data between the third UWB base station and the UWB tag module, ρ2 represents the relationship function between the first UWB base station and the second UWB base station, and ρ2 represents the relationship function between the second UWB base station and the third UWB base station. L2. Based on the multimodal path planning function P, data information for multiple planned paths is obtained; L3. Based on the data information of the multiple planned paths, establish a path evaluation and optimization function V. ， Where n is the total number of samples, P0 is the mean of the multiple planned paths, and P j Given the data information for the j-th planned path, we obtain the optimized path data information for the wheelchair vehicle.

5. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that: The wheelchair trolley includes a body, a power battery, a drive motor, a motor controller, a rocker arm, and a display screen. The power battery provides power to the wheelchair trolley. The wheelchair trolley relies on the drive motor and the motor controller to achieve driving, steering, and braking. The rocker arm controls the vehicle's forward and backward movement, acceleration and deceleration, and left and right steering by moving it forward, backward, left, and right. The display screen shows the wheelchair trolley's real-time speed, gear, and battery level information.

6. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that: The health detector monitors the user's basic health information, such as blood pressure, body temperature, and respiratory rate, making it convenient for medical staff and caregivers to understand the user's basic health status in a timely manner.

7. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that: The multi-sensor module includes a lidar, a front-view camera, a rear-view camera, and an IMU. The lidar is used to acquire point cloud data of the surrounding environment with distance and angle information, and the IMU acquires the attitude and acceleration data of the wheelchair vehicle in real time.

8. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that: The system also includes an MCU, which is connected to the UWB base station and is used to receive path optimization data information output by the UWB base station and control the rotation of the left and right motors of the wheelchair trolley to achieve automatic following.

9. The intelligent assisted travel system based on multi-sensor information fusion and health detection according to claim 1, characterized in that: The system also includes a voice interaction module, which is used for voice wake-up, voice search, and reading audiobooks, providing a voice companion experience.

Images