Intelligent obstacle avoidance system of walking assisting device for disabled people

Abstract

The invention discloses an intelligent obstacle avoidance system of an auxiliary walking device for disabled people, and belongs to the technical field of intelligent obstacle avoidance. A multi-type sensor module of the intelligent obstacle avoidance system comprises a laser radar, an ultrasonic sensor array and a visual sensor and is used for collecting environmental data. And the data fusion processing unit fuses and processes data and constructs a map model by means of high-performance hardware and software based on algorithms such as Kalman filtering and the like. And the obstacle avoidance decision and path planning module plans an obstacle avoidance path by adopting an improved A * algorithm and performs tracking control. And the user interaction and reminding module interacts with a user through voice, vibration and a visual display interface. According to the system, multiple sensors are fused to realize accurate environment perception, an intelligent algorithm guarantees efficient obstacle avoidance and path planning, diversified interaction improves user experience, the defects of an existing auxiliary walking device obstacle avoidance technology in the aspects of sensor fusion, environment perception, path planning and user interaction are effectively overcome, and the user experience is improved. And a safer, more intelligent and more convenient guarantee is provided for the handicapped to go out.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent obstacle avoidance, and in particular to an intelligent obstacle avoidance system for a walking aid for disabled persons. Background Art

[0002] In today's society, care for the disabled has received much attention, and the application of walking assistive devices for the disabled has become more and more widespread. However, such devices currently still have many defects in the field of obstacle avoidance.

[0003] The obstacle avoidance technologies used in existing walking assistance devices for the disabled vary. Some devices are only equipped with a single sensor for obstacle avoidance detection. For example, those that rely solely on ultrasonic sensors can detect obstacles at close range, but due to the ultrasonic propagation characteristics, the detection range is limited and the accuracy is poor. It is very easy to miss or misjudge obstacles that are slightly farther away, have strange shapes, or are small in size, making it difficult to effectively avoid obstacles in complex environments.

[0004] Some other devices use laser radar, which has certain advantages in detection accuracy and range, but cannot accurately identify and classify objects such as pedestrians and traffic signs that contain specific semantic information. This makes it difficult to formulate practical and accurate obstacle avoidance strategies in actual traffic scenarios or crowded areas, posing a hidden danger to the safe travel of people with disabilities.

[0005] In addition, existing obstacle avoidance systems also have deficiencies in path planning. They often fail to fully consider the motion characteristics of the disabled person's walking assistive device. For example, factors such as the turning radius, speed limit, and climbing ability of the device are not properly included in the path planning considerations. As a result, the device may encounter operational jams while moving, or even be unable to travel along the planned path, which will undoubtedly have a negative impact on the disabled person's experience and walking safety.

[0006] In addition, the user interaction experience is also a shortcoming of existing technologies. Most devices can only provide simple and single sound prompts, and cannot provide detailed and accurate guidance to disabled people based on the degree of danger or environmental changes. At the same time, the lack of convenient visual interactive interfaces and diversified interactive methods makes it difficult for disabled people to fully understand the surrounding environment, and they cannot independently adjust and intervene in obstacle avoidance strategies and walking paths. This greatly limits the practicality and intelligence level of auxiliary walking devices.

[0007] In summary, the existing obstacle avoidance technology for assistive walking devices for the disabled has significant limitations in many core aspects such as sensor fusion, environmental perception, path planning, and user interaction. In view of this, there is an urgent need for a more advanced, complete, and intelligent obstacle avoidance system to improve the safety, adaptability, and user experience of assistive walking devices for the disabled. Summary of the invention

[0008] The main purpose of the present invention is to provide an intelligent obstacle avoidance system for a disabled person's walking assistance device, which can effectively solve the problems mentioned in the background technology.

[0009] To achieve the above object, the technical solution adopted by the present invention is:

[0010] An intelligent obstacle avoidance system for a disabled person's walking aid device, the intelligent obstacle avoidance system comprises a multi-type sensor module, a data fusion processing unit, an obstacle avoidance decision and path planning module, and a user interaction and reminder module; wherein:

[0011] The multi-type sensor module comprises:

[0012] LiDAR sensors are installed in front, on both sides and at the rear of the walking aid, and can scan the surrounding environment 360 degrees and obtain high-precision distance information and environmental profile data;

[0013] Ultrasonic sensor arrays, distributed around the device, are used for close-range obstacle detection and to supplement lidar blind spot monitoring;

[0014] The visual sensor is installed in front of the auxiliary walking device to obtain real-time image information of the front environment and identify obstacle features and specific targets;

[0015] The data fusion processing unit has an embedded processor and a memory chip as its hardware part, and a multi-sensor data fusion algorithm and an environment modeling and map building algorithm as its software part;

[0016] The obstacle avoidance decision and path planning module comprises:

[0017] Intelligent obstacle avoidance algorithm, based on A* algorithm, combines environmental model and obstacle information to search for the optimal obstacle avoidance path on the map and considers the walking characteristics of disabled people and the motion characteristics of auxiliary walking devices. It also introduces dynamic obstacle avoidance strategies and presets a library of multiple obstacle avoidance strategies.

[0018] The path tracking and control unit generates control instructions based on the planned obstacle avoidance path to control the action of the auxiliary walking device drive system and uses closed-loop control technology to correct walking deviations in real time based on sensor feedback;

[0019] The user interaction and reminder module includes: a voice reminder system, which integrates speech synthesis and recognition functions, issues voice reminders to users based on obstacle avoidance conditions and receives user voice command interactions;

[0020] A vibration reminder device is installed on the handrails of the walking aid, the seat and the contact part of the user's body, and generates vibration reminders of different intensities and frequencies according to the degree of danger;

[0021] The visual display interface is a touch screen, which displays the surrounding environment map, obstacle distribution, obstacle avoidance path planning, and the current position and motion state information of the device, and supports user manual operation interaction;

[0022] The control method of the intelligent obstacle avoidance system includes the following steps:

[0023] S1. Sensor data acquisition and preprocessing step: After the system starts, the lidar sensor, ultrasonic sensor array, and vision sensor collect the surrounding environment data, and perform data cleaning, filtering, and noise reduction on the collected raw data, including performing point cloud filtering on lidar data, smoothing ultrasonic sensor data, and performing image enhancement and grayscale transformation on vision sensor images;

[0024] S2. Multi-sensor data fusion and environment modeling step: Input the preprocessed sensor data into the data fusion processing unit, perform fusion processing through the multi-sensor data fusion algorithm, and use the environment modeling and map construction algorithm to construct the surrounding environment map model of the assistive walking device based on the fused data, identify and mark the position and shape parameters of obstacles, divide the passable area and non-passable area, and mark special areas and important landmark information, and store and update the map data in real time;

[0025] S3. Obstacle avoidance decision-making and path planning step: According to the current position and target position information, combined with the environment map model and obstacle distribution, the obstacle avoidance decision-making and path planning module uses the intelligent obstacle avoidance algorithm to search for the optimal obstacle avoidance path, considers the walking characteristics of disabled people and the motion characteristics of the assistive walking device, and selects a strategy from the preset obstacle avoidance strategy library according to different traffic scenarios and obstacle types, sends the planned obstacle avoidance path information to the path tracking and control unit and displays it on the visual display interface, and at the same time broadcasts it to the user through the voice reminder system;

[0026] S4. Path tracking and control step: The path tracking and control unit generates control instructions according to the obstacle avoidance path information to control the action of the drive system of the assistive walking device, and uses sensors to monitor the actual position and motion state of the device in real time, compares and analyzes the deviation information with the planned path, and uses closed-loop control technology to adjust the control instructions according to the deviation to correct the walking deviation;

[0027] S5. User interaction and reminder step: The user interacts with the system through voice commands or by touching the visual display interface, queries relevant information, manually adjusts the destination or obstacle avoidance strategy. When the system detects obstacles or dangerous situations in the surrounding environment, corresponding reminder information is sent to the user through the voice reminder system, vibration reminder device, and visual display interface according to the degree of danger.

[0028] Preferably, by emitting a laser beam and receiving a reflected light signal, the distance information to the surrounding objects is calculated according to the time difference between the laser emission and the reception using the formula d=c×Δt / 2, where d represents the distance, c is the speed of light, and Δt is the time difference, data is collected at fixed time intervals and point cloud data of the surrounding environment is constructed, and then the point cloud data is transmitted to the data fusion processing unit;

[0029] The ultrasonic sensor array transmits ultrasonic pulse signals at a set frequency and receives reflected waves, and calculates the distance to the obstacle using the formula d=v×t / 2 according to the time interval t between the transmitted pulse and the received reflected pulse, where v is the propagation speed of ultrasonic waves in the air, and integrates the data of each sensor and transmits it to the data fusion processing unit;

[0030] The visual sensor collects image data of the front environment at a set frame rate, performs grayscale, filtering, and edge detection processing on the image in sequence, then uses a deep learning target detection algorithm to identify the target and extract feature information, and transmits the results of target identification and feature extraction to the data fusion processing unit.

[0031] Preferably, the embedded processor is connected to a memory chip, a power management module, and a communication interface circuit, and the communication interface circuit includes an Ethernet interface, a CAN bus interface, and a USB interface; the memory chip stores sensor acquisition data, map information, and system operation intermediate data and historical records, and performs partition management and data index establishment;

[0032] The multi-sensor data fusion algorithm is based on the Kalman filter algorithm, including data synchronization and time alignment, which are realized by hardware clock synchronization circuit or software timestamp marking; coordinate conversion and unification, using the coordinate transformation matrix to convert data in different coordinate systems to the world coordinate system according to the sensor installation position and posture information; data fusion, using the obstacle position and speed as state variables, the laser radar and ultrasonic sensor distance information as measurement values, and the visual sensor target feature information as auxiliary information for Kalman filter fusion estimation;

[0033] The environment modeling and map building algorithm includes:

[0034] Obstacle identification and marking: Based on the data fusion results, the obstacle position and shape range are determined according to the distance information of the lidar and ultrasonic sensors, and the obstacles are classified and marked in combination with the target type information of the visual sensor;

[0035] Map construction uses SLAM technology to mark the area where the obstacle is located as an inaccessible area and the rest as a accessible area. Environmental feature points are extracted as key node storage marks. Based on the motion trajectory information of the auxiliary walking device, the feature points at different times are associated and integrated to construct a two-dimensional grid map or a three-dimensional point cloud map, and the map information is updated in real time.

[0036] Preferably, the intelligent obstacle avoidance algorithm is improved based on the A* algorithm, including environmental information acquisition and analysis, and construction of an environmental representation model; target position setting and path search, based on user input or preset destination, using the evaluation function f(n)=g(n)+h(n) in the environmental graph model to search for the optimal obstacle avoidance path, wherein f(n) is the comprehensive evaluation value of node n, g(n) is the actual cost from the starting node to node n, h(n) is the heuristic estimated cost from node n to the target node, g(n) is calculated considering the motion characteristics of the auxiliary walking device, and h(n) is estimated using the Manhattan distance or Euclidean distance heuristic function, and avoidance rules and cost penalties are set for different types of obstacles; dynamic obstacle avoidance and path adjustment, real-time monitoring of environmental changes, and recalculation of the obstacle avoidance path based on the changes by combining the local path planning algorithm with the global path planning algorithm, and considering the dynamic characteristics of the auxiliary walking device to ensure smooth and safe path adjustment.

[0037] Preferably, the path tracking and control unit generates control instructions to control the action of the auxiliary walking device driving system according to the obstacle avoidance path planned by the intelligent obstacle avoidance algorithm, and the control instructions include the speed, steering angle and brake signal of the driving motor;

[0038] The encoders, gyroscopes and accelerometers installed on the auxiliary walking device monitor the actual position, speed and posture of the device in real time and provide feedback. The position deviation, direction deviation and speed deviation are calculated by comparing with the planned path. The closed-loop control technology is used to correct the control instructions based on the error information. The integral separation PID control and fuzzy PID control strategies are used to improve the control stability and accuracy.

[0039] Preferably, the voice reminder system initializes the speech synthesis engine and speech recognition module when the system is started. The speech synthesis engine adopts a speech synthesis model based on deep learning, and the speech recognition module is based on the speech recognition algorithm and model and is trained and optimized. Voice reminder information is generated and broadcast according to the operating status and environmental information of the obstacle avoidance system, and the user's voice commands are monitored in real time. After collection, preprocessing, recognition and analysis, the corresponding operation is performed according to the command type and the result is fed back.

[0040] Preferably, the vibration reminder device installs a vibration motor on the armrest or seat of the auxiliary walking device and connects the drive circuit to a data fusion processing unit; a vibration reminder strategy is formulated based on the distance and danger level of obstacles detected by the obstacle avoidance system, and the data fusion processing unit calculates the danger level based on the sensor data and sends a vibration control signal to the drive circuit to control the operation of the vibration motor.

[0041] Preferably, the visual display interface is a touch screen, and the interface layout includes a map display area, an information prompt area, and an operation button area; the map display area graphically displays the surrounding environment map model, obstacles and planned obstacle avoidance paths; the information prompt area displays the system status, obstacle avoidance prompts, location, and destination information; the operation button area sets query, setting, and destination input buttons, and users can touch to interact and view detailed environmental information, manually adjust the destination or obstacle avoidance strategy, and view system setting options and historical walking records.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention forms an intelligent obstacle avoidance system dedicated to the walking assistance device for the disabled by designing multiple types of sensor modules, data fusion processing units, obstacle avoidance decision and path planning modules, and user interaction and reminder modules. In terms of safety, multi-sensor fusion and intelligent algorithms achieve accurate and dynamic obstacle avoidance, greatly reducing the risk of collision. In terms of user experience, personalized interaction methods include voice, vibration and visual display, which facilitate the disabled to obtain information and control the device. At the same time, path planning that considers the characteristics of the device makes walking more comfortable and convenient. In terms of intelligence, intelligent decision-making enables the system to select appropriate strategies and follow rules according to different situations, and can continuously learn and optimize through data collection, with strong adaptability. Overall, it provides a safer, smarter, more comfortable and humane obstacle avoidance solution for the disabled. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The present invention is a schematic diagram of the composition architecture flow of an intelligent obstacle avoidance system of a disabled person's auxiliary walking device. DETAILED DESCRIPTION

[0045] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0046] like Figure 1 As shown, the intelligent obstacle avoidance system of the disabled walking assistance device includes multiple types of sensor modules, a data fusion processing unit, an obstacle avoidance decision and path planning module, and a user interaction and reminder module.

[0047] A multi-type sensor module is used to collect surrounding environment data, and the multi-type sensor module includes:

[0048] LiDAR sensors are installed in front, on both sides and at the rear of the walking aid, and can scan the surrounding environment 360 degrees and obtain high-precision distance information and environmental profile data;

[0049] Ultrasonic sensor arrays, distributed around the device, are used for close-range obstacle detection and to supplement lidar blind spot monitoring;

[0050] The visual sensor is installed in front of the auxiliary walking device to obtain real-time image information of the front environment and identify obstacle features and specific targets;

[0051] A data fusion processing unit, the hardware of which adopts a high-performance embedded processor and a large-capacity storage chip, and the software of which runs a multi-sensor data fusion algorithm and an environmental modeling and map building algorithm. The multi-sensor data fusion algorithm fuses the data of the laser radar, ultrasonic sensor and visual sensor based on the Kalman filter algorithm, and the environmental modeling and map building algorithm uses the fused data to build a two-dimensional or three-dimensional map model of the environment around the auxiliary walking device and updates it in real time;

[0052] Obstacle avoidance decision and path planning module, which includes:

[0053] Intelligent obstacle avoidance algorithm, based on the improvement of A* algorithm, combines the environmental model and obstacle information to search for the optimal obstacle avoidance path on the map and takes into account the walking characteristics of disabled people and the motion characteristics of auxiliary walking devices. At the same time, it introduces dynamic obstacle avoidance strategies and presets a variety of obstacle avoidance strategy libraries;

[0054] The path tracking and control unit generates control instructions based on the planned obstacle avoidance path to control the action of the auxiliary walking device drive system and uses closed-loop control technology to correct walking deviations in real time based on sensor feedback;

[0055] User interaction and reminder module, including:

[0056] The voice reminder system integrates speech synthesis and recognition functions, sends voice reminders to users based on obstacle avoidance conditions and receives user voice command interactions;

[0057] Vibration reminder device, installed on the handrails, seats and other parts of the auxiliary walking device that come into contact with the user's body, to generate vibration reminders of different intensities and frequencies according to the degree of danger;

[0058] The visual display interface is a touch screen that displays the surrounding environment map, obstacle distribution, obstacle avoidance path planning, and the current position and motion status information of the device, and supports manual operation and interaction by users.

[0059] Example 1: Obstacle avoidance application in indoor environment

[0060] Scene Setting

[0061] This embodiment is set in a typical indoor environment, such as a barrier-free passage area in a large shopping mall. This area has a large flow of people and various obstacles, including static pillars, shelves, trash cans, and moving pedestrians and shopping carts.

[0062] System initialization and startup

[0063] After the disabled person turns on the assistive walking device, the multi-type sensor modules start working. The lidar sensor emits a laser beam at a frequency of 10 times per second and receives the reflected light, quickly constructing the initial point cloud data of the surrounding environment. Its detection range covers an area of ​​10 meters in front of the device, 5 meters on both sides and 3 meters behind, with an accuracy of 0.1 meters. The 8 sensors of the ultrasonic sensor array are distributed on the bottom edge of the front side, the middle of both sides and the bottom of the rear side of the device, and periodically emit ultrasonic pulses at a frequency of 40kHz to detect obstacles at close range (0.5-3 meters). The visual sensor collects images of the front environment at a rate of 30 frames per second, with an image resolution of 1280×720 pixels.

[0064] Data collection and fusion processing

[0065] After the lidar sensor collects the point cloud data, it transmits the data to the data fusion processing unit through the CAN bus. For example, at a certain moment, a point cloud gathering area suspected of being an obstacle is detected 3 meters ahead. The ultrasonic sensor array works at the same time, and the sensor located on the left side of the front side of the device detects a small object (possibly a bump on the ground) 1 meter away. The image collected by the visual sensor is grayed, Gaussian filtered, and processed with Canny edge detection, and the YOLOv5 target detection algorithm is used to identify a pedestrian walking slowly 2 meters ahead.

[0066] After receiving these data, the data fusion processing unit first performs time synchronization and coordinate conversion. The data of the laser radar, ultrasonic sensor and visual sensor are unified into the world coordinate system with the auxiliary walking device as the origin. Then, data fusion is performed based on the Kalman filter algorithm. Based on the suspected obstacle position information detected by the laser radar, combined with the ultrasonic sensor's detection results of close-range objects and the visual sensor's recognition and feature extraction information of pedestrians, it is accurately determined that a pedestrian is approaching 2 meters ahead, and his walking path may intersect with the travel route of the auxiliary walking device. At the same time, it is determined that the suspected obstacle 3 meters ahead is a stationary trash can, and its position and shape information are accurately marked.

[0067] Obstacle avoidance decision and path planning

[0068] The obstacle avoidance decision and path planning module analyzes the fused environmental information. The intelligent obstacle avoidance algorithm is based on an improved version of the A* algorithm. Taking into account the minimum turning radius of the auxiliary walking device of 1 meter, the maximum speed of 0.5 meters per second, and the relative position relationship between the current position and the pedestrian and the trash can, the optimal obstacle avoidance path is searched in the constructed environmental graph model. Since the pedestrian is moving slowly, the algorithm predicts the movement trajectory of the pedestrian and plans a path that bypasses the left side of the pedestrian and avoids the trash can. The total length of the path is about 5 meters. It needs to drive 2 meters to the left front at a speed of 0.3 meters per second, then turn left with a turning radius of 1 meter, and then drive forward 3 meters.

[0069] Path tracking and control

[0070] The path tracking and control unit generates control instructions based on the planned obstacle avoidance path. The control instructions control the speed and steering angle of the drive motor through PWM signals. For example, in order to achieve driving to the left front, the left motor is driven to run at a higher speed and the right motor is driven to run at a lower speed, so that the device turns to the left front. At the same time, the actual driving distance and direction of the device are monitored in real time through the encoder installed on the drive wheel, and fed back to the path tracking and control unit. When it is found that the actual driving direction of the device deviates from the planned path, for example, the deviation angle reaches 5 degrees, the PID control algorithm is used to adjust the speed difference of the motor so that the device gradually returns to the correct path.

[0071] User interaction and reminders

[0072] The user interaction and reminder module plays an important role in the whole process. When the voice reminder system detects that a pedestrian is approaching, it will announce "There is a pedestrian approaching 2 meters ahead, please be careful to avoid him, and an obstacle avoidance path will be planned soon." When the obstacle avoidance path planning is completed and begins to be executed, it will announce "The obstacle avoidance path has been planned, please follow the guidance and drive to the left front." When the vibration reminder device is close to the trash can, it will produce a slight vibration (vibration intensity is level 3, frequency is 10Hz) to remind the user to pay attention to the surrounding environment. The visual display interface shows the location of pedestrians and trash cans, as well as the planned obstacle avoidance path in different colors and icons in the map display area. The information prompt area displays the current location, destination (assuming it is the mall exit) and system status information. Users can also view more detailed environmental information or manually adjust the destination through the touch operation interface.

[0073] Example 2: Obstacle avoidance application in outdoor complex traffic environment

[0074] Scene Setting

[0075] The scene of this embodiment is near a crossroads of a city street, where there are various traffic signs, vehicles traveling at different speeds, pedestrians crossing the road, parked vehicles on the roadside and other obstacles.

[0076] System initialization and startup

[0077] After the disabled person starts the assisted walking device, the multi-type sensor module starts working as in Example 1. The detection range of the lidar sensor is adjusted to 15 meters in front, 8 meters on both sides and 5 meters in the back, and the accuracy is still 0.1 meters to adapt to the wider outdoor environment. The 10 sensors of the ultrasonic sensor array are distributed around the device to continuously monitor the close-range situation. The visual sensor collects the front image at 30 frames per second with a resolution of 1920×1080 pixels to more clearly identify traffic signs and vehicles in the distance.

[0078] Data collection and fusion processing

[0079] The lidar sensor scanned a car that was slowing down to stop 8 meters ahead, and detected a traffic light pole 5 meters to the right. The ultrasonic sensor array did not detect any sudden obstacles at close range. The images collected by the visual sensor were processed to identify the type of car in front (sedan), the state of the traffic light (red light), and the pedestrians waiting to cross the road on the opposite side of the road.

[0080] The data fusion processing unit fuses these data. It combines the distance data of the lidar and ultrasonic sensors with the target information identified by the visual sensor to determine the exact position and motion status of the car, as well as the position information of the traffic lights and pedestrians, and marks them in a unified coordinate system. For example, it is clear that the length of the car is 4 meters, the width is 1.8 meters, the current speed is 0.2 meters / second and is slowing down, the position coordinates of the traffic light pole are 5 meters to the right and 10 meters ahead, and the pedestrian is 12 meters ahead across the road.

[0081] Obstacle avoidance decision and path planning

[0082] The obstacle avoidance decision and path planning module makes decisions based on environmental information and traffic rules. Since the traffic light is red and the car is slowing down to stop, the intelligent obstacle avoidance algorithm plans a path to bypass the rear of the car and then safely cross the road to the opposite sidewalk during the red light. Considering the motion characteristics of the auxiliary walking device, such as the maximum speed and turning radius, the path is planned to ensure that the device can smoothly bypass the car and keep a safe distance from pedestrians when crossing the road. For example, first drive 3 meters to the right front at a speed of 0.4 meters / second to bypass the rear of the car, and then when it is confirmed that there are no vehicles on the left and right sides and pedestrians start to cross the road, cross the road in a straight line at a speed of 0.3 meters / second. The total path length is about 10 meters.

[0083] Path tracking and control

[0084] The path tracking and control unit generates corresponding control instructions to control the drive system. According to the planned path, the motor speed and steering angle are controlled to make the device travel along the predetermined trajectory. During the driving process, the inertial measurement unit (IMU) and encoder monitor the device's posture (such as tilt angle) and driving distance, direction and other information in real time. When the device is bypassing the car, the device has a certain tilt due to the uneven ground. The IMU detects that the tilt angle is 3 degrees. The path tracking and control unit adjusts the motor output power according to the tilt angle to ensure that there is enough friction between the drive wheel and the ground to make the device travel stably. At the same time, compared with the planned path, if there is a deviation, such as a 3-degree deviation in the driving direction, the fuzzy PID control algorithm is used to correct the deviation in time to ensure that the device accurately tracks the path.

[0085] User interaction and reminders

[0086] When the voice reminder system detects a car and a traffic light, it will announce "A car is slowing down and stopping 8 meters ahead, and the traffic light is red. Please wait for the right time to cross the road." When the obstacle avoidance path is planned, it will announce "The path to cross the road has been planned. Please follow the guidance and go around the car to the right and front first." When approaching a car or about to cross the road, the vibration reminder device will generate vibrations of different intensities and frequencies according to the degree of danger. For example, when approaching a car, the vibration intensity is level 2 and the frequency is 8Hz, reminding the user to pay attention. The visual display interface shows the location and status of cars, traffic lights, pedestrians, etc. in the map display area, as well as the planned obstacle avoidance path. The information prompt area shows the current location, destination (assuming it is the bus stop across the street) and system status information. Users can view detailed environmental information, adjust the destination or obstacle avoidance strategy through touch operation, such as choosing to wait for the next green light cycle before crossing the road.

[0087] Through the above two embodiments, it can be seen that the intelligent obstacle avoidance system of the disabled person's auxiliary walking device can effectively collect environmental data, perform data fusion, make reasonable obstacle avoidance decisions and path planning in different environments, and ensure the safe walking of disabled people through precise path tracking and control as well as good user interaction and reminders, thereby improving their autonomy of movement and user experience.

[0088] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An intelligent obstacle avoidance system for a disabled person's walking assistance device, characterized in that: The intelligent obstacle avoidance system includes multi-type sensor modules, data fusion processing unit, obstacle avoidance decision and path planning module and user interaction and reminder module; among which: The multi-type sensor module comprises: LiDAR sensors are installed in front, on both sides and at the rear of the walking aid, and can scan the surrounding environment 360 degrees and obtain high-precision distance information and environmental profile data; Ultrasonic sensor arrays, distributed around the device, are used for close-range obstacle detection and to supplement lidar blind spot monitoring; The visual sensor is installed in front of the auxiliary walking device to obtain real-time image information of the front environment and identify obstacle features and specific targets; The data fusion processing unit has an embedded processor and a memory chip as its hardware part, and a multi-sensor data fusion algorithm and an environment modeling and map building algorithm as its software part; The obstacle avoidance decision and path planning module comprises: Intelligent obstacle avoidance algorithm, based on A* algorithm, combines environmental model and obstacle information to search for the optimal obstacle avoidance path on the map and considers the walking characteristics of disabled people and the motion characteristics of auxiliary walking devices. It also introduces dynamic obstacle avoidance strategies and presets a variety of obstacle avoidance strategy libraries; The path tracking and control unit generates control instructions based on the planned obstacle avoidance path to control the action of the auxiliary walking device drive system and uses closed-loop control technology to correct walking deviations in real time based on sensor feedback; The user interaction and reminder module includes: a voice reminder system, which integrates speech synthesis and recognition functions, issues voice reminders to users based on obstacle avoidance conditions and receives user voice command interactions; A vibration reminder device is installed on the handrails of the walking aid, the seat and the contact part of the user's body, and generates vibration reminders of different intensities and frequencies according to the degree of danger; The visual display interface is a touch screen that displays the surrounding environment map, obstacle distribution, obstacle avoidance path planning, and the current position and motion status of the device, and supports manual operation and interaction by users; The control method of the intelligent obstacle avoidance system comprises the following steps: S1, sensor data collection and preprocessing step, after the system is started, the laser radar sensor, ultrasonic sensor array and visual sensor collect surrounding environment data, and perform data cleaning, filtering and noise reduction on the collected raw data, including point cloud filtering on the laser radar data, smoothing on the ultrasonic sensor data, and image enhancement and grayscale transformation on the visual sensor image; S2, multi-sensor data fusion and environment modeling step, inputting the pre-processed sensor data into the data fusion processing unit, performing fusion processing through the multi-sensor data fusion algorithm, and constructing a map model of the environment around the auxiliary walking device based on the fused data using the environment modeling and map construction algorithm, identifying and marking the location and shape parameters of obstacles, dividing the passable area and the impassable area and marking the special area and important landmark information, and storing and updating the map data in real time; S3, obstacle avoidance decision and path planning step, according to the current position and target position information, combined with the environmental map model and obstacle distribution, the obstacle avoidance decision and path planning module uses an intelligent obstacle avoidance algorithm to search for the optimal obstacle avoidance path, considering the walking characteristics of the disabled and the motion characteristics of the auxiliary walking device, and selecting a strategy from the preset obstacle avoidance strategy library according to different traffic scenes and obstacle types, sending the planned obstacle avoidance path information to the path tracking and control unit and displaying it on the visual display interface, and broadcasting it to the user through the voice reminder system; S4, path tracking and control step, the path tracking and control unit generates a control command based on the obstacle avoidance path information to control the action of the auxiliary walking device drive system, monitors the actual position and motion state of the device in real time through sensors, compares and analyzes the deviation information with the planned path, and uses closed-loop control technology to adjust the control command according to the deviation to correct the walking deviation; S5, user interaction and reminder step, the user interacts with the system through voice commands or touching the visual display interface to query relevant information, manually adjust the destination or obstacle avoidance strategy. When the system detects obstacles or dangerous conditions in the surrounding environment, it sends corresponding reminder information to the user through the voice reminder system, vibration reminder device and visual display interface according to the degree of danger.

2. The intelligent obstacle avoidance system for a disabled person's walking assistance device according to claim 1, characterized in that: The method transmits a laser beam and receives a reflected light signal, calculates the distance information to the surrounding objects according to the time difference between the laser emission and the reception using the formula d=c×Δt / 2, where d represents the distance, c is the speed of light, and Δt is the time difference, collects data at fixed time intervals and constructs point cloud data of the surrounding environment, and then transmits the point cloud data to the data fusion processing unit; The ultrasonic sensor array transmits ultrasonic pulse signals at a set frequency and receives reflected waves, and calculates the distance to the obstacle using the formula d=v×t / 2 according to the time interval t between the transmitted pulse and the received reflected pulse, where v is the propagation speed of ultrasonic waves in the air, and integrates the data of each sensor and transmits it to the data fusion processing unit; The visual sensor collects image data of the front environment at a set frame rate, performs grayscale, filtering, and edge detection processing on the image in sequence, then uses a deep learning target detection algorithm to identify the target and extract feature information, and transmits the results of target identification and feature extraction to the data fusion processing unit.

3. The intelligent obstacle avoidance system for a disabled person's walking assistance device according to claim 1, characterized in that: The embedded processor is connected to the memory chip, the power management module and the communication interface circuit, and the communication interface circuit includes an Ethernet interface, a CAN bus interface and a USB interface; the memory chip stores sensor collected data, map information and system operation intermediate data and historical records, and performs partition management and data index establishment; The multi-sensor data fusion algorithm is based on the Kalman filter algorithm, including data synchronization and time alignment, which is achieved through hardware clock synchronization circuit or software time stamping; Coordinate conversion and unification: using the coordinate transformation matrix to convert data in different coordinate systems to the world coordinate system based on the sensor installation position and attitude information; data fusion: using the obstacle position and speed as state variables, the laser radar and ultrasonic sensor distance information as measurement values, and the visual sensor target feature information as auxiliary information for Kalman filter fusion estimation; The environment modeling and map building algorithm includes: Obstacle identification and marking: Based on the data fusion results, the obstacle position and shape range are determined according to the distance information of the lidar and ultrasonic sensors, and the obstacles are classified and marked in combination with the target type information of the visual sensor; Map construction uses SLAM technology to mark the area where the obstacle is located as an inaccessible area and the rest as a accessible area. Environmental feature points are extracted as key node storage marks. Based on the motion trajectory information of the auxiliary walking device, the feature points at different times are associated and integrated to construct a two-dimensional grid map or a three-dimensional point cloud map, and the map information is updated in real time.

4. The intelligent obstacle avoidance system for a disabled person's walking assistance device according to claim 1, characterized in that: The intelligent obstacle avoidance algorithm is improved based on the A* algorithm, including environmental information acquisition and analysis, and building an environmental representation model; target position setting and path search, based on user input or preset destination, using the evaluation function f(n)=g(n)+h(n) in the environmental graph model to search for the optimal obstacle avoidance path, wherein f(n) is the comprehensive evaluation value of node n, g(n) is the actual cost from the starting node to node n, h(n) is the heuristic estimated cost from node n to the target node, g(n) is calculated considering the motion characteristics of the auxiliary walking device, h(n) is estimated using the Manhattan distance or Euclidean distance heuristic function, and avoidance rules and cost penalties are set for different types of obstacles; dynamic obstacle avoidance and path adjustment, real-time monitoring of environmental changes, and recalculation of the obstacle avoidance path based on the changes by combining the local path planning algorithm with the global path planning algorithm, and considering the dynamic characteristics of the auxiliary walking device to ensure smooth and safe path adjustment.

5. The intelligent obstacle avoidance system for a disabled person's walking assistance device according to claim 4, characterized in that: The path tracking and control unit generates control instructions to control the action of the auxiliary walking device drive system according to the obstacle avoidance path planned by the intelligent obstacle avoidance algorithm, and the control instructions include the speed, steering angle and brake signal of the drive motor; The encoders, gyroscopes and accelerometers installed on the auxiliary walking device monitor the actual position, speed and posture of the device in real time and provide feedback. The position deviation, direction deviation and speed deviation are calculated by comparing with the planned path. The closed-loop control technology is used to correct the control instructions based on the error information. The integral separation PID control and fuzzy PID control strategies are used to improve the control stability and accuracy.

6. The intelligent obstacle avoidance system for a disabled person's walking assistance device according to claim 1, characterized in that: The voice reminder system initializes the speech synthesis engine and speech recognition module when the system is started. The speech synthesis engine adopts a speech synthesis model based on deep learning, and the speech recognition module is based on the speech recognition algorithm and model and is trained and optimized. It generates and broadcasts voice reminder information based on the operating status and environmental information of the obstacle avoidance system, and monitors the user's voice commands in real time. After collection, preprocessing, recognition and analysis, it executes corresponding operations according to the command type and feeds back the results.

7. The intelligent obstacle avoidance system for a disabled person's walking assistance device according to claim 6, characterized in that: The vibration reminder device installs the vibration motor on the armrest and seat of the auxiliary walking device and connects the drive circuit to the data fusion processing unit; A vibration reminder strategy is formulated based on the distance and danger level of obstacles detected by the obstacle avoidance system. The data fusion processing unit calculates the danger level based on the sensor data and sends a vibration control signal to the drive circuit to control the operation of the vibration motor.

8. The intelligent obstacle avoidance system for a disabled person's walking assistance device according to claim 7, characterized in that: The visual display interface is a touch-screen display, and the interface layout includes a map display area, an information prompt area, and an operation button area; the map display area graphically displays the surrounding environment map model, obstacles, and planned obstacle avoidance paths; the information prompt area displays the system status, obstacle avoidance prompts, location, and destination information; the operation button area sets query, setting, and destination input buttons, and users can touch to interact and view detailed environmental information, manually adjust the destination or obstacle avoidance strategy, and view system setting options and historical walking records.

Images