Multi-mode navigation control method and system of electric wheelchair controller

Abstract

The invention relates to the technical field of electric wheelchair control, in particular to a multi-mode navigation control method and system of an electric wheelchair controller. The method comprises the following steps: acquiring a real-time environment monitoring video based on an electric wheelchair high-definition camera; video jitter distortion compensation and dynamic object frame-by-frame continuous optical flow tracking are carried out on the real-time environment monitoring video, and a moving track of each dynamic object is generated; performing environment static obstacle accurate identification on the real-time environment monitoring video, and performing dynamic environment perception modeling based on a moving track of each dynamic object so as to generate a dynamic environment map; a navigation mode switching decision is made based on a real-time wheelchair operation instruction of the user to obtain a current navigation mode of the user, and the current navigation mode of the user comprises a rocker control mode and an intelligent navigation mode. According to the invention, efficient and intelligent multi-mode control of the electric wheelchair is realized, so that the electric wheelchair can more flexibly and safely cope with a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of electric wheelchair control, and in particular to a multi-mode navigation control method and system for an electric wheelchair controller. Background Art

[0002] As an important auxiliary mobility tool, electric wheelchairs are widely used in the daily lives of the elderly, disabled people and people with limited mobility. With the continuous advancement of science and technology and the increasing attention of society to the aging problem, electric wheelchairs, while providing convenience and comfort, also face increasingly complex usage requirements. Especially in complex environments, how to safely and efficiently guide electric wheelchairs to perform autonomous navigation has become a key issue in improving the intelligence level of electric wheelchairs.

[0003] Traditional electric wheelchair navigation control methods mainly rely on simple manual operations or preset path planning, and lack the ability to perceive and adapt to environmental changes in real time. Although this method can achieve basic navigation functions in relatively simple environments, it is prone to collisions, jamming, or taking the wrong path in dynamic environments, especially in complex indoor and outdoor scenes full of obstacles. In addition, existing electric wheelchair controllers usually only provide a single operation mode, such as joystick control mode or simple intelligent navigation mode, but in different usage scenarios, users often need to switch and work together with multiple control modes.

[0004] With the development of artificial intelligence technology and sensor technology, the intelligence level of electric wheelchairs has been continuously improved. In recent years, electric wheelchair navigation systems based on sensor data fusion, machine learning algorithms and environmental perception technology have gradually entered the market, which can achieve more efficient path planning and collision avoidance functions. However, the existing intelligent electric wheelchair navigation systems often have problems such as complex operation and lack of flexibility. Especially when facing sudden obstacles or rapidly changing environments, the existing multi-mode switching function is still not smart enough and cannot adapt to user needs and environmental changes in real time.

[0005] Therefore, in order to further improve the adaptability and intelligence level of electric wheelchairs, a new multi-mode navigation control method for electric wheelchair controllers is needed. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a multi-mode navigation control method and system for an electric wheelchair controller to solve at least one of the above technical problems.

[0007] To achieve the above object, the present invention provides a multi-mode navigation control method for an electric wheelchair controller, comprising the following steps: Step S1: acquiring a real-time environment monitoring video based on a high-definition camera of the electric wheelchair; performing video jitter distortion compensation and frame-by-frame continuous optical flow tracking of dynamic objects on the real-time environment monitoring video to generate a moving trajectory of each dynamic object; Step S2: accurately identifying static obstacles in the environment on the real-time environment monitoring video, and performing dynamic environment perception modeling based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map; Step S3: identifying the user's real-time wheelchair operation instructions; making navigation mode switching decisions based on the user's real-time wheelchair operation instructions to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode; Step S4: When the user's current navigation mode is the intelligent navigation mode, dynamic feasible path planning and path-by-path slope restriction constraint analysis are performed based on the dynamic environment map to generate a dynamically adjusted navigation path; Step S5: predicting the moving trajectory of the dynamic object in the dynamic adjustment navigation path based on the dynamic environment map, and performing adaptive collision path avoidance to construct a collision avoidance navigation path; Step S6: When the user's current navigation mode is the joystick control mode, the dynamic wheelchair control parameters of the joystick control mode are adjusted, and intelligent collaborative control optimization is performed based on the collision avoidance navigation path to build an intelligent navigation control model.

[0008] The present invention obtains environmental video in real time through a high-definition camera to provide comprehensive monitoring of the surrounding environment. Video jitter distortion compensation and dynamic object frame-by-frame optical flow tracking can effectively improve video quality and the accuracy of dynamic object tracking. This is crucial for real-time analysis and prediction of the movement trajectory of obstacles, and can provide accurate data support for subsequent path planning. By accurately identifying static obstacles, the electric wheelchair can better perceive fixed objects in the environment, such as walls, furniture, etc. Modeling is performed in combination with the movement trajectory of dynamic objects to generate a dynamic environment map, which enables the electric wheelchair to identify and adapt to dynamic changes in the environment, such as pedestrians, other vehicles, etc., significantly improving the safety and accuracy of driving. By identifying the user's real-time operation instructions, the electric wheelchair can dynamically switch the navigation mode according to the user's intention. The switching between the rocker control mode and the intelligent navigation mode enables the user to flexibly choose the operation mode according to different usage scenarios or needs. In the intelligent navigation mode, the electric wheelchair automatically plans and optimizes the path, and in the rocker control mode, the user directly controls the wheelchair to make detailed adjustments. After entering the intelligent navigation mode, dynamic path planning is performed based on the dynamic environment map, and combined with the slope restriction analysis, the feasibility and safety of the path can be ensured. Slope analysis is particularly important because it helps prevent electric wheelchairs from traveling on paths with excessive slopes or inappropriate paths, reducing the dangers caused by unreasonable paths. By predicting the movement trajectory of dynamic objects, potential collision risks are identified in advance, and adaptive collision path avoidance is used to ensure that electric wheelchairs do not collide with dynamic objects in complex environments. This strategy can greatly improve the safety of electric wheelchairs and enable them to operate stably in complex and dynamic environments. In the joystick control mode, the response speed and stability of the wheelchair are optimized according to different control requirements and environmental changes by dynamically adjusting the wheelchair control parameters. In addition, intelligent collaborative control optimization based on the collision avoidance path helps users get more assistance and safety guarantees when manually controlling. This intelligent collaborative control adjusts the path and speed according to the user's operation, allowing the electric wheelchair to cope with complex environments more flexibly and safely.

[0009] Preferably, step S1 comprises the following steps: Step S11: Acquire real-time environmental monitoring video based on the electric wheelchair high-definition camera; Step S12: performing global contrast optimization on the real-time environment monitoring video to generate a globally optimized monitoring video; Step S13: performing video jitter distortion compensation on the global optimized monitoring video to construct a jitter distortion compensated monitoring video; Step S14: Performing dynamic object visual recognition on the jitter distortion compensation monitoring video, marking each dynamic object in the environment; Step S15: Perform frame-by-frame continuous optical flow tracking on each dynamic object in the environment to generate a moving trajectory of each dynamic object.

[0010] The present invention can provide high-resolution video data through a high-definition camera, helping the electric wheelchair to obtain detailed information about the surrounding environment in real time. This provides high-quality visual data input for subsequent video processing, environmental analysis, dynamic object recognition and path planning. High-resolution video helps to accurately identify small-sized obstacles and dynamic changes, laying the foundation for environmental perception. Global contrast optimization significantly improves the details in video images, especially in dark or bright environments. By enhancing the contrast of the image, obstacles and dynamic objects in the environment are clearer and easier to see, thereby improving the accuracy of subsequent image processing steps (such as dynamic object recognition and tracking). This is particularly important for image analysis in low-light environments or complex lighting conditions, ensuring that the electric wheelchair can effectively perceive the surrounding environment under various environmental conditions. During movement, the camera of the electric wheelchair will produce image jitter or blur due to vibration, which will affect the accuracy of environmental perception. By performing jitter distortion compensation, the distortion caused by camera jitter is eliminated or reduced, ensuring the stability and clarity of the video image. In this way, tasks such as dynamic object recognition, path planning and environmental modeling can rely on more accurate and stable visual information, thereby improving the overall performance and reliability of the navigation system. Through dynamic object visual recognition, each moving object in the environment (such as pedestrians, other vehicles, etc.) is automatically detected and marked. This process is the basis for automatic navigation and collision avoidance. Marking dynamic objects helps the wheelchair understand changes in the surrounding environment and provides key data support for dynamic path planning, obstacle avoidance decisions and path optimization. In addition, accurate identification of dynamic objects can help electric wheelchairs determine which objects are potential obstacles and take corresponding countermeasures. Frame-by-frame continuous optical flow tracking can track the motion path of each dynamic object in the video and generate its movement trajectory. This technology accurately estimates the speed, direction and position of dynamic objects by analyzing the pixel changes between consecutive frames, and then predicts their future positions. For electric wheelchairs, real-time tracking of the movement trajectory of dynamic objects is crucial, which can provide accurate data support for subsequent path planning and collision avoidance. Through this prediction of the movement trajectory of dynamic objects, electric wheelchairs can react in advance to avoid collisions with fast-moving objects and ensure the safety of users.

[0011] Preferably, the specific steps of step S12 are: Extracting timestamp information of the real-time environment monitoring video; Perform video frame synchronization optimization according to the timestamp information to obtain a time-synchronized monitoring video; Perform non-local mean denoising on the time-synchronized surveillance video to obtain the noise-filtered surveillance video; Divide the noise-filtered surveillance video into multiple regions to obtain multiple video regions; Performing color feature analysis on multiple video regions one by one to generate a color feature of each region; According to the color characteristics of each area, the color deviation between areas is identified to obtain the color deviation area; Performing local color correction processing on the color deviation area to obtain a color deviation correction area; The color deviation correction region is used to perform global contrast optimization on the noise-filtered surveillance video, thereby generating a globally optimized surveillance video.

[0012] The present invention accurately marks the time position of each frame in the video by extracting the timestamp of each frame. This information helps to optimize the subsequent frame synchronization, ensures that the video frames collected by multiple cameras can be consistent in time, and avoids errors and information loss caused by camera asynchrony. It is crucial for real-time dynamic object tracking and path planning to ensure the timeliness and accuracy of data. By optimizing the video frames synchronously based on the timestamp information, the time error problem between different frames in the video is solved, and all video frames are aligned on the same time axis. In this way, the system processes the video data more accurately, avoiding environmental perception errors caused by time asynchrony, especially in multi-camera video synthesis and dynamic object tracking. This step provides a stable time basis for the precise navigation and path planning of the electric wheelchair, and ensures the efficiency of data processing. The non-local mean denoising technology can effectively remove noise and unnecessary image interference in the video. This denoising method not only retains the details of the image, but also significantly improves the clarity of the image. When the electric wheelchair perceives the environment, it faces low light or complex background interference. Through the denoising technology, the messy noise in the background is removed, which helps to improve the accuracy of dynamic object recognition. The processed image is cleaner, reducing the risk of misidentification and misjudgment. By dividing the video into multiple regions, the video frames are divided into different regions, making the analysis of each region more refined. The environmental characteristics of different regions are different, and the partition processing helps to optimize the local environment in complex environments. This is especially important for improving the accuracy of dynamic object recognition and path planning in complex scenes. Electric wheelchairs need to process static and dynamic regions separately to adapt to different operating requirements. Color feature analysis can help the system understand the visual information of each region in the video, such as background color, object color, etc. This is important for identifying different types of objects and dynamic changes. Different color features represent different objects (such as red for traffic lights or warning signs). Color analysis of each region can provide strong support for subsequent object recognition and environmental perception, helping the system to distinguish and identify different objects more accurately and enhance the intelligence of navigation. By identifying color deviations between regions, color differences caused by changes in lighting or different lens angles are detected. This is crucial to ensure video quality and color consistency, especially in the recognition of dynamic objects, where color deviations lead to misrecognition and path planning errors. Local color correction can solve color inconsistencies caused by lighting conditions or camera characteristics. By locally correcting the color deviation area and restoring the color consistency between areas, the objects and background in the image present more natural and realistic colors. By optimizing the global contrast using the corrected color area, the clarity and visibility of the video are further improved, especially in low-light or strong-light environments. The optimized global contrast improves the visibility of details in the image, which helps the electric wheelchair better identify obstacles and dynamic objects in various environments.Enhanced contrast also helps improve the safety of wheelchairs, allowing them to accurately perceive their surroundings even in complex or low-contrast environments.

[0013] Preferably, the specific steps of step S13 are: Decomposing the time-series image frames based on the global optimization surveillance video to obtain the image frame sequence; Label the environmental feature points of the global optimized surveillance video and extract multiple key environmental feature points; Calculating the relationship between feature points of multiple key environmental feature points according to the image frame sequence to generate a matching relationship between feature points of adjacent frames; The inter-frame global transformation parameters are estimated according to the feature point matching relationship between adjacent frames, thereby obtaining the image frame global transformation parameters; Based on the global transformation parameters of the image frames, the image frame sequence is subjected to frame-by-frame inverse transformation to obtain jitter distortion compensated image frames; The jitter-distortion-compensated image frames are temporally smoothed to construct jitter-distortion-compensated surveillance videos.

[0014] The present invention decomposes the optimized surveillance video into separate image frames and analyzes the detailed information in the video frame by frame. By marking the environmental feature points in the optimized video, key points with obvious features in the scene (such as corners, edges, intersections, etc.) are extracted. These feature points have strong distinguishability and can provide a reliable basis for subsequent object recognition, visual positioning and tracking. Environmental feature point extraction is a basic step for environmental modeling and object recognition in computer vision, which can ensure that the electric wheelchair accurately perceives changes in the environment, especially in complex or dynamic environments. By calculating the relationship between feature points between image frames, the temporal association between video frames is established. This process enables the system to identify changes in the same object or scene in the image and ensure the temporal consistency of the video sequence. This is crucial for dynamic object tracking and path planning, which enables the electric wheelchair to track changes in surrounding objects in real time and dynamically adjust its travel path to ensure safety and stability. By calculating the global transformation parameters between frames, the motion information between different frames is matched and converted, thereby correcting the image displacement caused by factors such as camera jitter. The estimation of global transformation parameters helps to restore the stability of the video image and reduce visual distortion caused by vibration. This step can provide more stable and accurate video information for the electric wheelchair, and improve the accuracy of subsequent image processing and dynamic object recognition, especially in dynamic environments. Through the inverse transformation of the image frame, the image offset and distortion caused by jitter are eliminated or reduced, and the correct position of the image is restored. This step effectively improves the stability of the image, so that subsequent image processing (such as dynamic object recognition and path planning) can make decisions based on more accurate visual data, thereby improving the environmental perception and navigation capabilities of the electric wheelchair. Removing jitter distortion can enable the electric wheelchair to clearly identify and avoid obstacles in an unstable environment. Temporal smoothing optimizes the smoothing algorithm of the image frame to eliminate the small fluctuations generated by the image restoration process, ensuring the continuity and smoothness of the video image. By reducing the jump and vibration of the image, a more stable video output can be obtained, which is crucial to the performance of the real-time environmental perception and navigation system of the electric wheelchair. The smoothed image will make it easier to perform tasks such as dynamic object recognition and path planning, further improving the navigation accuracy of the wheelchair and user safety.

[0015] Preferably, step S2 specifically comprises the following steps: Step S21: performing a road condition deep semantic analysis on the jitter distortion compensation monitoring video to generate a road condition semantic feature; Step S22: accurately identifying environmental static obstacles based on the semantic features of the road conditions, and marking each environmental static obstacle; Step S23: performing geometric morphology mining on each static obstacle in the environment to generate geometric morphology information of each obstacle; Step S24: performing obstacle positioning calculation on the geometric information of each obstacle to obtain the environmental positioning coordinates of each obstacle; Step S25: performing an environmental three-dimensional structure analysis on the environmental positioning coordinates of each obstacle based on the semantic features of the road condition to generate an environmental three-dimensional structure diagram; Step S26: Dynamic environment perception modeling is performed on the three-dimensional structure diagram of the environment based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map.

[0016] The present invention extracts information such as the structure, signs, slope, and road surface conditions of the road by performing deep semantic analysis of the road conditions on the compensated monitoring video. These semantic features provide a basis for subsequent path planning and obstacle identification, ensuring that the electric wheelchair can understand the road information in the environment. The system identifies uneven road surfaces, potholes, or obstacle areas, predicts the potential risks of these areas in advance, and optimizes the navigation path, thereby improving user safety. By analyzing the semantic features of the road conditions, the system can accurately identify and mark static obstacles, such as stones, railings, and roadblocks on the roadside. This step ensures that the electric wheelchair can accurately perceive fixed obstacles in the environment to avoid collisions due to misidentification. This accurate obstacle identification provides reliable data support for subsequent path planning and collision avoidance, which is particularly important in complex environments. By extracting the geometric morphological information of each static obstacle, the system can obtain the size, shape, and structure of the obstacle. This information is crucial for accurately planning obstacle avoidance paths, especially when accurate calculations of distances and spatial relationships are required. For a larger obstacle, the system calculates how to bypass it, and for a small obstacle, the system decides whether to pass directly. This geometric mining enhances the system's deep understanding of the environment and improves the navigation accuracy of the electric wheelchair. By calculating the environmental positioning coordinates of each obstacle, the system can accurately determine the position of the obstacle in three-dimensional space. This step provides accurate position information for the path planning of the electric wheelchair, allowing the navigation system to avoid collisions with obstacles during driving. Accurate obstacle positioning not only helps to avoid real-time collisions, but also optimizes the modeling of the entire environment and improves the intelligence and flexibility of the navigation system. Based on the positioning coordinates of the obstacles and the semantic features of the road conditions, a three-dimensional structural analysis is performed to construct a complete three-dimensional structural map of the environment. This provides a global and accurate environmental model for the electric wheelchair, allowing the wheelchair to perceive the three-dimensional layout of the surrounding environment in real time. In complex environments, the three-dimensional environmental map can help the wheelchair identify more complex obstacles, such as irregularly shaped objects or protruding obstacles, so as to avoid collisions and choose the optimal driving route. By tracking the movement trajectory of dynamic objects (such as pedestrians, vehicles, etc.) and combining their information with the three-dimensional structural map, the system can update the dynamic environmental map in real time. This process helps the electric wheelchair adapt to changes in the environment, predict the movement trajectory of dynamic obstacles and issue collision warnings. When the wheelchair detects a fast-approaching pedestrian in front, it adjusts the driving path in time to avoid collision and ensure the safety of the user. The dynamic environment map provides real-time understanding of the environment, allowing the electric wheelchair to navigate more intelligently in complex and dynamic environments.

[0017] Preferably, step S3 specifically comprises the following steps: Step S31: Identify the user's real-time wheelchair operation instructions; Step S32: analyzing the user's real-time wheelchair operation instructions and extracting user instruction intention features; Step S33: mining instant navigation needs according to the user's instruction intention characteristics, thereby generating the user's instant navigation needs data; Step S34: making a navigation mode switching decision based on the user's real-time navigation demand data to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode.

[0018] The present invention realizes the intelligent control of electric wheelchairs by real-time recognition of user operation instructions. By recognizing user instructions (such as joystick operation, button click, etc.), the system quickly captures the user's intention and responds immediately. Through the intention analysis of user instructions, the system not only recognizes the content of the instruction, but also deeply understands the real needs of the user. The user indicates that he wants to move forward slowly by slightly shaking the joystick, or requires a higher speed by quickly pushing and pulling the joystick. According to the characteristics of the user's instruction intention, the system generates real-time navigation demand data, and can dynamically adjust its navigation strategy. If the user shows a strong intention to move forward or turn, the system infers the user's current environmental state based on these data and generates an instant navigation plan. Based on the user's instant navigation needs, the system makes a decision to switch the navigation mode. If the user needs full autonomous control (such as precise manual control), the system will switch to the joystick control mode; if the user wants the wheelchair to guide automatically, the system will switch to the intelligent navigation mode. This mode switching enables the electric wheelchair to make adaptive adjustments according to different usage scenarios and user needs, improves the user's control experience, increases flexibility, and also ensures safety. In a busy environment, the intelligent navigation mode helps users avoid obstacles and ensure smoother navigation.

[0019] Preferably, the specific steps of step S4 are: Step S41: when the user's current navigation mode is the intelligent navigation mode, locating the user's preset destination based on the user's real-time wheelchair operation instructions; Step S42: dynamically planning a feasible path for the user's preset destination based on the dynamic environment map, thereby generating multiple feasible navigation paths; Step S43: performing path-by-path slope limit constraint analysis on the dynamic environment map to generate a slope limit constraint feature for each path; Step S44: dynamically adjusting the multiple feasible navigation paths based on the slope limit constraint characteristics of each path to generate a dynamically adjusted navigation path.

[0020] In the intelligent navigation mode of the present invention, the system can locate the preset destination of the user according to the real-time operation instructions of the user. This means that the user directly tells the system the destination through simple operations or instructions, and the system will automatically identify and locate the target. This greatly simplifies the operation process. The user does not need to set the destination cumbersomely. The system automatically handles the identification and positioning of the destination, which improves the convenience and efficiency of the user experience. The dynamic environment map is used to plan the path of the user's preset destination and generate multiple feasible navigation paths. The dynamic environment map provides real-time environmental information, including obstacles, pedestrians, road conditions, etc., so that the system can select multiple obstacle avoidance paths for the user. This multi-path planning not only improves the flexibility of path selection, but also can cope with variable factors in complex environments and avoid the infeasibility of the path due to obstacles or emergencies. The user selects the most suitable path according to actual needs. In the path planning process of the electric wheelchair, the slope is an important factor affecting the driving safety and comfort. By analyzing the slope limit of each path, the system can evaluate the feasibility of the path and ensure that a too steep route will not be selected to avoid discomfort or potential danger to the user. The system can identify and avoid slopes with excessive slopes or roads that are not suitable for wheelchairs, ensuring that the user reaches the destination smoothly and safely. Based on the slope limit constraint feature, the system dynamically adjusts multiple feasible paths. Through this process, the electric wheelchair can automatically optimize the path selection according to real-time environmental changes. When it is detected that the slope of a certain path exceeds the set safety threshold, the system automatically adjusts the path or selects a route with a smaller slope. This dynamic adjustment function not only enhances the adaptability of the system, but also provides a safer and more comfortable driving experience according to different terrain and road conditions. Whether on a flat road or a ramp, the electric wheelchair can provide the user with the most suitable path planning.

[0021] Preferably, the specific steps of step S5 are: Step S51: performing a dynamic object motion trend analysis on the dynamic environment map to generate a motion trend feature of each dynamic object; Step S52: performing dynamic movement trajectory prediction on the movement trend characteristics of each dynamic object to generate a movement prediction trajectory of each object; Step S53: performing path intersection identification on the dynamically adjusted navigation path according to the predicted movement trajectory of each object, and marking potential collision path points; Step S54: performing adaptive collision path avoidance based on potential collision path points to construct a collision avoidance navigation path.

[0022] The present invention analyzes the movement trend of objects in the dynamic environment map, and the system can identify the movement direction, speed, acceleration and other characteristics of the objects. The electric wheelchair adjusts its driving strategy according to the movement trend of these dynamic objects, thereby avoiding sudden collisions or interference. Analyzing the movement trend of the object enhances the system's ability to perceive the environment, especially in complex environments, and can adjust the strategy in real time to deal with emergencies. By predicting the future movement trajectory of each dynamic object, the electric wheelchair can know the path of the object in the future in advance. This step provides a powerful prediction capability for real-time navigation in a dynamic environment, especially in a complex or dynamic environment, when the movement trajectory of the object changes greatly, the system generates an optimal obstacle avoidance strategy based on the predicted trajectory. When the system predicts that a pedestrian or obstacle will enter the driving path of the electric wheelchair, the wheelchair adjusts the driving route in advance to avoid potential collision points. The movement trajectory prediction significantly improves the safety and accuracy of path planning. Based on the predicted trajectory of the object, the system can identify and mark potential collision points that intersect with the electric wheelchair path. By identifying potential collision points in real time, the system can provide timely feedback to the electric wheelchair, warn of upcoming obstacles or moving bodies in advance, and take corresponding obstacle avoidance measures. Path intersection recognition enables the electric wheelchair to continuously update its navigation path in a dynamically changing environment, avoiding collisions with pedestrians and other objects and ensuring the safety of the user. Adaptive obstacle avoidance not only takes into account the impact of static obstacles, but also responds to changes in moving objects in real time, significantly improving the navigation capabilities of the electric wheelchair in complex environments. This process ensures that the electric wheelchair maintains a safe distance between the contact points with obstacles, thereby optimizing path planning and enhancing driving safety.

[0023] Preferably, the specific steps of step S6 are: Step S61: when the user's current navigation mode is the joystick control mode, the joystick control mode is personalized and evolved to generate user personalized features; Step S62: adjusting the dynamic wheelchair control parameters based on the user's personalized characteristics to generate dynamic control adjustment parameters; Step S63: Perform intelligent collaborative control optimization on the collision avoidance navigation path and dynamic control adjustment parameters to build an intelligent navigation control model.

[0024] The present invention generates customized operation experience by analyzing multiple dimensions such as the user's operation mode, historical behavior data, physical condition, etc., thereby improving the user's comfort and control accuracy. This process makes the electric wheelchair a more flexible and intelligent device that can adapt between different users to meet personalized needs. By adjusting the control parameters of the wheelchair according to the user's personalized characteristics, the system achieves a more precise and comfortable control experience. The system adjusts the control parameters such as steering sensitivity, forward speed and reverse speed according to the user's joystick operation habits to ensure that the user does not feel discomfort or overreact during operation. The generation of dynamic control adjustment parameters is based on real-time analysis of the user's usage, physical condition, current environment and other factors to provide a smoother and more personalized control response. This dynamic adjustment ensures that the user can use the wheelchair in different situations. In any environment, the operation of the electric wheelchair can adapt to the needs of the user and provide a safer and more comfortable use experience. Through intelligent collaborative control optimization of the collision avoidance navigation path and dynamic control adjustment parameters, the system can automatically optimize and adjust according to the real-time dynamic environment and user needs. This means that in addition to the obstacle avoidance function of path planning, the system will also adjust the control parameters in real time to ensure that the user can maintain a comfortable operating experience during the obstacle avoidance process. When avoiding obstacles, the speed of the wheelchair needs to be adjusted, and the control sensitivity and acceleration parameters will also be adjusted accordingly to ensure the smoothness and safety of the entire process. Intelligent collaborative control optimization ensures the coordination between collision avoidance and user control, allowing electric wheelchairs to respond to changes in complex and dynamic environments with ease and minimize unnecessary collision risks.

[0025] In this specification, a multi-mode navigation control system of an electric wheelchair controller is provided, which is used to execute the multi-mode navigation control method of the electric wheelchair controller as described above, including: The video enhancement module is used to obtain real-time environmental monitoring video based on the high-definition camera of the electric wheelchair; perform video jitter distortion compensation and frame-by-frame continuous optical flow tracking of dynamic objects on the real-time environmental monitoring video to generate the movement trajectory of each dynamic object; A dynamic environment modeling module is used to accurately identify static obstacles in the environment based on the real-time environment monitoring video, and to perform dynamic environment perception modeling based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map; A navigation mode decision module is used to identify the user's real-time wheelchair operation instructions; based on the user's real-time wheelchair operation instructions, a navigation mode switching decision is made to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode; The path planning module is used to perform dynamic feasible path planning and path-by-path slope constraint analysis based on the dynamic environment map when the user's current navigation mode is the intelligent navigation mode, so as to generate a dynamically adjusted navigation path; The collision avoidance module is used to predict the moving trajectory of dynamic objects in the dynamic adjustment navigation path based on the dynamic environment map, and to perform adaptive collision path avoidance to construct a collision avoidance navigation path; The intelligent navigation control module is used to adjust the dynamic wheelchair control parameters of the joystick control mode when the user's current navigation mode is the joystick control mode, and to perform intelligent collaborative control optimization based on the collision avoidance navigation path to build an intelligent navigation control model.

[0026] The camera of the present invention captures environmental images in real time and provides the electric wheelchair with instant surrounding environment information. This environmental perception capability enables the wheelchair to sense the presence of dynamic objects and detect obstacles or moving objects in front in real time to avoid collisions or interference. Through jitter distortion compensation, the video image becomes more stable, avoiding blurred images caused by equipment movement or external disturbances, ensuring that the wheelchair can obtain clear real-time monitoring video. The dynamic object frame-by-frame optical flow tracking technology enables the system to identify and record the trajectory of dynamic objects in real time, ensuring accurate prediction of objects and providing accurate data support for subsequent path planning, accurately identifying static obstacles in the environment (such as walls, furniture, etc.) and locating these obstacles. , ensuring that the wheelchair avoids collision with these objects during navigation. According to the movement trajectory of dynamic objects, the system can update the environmental map in real time to reflect the position and movement trend of the objects, which enables the electric wheelchair to adjust the path in time and avoid potential dangers. The generated dynamic environmental map can reflect environmental changes in real time and enhance the adaptability of the wheelchair in complex and dynamic environments. By real-time identification of user operation instructions, the system can dynamically switch navigation modes according to user needs and provide personalized control experience. Users can choose manual control or intelligent navigation according to scene needs. Automatic mode switching allows users to easily switch control methods in different occasions, reduce operational complexity, and improve usage efficiency. The system is dynamic in real time. The system generates multiple feasible navigation paths in a dynamic environment. These paths take into account factors such as obstacles and dynamic objects in the current environment, thereby ensuring that the electric wheelchair can choose the safest and most appropriate path. For each path, the system will analyze the slope of the path to ensure that the wheelchair will not choose a path with a steep slope and avoid inappropriate slopes or roads to ensure the comfort and safety of the user. The system can predict the movement trajectory of dynamic objects (such as pedestrians, other vehicles, etc.) and calculate the collision points to avoid collisions by adjusting the path. Through dynamic prediction and adaptive obstacle avoidance, the electric wheelchair can actively avoid potential dangers to ensure that the user will not be disturbed by obstacles during driving, thereby improving driving safety and collision avoidance. The generation of the collision avoidance path is real-time, ensuring that the electric wheelchair can respond quickly when the environment changes and avoid any sudden danger. By dynamically adjusting the parameters of the joystick control mode (such as sensitivity, acceleration, etc.), the system can be personalized according to the user's control habits to make the operation smoother and more comfortable. In the joystick control mode, the electric wheelchair is intelligently optimized based on the collision avoidance path to ensure that when the user is manually controlling the wheelchair, the response and obstacle avoidance functions of the wheelchair can work seamlessly together to avoid accidental collisions and provide a smooth driving experience. Through the intelligent optimization control model, the electric wheelchair can dynamically adapt to complex environmental changes in the joystick control mode, while providing users with a safer and more comfortable control experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic flow chart of the steps of a multi-mode navigation control method for an electric wheelchair controller of the present invention; Figure 2 Detailed implementation flow chart of step S1; Figure 3 Detailed implementation flow chart of step S2; Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0028] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0029] The present application example provides a multi-mode navigation control method and system for an electric wheelchair controller. The execution subject of the multi-mode navigation control method and system for an electric wheelchair controller includes but is not limited to: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0030] See also Figures 1 to 4 The present invention provides a multi-mode navigation control method for an electric wheelchair controller, the multi-mode navigation control method for an electric wheelchair controller comprising the following steps: Step S1: acquiring a real-time environment monitoring video based on a high-definition camera of the electric wheelchair; performing video jitter distortion compensation and frame-by-frame continuous optical flow tracking of dynamic objects on the real-time environment monitoring video to generate a moving trajectory of each dynamic object; Step S2: accurately identifying static obstacles in the environment on the real-time environment monitoring video, and performing dynamic environment perception modeling based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map; Step S3: identifying the user's real-time wheelchair operation instructions; making navigation mode switching decisions based on the user's real-time wheelchair operation instructions to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode; Step S4: When the user's current navigation mode is the intelligent navigation mode, dynamic feasible path planning and path-by-path slope restriction constraint analysis are performed based on the dynamic environment map to generate a dynamically adjusted navigation path; Step S5: predicting the moving trajectory of the dynamic object in the dynamic adjustment navigation path based on the dynamic environment map, and performing adaptive collision path avoidance to construct a collision avoidance navigation path; Step S6: When the user's current navigation mode is the joystick control mode, the dynamic wheelchair control parameters of the joystick control mode are adjusted, and intelligent collaborative control optimization is performed based on the collision avoidance navigation path to build an intelligent navigation control model.

[0031] In the embodiment of the present invention, refer to Figure 1 , is a flowchart of a multi-mode navigation control method for an electric wheelchair controller of the present invention. In this example, the steps of the multi-mode navigation control method for an electric wheelchair controller include: Step S1: acquiring a real-time environment monitoring video based on a high-definition camera of the electric wheelchair; performing video jitter distortion compensation and frame-by-frame continuous optical flow tracking of dynamic objects on the real-time environment monitoring video to generate a moving trajectory of each dynamic object; In this embodiment, a high-definition camera is installed on the electric wheelchair, and a resolution of 1920x1080 (1080p) is selected to ensure video quality. The camera should have good low-light performance to adapt to different environmental conditions. Set the viewing angle of the camera, usually a wide angle (such as 120 degrees) is selected to cover a larger field of view to ensure that the surrounding dynamic objects can be captured. Use video acquisition software (such as OpenCV) to capture the video stream in real time, and set the acquisition frame rate to 30fps to ensure the smoothness and continuity of the video. Record the metadata of the video stream, including timestamp, frame number, and ambient lighting conditions, for subsequent analysis. Use a jitter compensation technique based on the optical flow method, such as the Lucas-Kanade optical flow algorithm, to detect jitter in the video caused by device movement. The algorithm has high accuracy when processing small-scale displacements. Set the window size for optical flow calculation, usually a local window of 5x5 or 7x7 to improve the compensation effect. For each frame of video, calculate the optical flow vector of the object between consecutive frames to identify the pixel displacement caused by camera shake. According to the calculated optical flow vector, adjust the position of the current frame to eliminate the jitter effect. In the specific implementation, use OpenCV's cv2.calcOpticalFlowFarneback() function to calculate the optical flow and apply the inverse transformation to compensate for the jitter. By comparing the frames before and after compensation, observe the changes in image stability and clarity to ensure that the jitter compensation effect is significant. Use indicators such as SSIM (Structural Similarity Index) and PSNR (Peak Signal-to-Noise Ratio) for quantitative evaluation. Select an optical flow algorithm suitable for dynamic object tracking, such as DenseOpticalFlow or Kanade-Lucas-Tomasi (KLT) algorithm. The KLT algorithm performs better when processing specific feature points and is suitable for tracking dynamic objects. Set the feature point extraction method and use the Shi-Tomasi corner detection algorithm to extract key feature points. For each frame of the image, use the KLT algorithm to detect and track the feature points. Update the position of the feature points in each frame and record their coordinate changes. Use the cv2.calcOpticalFlowPyrLK() function in OpenCV to implement optical flow tracking, and the movement trajectory of each feature point is updated in real time. Generate the object's movement trajectory data based on the feature point trajectory of each dynamic object. Store the trajectory data as a two-dimensional coordinate sequence for subsequent analysis and visualization. For each dynamic object, calculate its speed and acceleration, and store this information together with the trajectory data for more in-depth dynamic analysis. Store the generated dynamic object trajectory data in a database, including each object's ID, trajectory coordinates, speed, acceleration and other information for subsequent use. Mark the trajectory data with a timestamp to ensure that the dynamic changes of each object can be tracked. Use visualization tools such as Matplotlib to draw the trajectory of dynamic objects on real-time monitoring videos to facilitate observation of object movement and trajectory accuracy.Evaluate the accuracy of dynamic object tracking, using error calculations such as root mean square error to quantify the accuracy of trajectory predictions.

[0032] Step S2: accurately identifying static obstacles in the environment on the real-time environment monitoring video, and performing dynamic environment perception modeling based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map; In this embodiment, the stable video frames obtained in the previous step are used to ensure good video quality and avoid the recognition effect affected by jitter and blur. The video frames should have a high resolution (such as 1920x1080), and 30 frames are obtained per second. The video frames are preprocessed, including grayscale, denoising and contrast enhancement to improve the accuracy of subsequent processing. The cv2.GaussianBlur() in OpenCV is used for denoising, and the cv2.cvtColor() is used for grayscale conversion. The deep learning method is used for static obstacle detection. Target detection models such as YOLOv5 and MaskR-CNN are selected. These models are good at processing objects in complex backgrounds. It performs well in body detection. The training parameters of the model are set, such as selecting the IoU threshold as 0.5 and the confidence threshold as 0.6 to ensure that the obstacles identified have high accuracy. Each processed frame is input into the trained target detection model to identify static obstacles (such as walls, furniture, etc.). The model will return the bounding box coordinates and category labels of each detected object. The detection results are post-processed, and the spatial position of each static obstacle is calculated based on the bounding box coordinates. The detection results with low confidence are filtered out to ensure the accuracy of recognition. The position information (bounding box coordinates, category labels) of the identified static obstacles is stored in the database to form static environment feature data. For subsequent use, the dynamic object trajectory data generated in step S1 is used to record the motion trajectory of each dynamic object and its speed, acceleration and other information, and the trajectory of the dynamic object is combined with the static obstacle information to create an environment model containing dynamic and static features. A modeling algorithm suitable for dynamic environment perception is selected, such as a grid map-based modeling method or a graph-based modeling method. The grid map method is suitable for real-time updating and dynamic feature display. The grid size is set, for example, a 0.5-meter grid is selected to accurately reflect the distribution and movement of objects in a dynamic environment. For each dynamic object, its position at each time step is calculated according to its trajectory data, and the corresponding value is updated in the grid map. Grid state. If a dynamic object overlaps with a static obstacle at a certain moment, record the interaction between the moving trajectory and the obstacle to capture the changes in the dynamic environment. Generate a dynamic environment map, including the location of static obstacles and the moving trajectory of dynamic objects. Visualize the map to show the distribution of static obstacles and the moving path of dynamic objects. Mark the category and location of obstacles on the map to ensure the intuitiveness and accuracy of the dynamic environment model. Store the generated dynamic environment map information in the database, including relevant data of static obstacles and dynamic objects, for subsequent navigation and decision-making. Use JSON or XML format to store map data in a structured manner for subsequent parsing and use.

[0033] Step S3: identifying the user's real-time wheelchair operation instructions; making navigation mode switching decisions based on the user's real-time wheelchair operation instructions to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode; In this embodiment, the input device of the wheelchair is configured, including a joystick, a button, and a voice recognition module. The joystick should have high sensitivity and low latency to ensure real-time response to the user's operation. Set the input range of the joystick, for example, set the displacement range of the joystick to ±100% (i.e., full-range displacement), and define the operation threshold for each direction (10% is the minimum valid operation). Use a microcontroller (such as Arduino or RaspberryPi) to collect the displacement signal of the joystick in real time and record the status of the button (such as start, stop, switch mode, etc.). De-noise the signal and use a low-pass filter to smooth the input signal to reduce misoperation caused by jitter. Define the category of operation instructions, such as "forward", "backward", "turn left", "turn right", "switch mode", etc. According to the input value of the joystick and the status of the button, the user's operation intention is analyzed in real time. Set the operation priority. If the user presses the mode switch button and pushes the joystick forward at the same time, the mode switch is executed first. According to the user's operation instructions, create a state machine model and define different operation states: STATE_IDLE: No operation status.

[0034] STATE_JOYSTICK_CONTROL: In joystick control mode.

[0035] STATE_INTELLIGENT_NAVIGATION: In intelligent navigation mode.

[0036] When the "switch mode" command is detected, transfer from STATE_JOYSTICK_CONTROL to STATE_INTELLIGENT_NAVIGATION. Use a simple conditional judgment structure to implement the mode switching logic. ifbutton_switch_mode_pressed:current_mode="intelligent navigation mode"ifcurrent_mode=="joystick control mode"else"joystick control mode", set a timer to avoid repeated triggering of mode switching. Set a minimum interval time (such as 1 second) to avoid frequent switching caused by user misoperation. Display the current navigation mode on the wheelchair user interface, and use LED lights or display screens to feedback the user's operation status in real time to ensure that the user can clearly understand the current control mode. Record the time and status of each mode switch for subsequent data analysis and user experience optimization. Store the user's operation instructions, current navigation mode and its switching time in the log file to facilitate subsequent analysis of the user's operation habits and mode switching frequency. Set the storage format, including timestamp, operation instructions, current mode and other information, to facilitate subsequent data mining and analysis. Regularly analyze the user's operation data and identify the user's operation habits in different environments, such as the preference for using intelligent navigation mode in complex environments. Optimize the mode switching algorithm based on the analysis results to improve the user experience. If it is identified that the user frequently switches modes in certain situations, consider automatic switching or provide intelligent suggestions.

[0037] Step S4: When the user's current navigation mode is the intelligent navigation mode, dynamic feasible path planning and path-by-path slope restriction constraint analysis are performed based on the dynamic environment map to generate a dynamically adjusted navigation path; In this embodiment, the dynamic environment map generated in the previous step is used, and the map contains the location of static obstacles and the moving track of dynamic objects. Ensure the real-time nature of the map information and update the location of obstacles and dynamic objects in a timely manner. Format the dynamic environment map into a data structure that can be used for path planning, such as using a grid or graph structure, and set the accessibility and obstacle information of each node. Set the resolution of the map, such as the size of each grid is 0.5 meters, so that the details of the environment can be fully reflected. Record the slope information in the environment, especially in special terrains such as slopes or steps, to ensure that the feasibility can be accurately judged in subsequent path planning. Select an algorithm suitable for dynamic environment path planning, such as the A* algorithm or the Dijkstra algorithm. These algorithms are efficient and accurate in dealing with obstacles and dynamic changes. Set a heuristic function, such as using Manhattan distance or Euclidean distance, to optimize the efficiency of path search. Input the user's current location and preset destination into the selected algorithm to generate a preliminary feasible path. Considering the movement of dynamic objects, set the influence range of dynamic obstacles. During the path generation process, check the location of dynamic objects in real time to ensure that the generated path can effectively avoid dynamic objects. Evaluate the generated path, including indicators such as path length, estimated time, and safety. Set thresholds, such as path length must not exceed 100 meters and estimated time must not exceed 5 minutes. If the path does not meet the set threshold conditions, repeat the path planning process until a path that meets the conditions is generated. Select a geometric analysis method to calculate the slope of the path. According to the node coordinates on the path, slope = height change / horizontal distance × 100%, set a safety threshold for the slope, such as a slope of no more than 8% is suitable for electric wheelchairs to ensure the safety and comfort of the user. For each generated path, calculate its slope change node by node, and identify nodes with slopes exceeding the safety threshold. If there is a slope exceeding the threshold in the path, adjust the path and seek alternative paths by recalculating the coordinates of adjacent nodes. Combined with the dynamic programming algorithm, adjust the path in real time for the path nodes that exceed the slope limit to generate a new feasible path. Record the information of each adjustment, including the original path, the adjusted path and its slope data, for subsequent analysis and optimization. Output the adjusted dynamic navigation path to the user interface to display the visualization effect of the path to ensure that the user can clearly understand the current navigation route. Provides detailed information about the route, including route length, estimated time, slope information, etc., to help users make informed decisions.

[0038] Step S5: predicting the moving trajectory of the dynamic object in the dynamic adjustment navigation path based on the dynamic environment map, and performing adaptive collision path avoidance to construct a collision avoidance navigation path; In this embodiment, the dynamic environment map and the moving trajectory data of the dynamic object generated in the previous step are used to ensure that the latest position information of the dynamic object has been stored and updated, including speed, acceleration and direction of movement, etc. The Kalman filter algorithm is applied to smooth the trajectory data of the dynamic object to reduce the influence of noise and improve the accuracy of prediction. A suitable moving trajectory prediction model is selected, such as a linear regression model or an LSTM (long short-term memory network). The LSTM model is suitable for processing time series data and can capture the movement pattern of dynamic objects. The input features of the model are set, including the speed, acceleration, direction and position information of the past few frames. The collected dynamic object features are input into the selected prediction model to generate the moving trajectory within the next few seconds. The prediction time window is set, for example, the trajectory of the next 3 seconds is predicted, and the sampling frequency is 30Hz. The predicted trajectory coordinates of each dynamic object are recorded to form time series data for subsequent collision path avoidance. A suitable collision path avoidance algorithm is selected, such as a dynamic window method (DWA) or a sampling-based RRT* (rapid random tree) algorithm. These algorithms can effectively deal with obstacles in a dynamic environment. A collision detection threshold is set. For example, the safety distance is set to 0.5 meters to ensure that the wheelchair maintains a sufficient safety distance from the dynamic object. A collision detection is performed between the predicted trajectory of the dynamic object and the current navigation path, and the distance between the two is calculated. If the distance between the predicted trajectory and the navigation path is less than the set safety threshold, a collision occurs, and the time point and position of the potential collision are recorded for subsequent avoidance processing. When a potential collision is detected, the navigation path is dynamically adjusted to avoid the dynamic object within the predicted trajectory range by replanning the path. An adjustment strategy is set, such as giving priority to deceleration, changing direction or replanning the path. A new path is dynamically generated using the RRT algorithm or the A algorithm to ensure that the new path avoids all potential collision points while maintaining the smoothness and efficiency of the path. During the path planning process, the predicted trajectory and position of the dynamic object are updated in real time to avoid information lag in the path planning process. The adjusted collision avoidance navigation path is output to the user interface to ensure that the user can clearly understand the new navigation route, provide a visualization effect of the path, display dynamic objects and avoidance paths, and provide detailed information on the path, including path length, estimated time and real-time dynamic object information, to help users understand the reasons for path selection.

[0039] Step S6: When the user's current navigation mode is the joystick control mode, the dynamic wheelchair control parameters of the joystick control mode are adjusted, and intelligent collaborative control optimization is performed based on the collision avoidance navigation path to build an intelligent navigation control model.

[0040] In this embodiment, key parameters of wheelchair control, including speed, acceleration, steering sensitivity, and brake response time, are determined, the user's joystick operation is monitored in real time, the user's operation mode and habits are analyzed, and the strength, direction, and duration of each user operation are obtained through sensors. A feedback mechanism is set. When the user's operation frequency exceeds a certain threshold (such as more than 2 times per second), the steering sensitivity is automatically increased so that the user can more easily control it in a complex environment. Conversely, when the user operates less, the sensitivity is reduced to ensure that the user can stably control the wheelchair. The control parameters of the wheelchair are adjusted in real time through a microcontroller. A PID control algorithm is used to dynamically adjust the control parameters according to the real-time feedback input by the user. An intelligent collaborative control model suitable for a dynamic environment is selected, such as a fuzzy controller or a reinforcement learning model. The fuzzy controller adjusts the motion state of the wheelchair according to the real-time input, and the reinforcement learning model can optimize the decision through historical operations. The input variables of the fuzzy control are set, such as the user's operation intention (forward, backward, steering) and the current environmental state (obstacle distance, speed, etc.). The dynamic environmental map and the collision avoidance path are combined to calculate the optimal control strategy in real time. When it is detected that there is an obstacle ahead, the optimal control strategy is calculated. When the wheelchair encounters an obstacle, the system automatically determines whether to slow down or turn to ensure the safety of the user. Decision rules are set, such as when the distance to the obstacle is less than 1 meter, it will immediately slow down and turn to avoid collision. Through the feedback mechanism, feedback information from users during use is collected to evaluate the effect of intelligent collaborative control optimization. Feedback indicators are set, such as user satisfaction and the number of collision events. The control model is continuously optimized based on the feedback information, and the parameters are adjusted to make it more in line with the user's operating habits and environmental changes. During use, if the user chooses to slow down multiple times, the system should automatically reduce the speed in similar environments. The dynamic wheelchair control parameter adjustment is combined with the intelligent collaborative control optimization to form a complete intelligent navigation control model, which can respond to user operations in real time and dynamically adjust the control strategy according to environmental changes. In the model, state transfer rules are set to ensure that when the user switches modes, the system can quickly adapt to the new control strategy. The model is verified in a simulation environment to test the control effect under different environmental conditions. In an environment with dense obstacles, the reaction speed and operating stability of the system are observed. Test data is collected, the performance of the model is evaluated, and adjustments and optimizations are made based on the test results.

[0041] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Step S11: Acquire real-time environmental monitoring video based on the electric wheelchair high-definition camera; Step S12: performing global contrast optimization on the real-time environment monitoring video to generate a globally optimized monitoring video; Step S13: performing video jitter distortion compensation on the global optimized monitoring video to construct a jitter distortion compensated monitoring video; Step S14: Performing dynamic object visual recognition on the jitter distortion compensation monitoring video, marking each dynamic object in the environment; Step S15: Perform frame-by-frame continuous optical flow tracking on each dynamic object in the environment to generate a moving trajectory of each dynamic object.

[0042] In this embodiment, a high-resolution camera (such as 1080p or 4K) is selected to ensure that it can capture clear video under different lighting conditions (such as day and night). It is recommended to use a camera with autofocus and wide dynamic range (WDR) functions to improve performance in complex lighting environments. Ensure that the camera has a good viewing angle (such as 120 degrees or wider) and anti-shake function to capture more comprehensive environmental information. The camera is installed in front of or above the electric wheelchair to ensure that its field of view is not blocked. The height and angle of the camera should be considered during installation to achieve the best monitoring effect. Configure the camera parameters, such as frame rate (set to 30fps to ensure smooth video), resolution, and encoding format (such as H.264 or H.265) for real-time video transmission, use appropriate software (such as OpenCV or FFmpeg) to obtain real-time video stream from the camera, ensure the stability and continuity of the video stream, set the save path and format of the video stream, ensure that the video data can be stored in time for subsequent processing, transmit the obtained video stream to local storage or cloud server in real time, use RTSP (Real Time Streaming Protocol) or RTMP (Real Time Messaging Protocol) for data transmission to ensure low latency, regularly check the quality of the video stream to ensure that there are no problems such as frame loss or delay, and choose an appropriate contrast enhancement algorithm, such as histogram equalization. ation) or adaptive histogram equalization (CLAHE) to improve the contrast and details of the video. The CLAHE algorithm is particularly suitable for processing scenes with uneven lighting, effectively enhancing local contrast without generating obvious noise. Use OpenCV and other computer vision libraries to extract frame data of real-time surveillance video frame by frame so that each frame can be processed. Apply the selected contrast optimization algorithm to each frame of the video to improve the image quality by adjusting the contrast coefficient and limiting noise. Resynthesize each optimized frame to generate a globally optimized surveillance video. Ensure that the frame rate and time of the video are synchronized during synthesis. Save the optimized video file and set a reasonable Resolution and encoding format, ensure the compatibility of the video in subsequent processing, select a suitable jitter compensation algorithm, such as compensation technology based on optical flow method or method based on feature point tracking, these methods effectively identify and correct jitter in the video, the optical flow method can more accurately compensate for the distortion caused by camera movement by calculating the motion vector between consecutive frames, use the optical flow calculation function in OpenCV (such as Lucas-Kanade method) to analyze the global optimization surveillance video frame by frame, calculate the motion vector of each frame, identify the jitter area in the video by estimating the motion vector, and perform corresponding compensation, according to the calculated motion information, for each Correct each frame to generate a surveillance video after jitter distortion compensation. Save the compensated video file to ensure that the video quality is improved and there is no obvious distortion. Select a suitable object recognition algorithm, such as YOLO (You Only Look Once) or SSD (Single Shot Multibox Detector). These algorithms can recognize dynamic objects in the video in real time. Use pre-trained deep learning models to speed up recognition and improve accuracy. Process the jitter compensated video frame by frame. Use the selected object recognition algorithm to detect dynamic objects in each frame. Configure the algorithm's parameters, such as the confidence threshold (for example, 0.5) to ensure that only objects with high confidence are marked, draw borders around the identified dynamic objects and mark the categories, use different colors to identify different types of objects (such as people, vehicles, etc.), generate marked dynamic object monitoring videos, save the processed results for subsequent analysis, select appropriate optical flow tracking algorithms, such as sparse optical flow tracking (Lucas-Kanade) or dense optical flow (Farneback), to meet different tracking needs. Sparse optical flow is suitable for tracking specific key points, while dense optical flow can provide motion information for each pixel. You can select the appropriate algorithm according to your needs, extract feature points (such as corner points) for each dynamic object, and use the selected optical flow algorithm to track the feature points frame by frame, record the position of each object in different frames, generate the object's movement trajectory data, visualize the movement trajectory of each object in the video, draw the motion path for easy analysis, and save the tracking results, including the movement trajectory data and the video file with trajectory markings, for further research and analysis. .

[0043] In this embodiment, the specific steps of step S12 are: Extracting timestamp information of the real-time environment monitoring video; Perform video frame synchronization optimization according to the timestamp information to obtain a time-synchronized monitoring video; Perform non-local mean denoising on the time-synchronized surveillance video to obtain the noise-filtered surveillance video; Divide the noise-filtered surveillance video into multiple regions to obtain multiple video regions; Performing color feature analysis on multiple video regions one by one to generate a color feature of each region; According to the color characteristics of each area, the color deviation between areas is identified to obtain the color deviation area; Performing local color correction processing on the color deviation area to obtain a color deviation correction area; The color deviation correction region is used to perform global contrast optimization on the noise-filtered surveillance video, thereby generating a globally optimized surveillance video.

[0044] In this embodiment, a computer vision library (such as OpenCV) is used to read the surveillance video file. Ensure that the timestamp information of each frame can be extracted during the reading process. When reading the video, the cv2.VideoCapture function in OpenCV is used to obtain the frame rate (FPS) of the video and the timestamp of each frame. The timestamp of each frame is calculated using the formula: timestamp=frame_index / fps, where frame_index is the index of the current frame and fps is the frame rate of the video. The extracted timestamp information is stored in a list or array, and the data structure should include the frame index, timestamp, and corresponding image frame. Select a suitable frame synchronization algorithm, commonly used linear interpolation or spline interpolation, which can smoothly fill in the missing frames to maintain the continuity of time. According to the extracted timestamp, the video frame is processed by the interpolation algorithm. Traverse the timestamp array and identify the missing time points. Use the interpolation algorithm to generate frame data for the missing time points to maintain the continuity of the video. Resynthesize the synchronized frame data into a new video file to ensure that the timestamp of each frame can correspond correctly during video playback. Select the Non-Local Means (NLM) denoising algorithm, which can effectively remove image noise while preserving details and edges in the image. Use the fast Nl Means Denoising Colored function in OpenCV to denoise each frame of the video. Set denoising parameters, such as the h value (usually set between 10-30) to control the intensity of denoising. Resynthesize each processed frame to generate a denoised surveillance video to ensure that the noise in the video is effectively filtered. Select an appropriate region partitioning algorithm, such as K-means clustering or watershed algorithm. These algorithms divide the image into different regions based on color or brightness information. Perform region partitioning on each denoised frame. Use OpenCV's cv2.kmeans function or watershed algorithm to divide each frame into several regions. Mark the divided regions for subsequent feature analysis. Generate a new region labeling image and save the processing results. Select an appropriate color feature extraction method, such as color histogram, average color value, or color space conversion (such as HSV or Lab color space). Calculate the color features for each divided region. Use OpenCV's cv2.calcHist function to calculate the color histogram of the region, or calculate the average color value of the region. Store the color features of each region in a data structure, including information such as region ID, histogram data, and average color value. Select a suitable color deviation identification method, such as Euclidean distance or cosine similarity, to compare the color features between different regions. Traverse all regions and calculate the difference in color features between each pair of regions. Compare the difference value with a preset threshold (for example, set to 0.1 or 0.2) to identify the color deviation area.Mark the identified color deviation areas for subsequent processing. Generate a list of deviation areas, recording the degree and location of the deviation. Select an appropriate local color correction algorithm, such as color mapping or histogram matching. These methods effectively adjust the color characteristics of a specific area to make it consistent with the surrounding area. Apply the selected correction algorithm to each color deviation area. Use OpenCV's cv2.matchHist function to perform histogram matching to adjust the color distribution of the deviation area. Merge the corrected area back to the original video frame to ensure the color consistency of the overall video. Save the processed results and generate a video of the color deviation correction area. Select a global contrast optimization algorithm, such as histogram equalization (HistogramEqualization) or adaptive histogram equalization (CLAHE) to enhance the visual effect of the video. Apply the selected contrast optimization algorithm to the denoised surveillance video. Use OpenCV's cv2.equalizeHist or cv2.createCLAHE function to process each frame. Resynthesize each processed frame to generate a globally optimized surveillance video to ensure the improvement of the video in visual effect.

[0045] In this embodiment, the specific steps of step S13 are: Decomposing the time-series image frames based on the global optimization surveillance video to obtain the image frame sequence; Label the environmental feature points of the global optimized surveillance video and extract multiple key environmental feature points; Calculating the relationship between feature points of multiple key environmental feature points according to the image frame sequence to generate a matching relationship between feature points of adjacent frames; The inter-frame global transformation parameters are estimated according to the feature point matching relationship between adjacent frames, thereby obtaining the image frame global transformation parameters; Based on the global transformation parameters of the image frames, the image frame sequence is subjected to frame-by-frame inverse transformation to obtain jitter distortion compensated image frames; The jitter-distortion-compensated image frames are temporally smoothed to construct jitter-distortion-compensated surveillance videos.

[0046] In this embodiment, the OpenCV library is used for video processing, which provides powerful functions for reading and processing video streams. Set video reading parameters, such as frame rate (set to 30fps), to ensure that the extracted frames have high temporal resolution. Open the global optimized monitoring video file through the cv2.VideoCapture() function of OpenCV, and use cv2.read() to read the video content frame by frame. Set a loop until the end of the video, extract and save it as an image file (such as JPEG or PNG format) frame by frame, and save it in a specified folder for subsequent access. Store the extracted image frames in a list or array, and name each frame with its serial number (such as frame_001.jpg, frame_002.jpg, etc.) for subsequent processing. Select a suitable feature point detection algorithm, such as Harris corner detection, SIFT (scale-invariant feature transform), or ORB (Oriented FAST and Rotated BRIEF). The ORB algorithm strikes a good balance between speed and accuracy and is suitable for real-time processing. Use the cv2.ORB_create() function in the OpenCV library to create an ORB feature point detector. Apply a feature point detection algorithm to each image frame and extract feature points and descriptors using the detectAndCompute() function. Mark the extracted feature points, draw circles on the image or use other visualization methods to ensure that these key feature points are highlighted. Store the feature points and their descriptors for each frame in a dictionary or data structure for subsequent processing. Choose a suitable feature point matching algorithm, such as BFMatcher (brute force matcher) or FLANN (fast approximate nearest neighbor), which is suitable for processing a large number of feature points. Use the cv2.BFMatcher() function or cv2.FlannBasedMatcher() in OpenCV. Match the feature points of adjacent frames and get the matching relationship between the feature points by calling the match() method of the selected matcher. Set matching parameters, such as the distance threshold (for example, 2.5), to filter out bad matches. Store the matching results of each pair of adjacent frames in a list or dictionary, record the matched feature point index and its corresponding frame information for subsequent use. Choose a suitable transformation model, such as rigid transformation (translation and rotation) or affine transformation, depending on the changes in the environment. Use the cv2.estimateAffinePartial2D() function in OpenCV to calculate the transformation matrix. Using the matched feature point pairs, call estimateAffinePartial2D() to calculate the transformation matrix between adjacent frames. Set the input parameters, such as the coordinates of the feature points and the matching information, to ensure the accuracy of the calculation. Store the estimated transformation parameters (such as translation and rotation information) in a data structure to facilitate subsequent inverse transformation and compensation processing.Apply the estimated global transformation parameters to each frame and perform the inverse transformation using OpenCV's cv2.warpAffine() function. Set the inverse transformation parameters, such as the target image size and interpolation method (e.g., linear interpolation), to ensure image quality. Save the inverse transformed image frames as a new file (e.g., image_stabilized_001.jpg) for subsequent processing and analysis. Select an appropriate temporal smoothing algorithm, such as MovingAverage or weighted average, to reduce flicker and jitter between image frames. Set the window size (e.g., 5 or 7 frames) to determine the intensity of the smoothing effect. Temporally smooth each frame and calculate the average of the current frame and the previous and next frames to generate a smoothed image frame. Set smoothing parameters, such as weight coefficients, to control the degree of smoothing. Synthesize the smoothed image frames into a video, set the video parameters (e.g., resolution, frame rate) using OpenCV's cv2.VideoWriter() function, and save it as a new video file (e.g., stabilized_video.mp4).

[0047] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Step S21: performing a road condition deep semantic analysis on the jitter distortion compensation monitoring video to generate a road condition semantic feature; Step S22: accurately identifying environmental static obstacles based on the semantic features of the road conditions, and marking each environmental static obstacle; Step S23: performing geometric morphology mining on each static obstacle in the environment to generate geometric morphology information of each obstacle; Step S24: performing obstacle positioning calculation on the geometric information of each obstacle to obtain the environmental positioning coordinates of each obstacle; Step S25: performing an environmental three-dimensional structure analysis on the environmental positioning coordinates of each obstacle based on the semantic features of the road condition to generate an environmental three-dimensional structure diagram; Step S26: Dynamic environment perception modeling is performed on the three-dimensional structure diagram of the environment based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map.

[0048] In this embodiment, a deep learning model is selected for semantic parsing, such as FCN (fully convolutional network), DeepLab or U-Net. These models perform well in image segmentation tasks. A pre-trained model (such as DeepLab trained on the Cityscapes dataset) is used to speed up training and improve accuracy. Each frame of the surveillance video after jitter distortion compensation is extracted frame by frame using OpenCV. The selected deep learning model is applied to each frame for semantic segmentation to identify key objects such as roads, pedestrians, and vehicles in the image. The segmentation result of each frame is converted into a semantic feature map, and the category label and confidence of each pixel are recorded. The generated semantic feature map will be used for subsequent obstacle recognition and environmental analysis. A suitable obstacle recognition algorithm is selected, such as YOLO or SSD. These algorithms can detect static and dynamic objects in real time. OpenCV is combined with a deep learning framework (such as TensorFlow or PyTorch) for obstacle detection. Based on the previous semantic feature map, set The threshold of obstacle detection ensures that only those static obstacles with high confidence (such as traffic signs, roadblocks, etc.) are marked. Draw a border around the detected obstacles and mark the category. Generate a marked surveillance video. Select appropriate geometric analysis methods, such as contour detection and shape descriptors (such as Hu moments), to extract the geometric features of obstacles. Use functions such as cv2.findContours() and cv2.contourArea() in OpenCV for shape analysis. Perform contour extraction on the detected obstacles. Calculate the area, perimeter and shape features of each obstacle. Generate geometric information of each obstacle, including shape type, size and position. Store the extracted geometric information in a database. Record the relevant features of each obstacle for subsequent use. Select an appropriate positioning algorithm, such as the positioning method based on the geometric center or the minimum enclosing rectangle algorithm, to determine the center coordinates of the obstacle. Use cv2.minAreaRect() or cv2.contourArea() performs positioning calculations, calculates the coordinates of the geometric center or the minimum circumscribed rectangle of each obstacle's geometric morphological information, and uses them as the positioning coordinates of the obstacle in the environment. The calculated coordinates are converted to the actual environment coordinate system (such as using calibration parameters for conversion), and the environment positioning coordinates of each obstacle are stored in the database. The corresponding obstacle information is recorded for subsequent environmental analysis, and an appropriate 3D reconstruction method is selected, such as stereo vision, structure from motion (SfM), or multi-view stereo reconstruction, to generate the 3D structure of the environment. Libraries such as OpenCV or PCL (PointCloudLibrary) are used for 3D reconstruction, and the known obstacle positioning coordinates and semantic features are combined with camera parameters to generate 3D point clouds. The 3D structure of the environment is generated through the 3D reconstruction algorithm. Compose a map, record the position of obstacles in three-dimensional space, use visualization tools (such as Matplotlib or PCLViewer) to display the generated three-dimensional structure diagram, ensure that obstacles and environmental structures can be observed intuitively, select appropriate dynamic modeling methods, such as dynamic models based on state estimation (such as Kalman filtering) or particle filtering, model the trajectory of dynamic objects, use appropriate algorithms to combine three-dimensional structure information to achieve dynamic environment perception, fuse the identified dynamic object movement trajectory with the three-dimensional structure diagram, record the motion state of dynamic objects in the environment, predict the future position of dynamic objects through state estimation algorithms, update the environmental map, combine the position and speed information of static obstacles and dynamic objects, generate a complete dynamic environmental map, and save the data structure of the dynamic environmental map for subsequent navigation and decision support. .

[0049] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include: Step S31: Identify the user's real-time wheelchair operation instructions; Step S32: analyzing the user's real-time wheelchair operation instructions and extracting user instruction intention features; Step S33: mining instant navigation needs according to the user's instruction intention characteristics, thereby generating the user's instant navigation needs data; Step S34: making a navigation mode switching decision based on the user's real-time navigation demand data to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode.

[0050] In this embodiment, a suitable input device is selected, such as a joystick, a touch screen or a voice recognition system, to ensure that the user's operation instructions can be obtained efficiently. The input device is connected to the wheelchair control system via a Bluetooth or USB interface to achieve real-time data transmission. The user's operation signal, such as the displacement of the joystick, the state of the button or the voice command, is collected through the input device. For the joystick, a sensor (such as a potentiometer) is used to obtain its displacement and angle information in real time; for voice input, a microphone is used in combination with a voice recognition API (such as Google Speech API) for processing. A processing algorithm is designed to analyze the acquired signal data in real time. For the joystick operation, a threshold is set, and when the displacement exceeds a specific value, it is recognized as a "forward", "backward", "left turn" or "right turn" instruction. The recognized instructions are stored in a buffer for use in subsequent steps. A suitable intent analysis model is selected, and a simple rule-based model or a deep learning model (such as LSTM or BERT) is used for semantic analysis. If a deep learning model is used, the model needs to be pre-trained to recognize different user instruction intentions. The user's real-time operation instructions are pre-processed, including denoising, normalization, etc., to ensure data quality. Convert the user's command data into a format accepted by the model, such as converting the joystick displacement data into a vector form. Input the processed data into the selected intent analysis model for real-time analysis. Based on the output of the model, extract the user's command intention features, such as "navigate to a specified location", "avoid obstacles", "stop", etc. Select a suitable demand mining model and use decision trees, random forests or deep learning models (such as CNN) for mining. If a deep learning model is used, a training data set containing examples of different user needs must be prepared in advance. Use the extracted user command intention features as input to construct a feature vector. Expand the feature set and combine the user's historical operation data, environmental information, etc. to improve the accuracy of mining. Input the feature vector into the selected demand mining model for real-time analysis. Based on the model output, generate the user's immediate navigation demand data, such as "go to the nearest exit", "avoid obstacles", etc. Select a suitable decision algorithm, such as a rule engine, fuzzy logic or machine learning model, to dynamically switch the navigation mode according to user needs. Design simple rules, such as switching to joystick control mode when the user's intention is "manual operation" and switching to smart navigation mode when the intention is "automatic navigation". Based on the stored instant navigation demand data, the selected decision algorithm is applied for analysis. Based on the user's instruction intention, the navigation mode to be used is determined and the system state is switched.

[0051] The current navigation mode state is saved to the system to ensure quick response when the user operates it. When switching navigation modes, provide user feedback, such as indicating the current mode through sound or visual, to improve the user experience.

[0052] In this embodiment, step S4 includes the following steps: Step S41: when the user's current navigation mode is the intelligent navigation mode, locating the user's preset destination based on the user's real-time wheelchair operation instructions; Step S42: dynamically planning a feasible path for the user's preset destination based on the dynamic environment map, thereby generating multiple feasible navigation paths; Step S43: performing path-by-path slope limit constraint analysis on the dynamic environment map to generate a slope limit constraint feature for each path; Step S44: dynamically adjusting the multiple feasible navigation paths based on the slope limit constraint characteristics of each path to generate a dynamically adjusted navigation path.

[0053] In this embodiment, the real-time wheelchair operation instructions extracted in the previous step are used to identify whether the user has a "set destination" instruction, and the destination information is obtained through a joystick, button or voice input. The instruction threshold is set. If the user continuously inputs the signal of setting the destination for more than 3 times, the system will consider the user's intention to set the destination. If the wheelchair is equipped with a GPS module, the user's current coordinates are located by GPS, and the user's movement direction is determined in combination with IMU (inertial measurement unit) sensor data (such as speed and acceleration). Through sensor fusion technology, the user's location information is ensured to be obtained in real time, the accuracy of positioning is improved, and the destination information entered by the user is combined with the current coordinates to generate the coordinate information of the user's preset destination. The system obtains information from the wheelchair, uses a map API (such as Google Maps API) to convert the address into geographic coordinates, saves the user's preset destination in the system for subsequent path planning, selects an appropriate path planning algorithm, such as the A algorithm, Dijkstra algorithm, or RRT (rapid random tree), and the A algorithm is suitable for path planning in a dynamic environment due to its high efficiency and adaptability. Set a heuristic function based on the characteristics of the wheelchair to optimize the efficiency of path planning. Use the dynamic environment map created in the previous steps to ensure that the map contains the user's current location, obstacle information, and traversable areas. Update the map information, obtain environmental changes (such as new obstacles) in real time, and input the user's preset destination and current location into the path. In the path planning algorithm, multiple feasible navigation paths are generated, and the length, obstacle avoidance and safety of the path are set as optimization goals. The generated path information is recorded, including the coordinate points, length and estimated time of each path, etc. The slope limitation analysis method is selected. The geometric analysis method is usually used to calculate the slope of each point on the path. The chain rule and derivative calculation are used to analyze the height change of the path. The slope threshold is set. For electric wheelchairs, the slope should be limited to between 5% and 10% to ensure safety. For each generated path, the slope of the path coordinates is calculated point by point, using the formula: slope = (height change / horizontal distance) × 100%. The slope characteristic data of each path is generated, and the maximum slope and minimum slope of each path are recorded. and its changing trend, store the slope limit characteristics of each path in the database for subsequent path adjustment, select the appropriate dynamic path adjustment strategy, and use the path smoothing method based on slope adjustment to smooth the path using Bezier curves or spline curves. Set the threshold for dynamic adjustment to ensure that the adjusted path is within a safe range, analyze each path to check whether its slope characteristics exceed the set safety threshold. If the slope of a path is too large, use the selected path smoothing method to regenerate the path to ensure that its slope is within a safe range. Update the adjusted path information to the system, record the final coordinate points, length, and safety characteristics of each path, and provide a visual display on the user interface.

[0054] In this embodiment, step S5 includes the following steps: Step S51: performing a dynamic object motion trend analysis on the dynamic environment map to generate a motion trend feature of each dynamic object; Step S52: performing dynamic movement trajectory prediction on the movement trend characteristics of each dynamic object to generate a movement prediction trajectory of each object; Step S53: performing path intersection identification on the dynamically adjusted navigation path according to the predicted movement trajectory of each object, and marking potential collision path points; Step S54: performing adaptive collision path avoidance based on potential collision path points to construct a collision avoidance navigation path.

[0055] In this embodiment, computer vision technology is used to detect dynamic objects (such as pedestrians, vehicles, etc.) in real time, and deep learning models such as YOLO or SSD are used for object detection to ensure efficiency and accuracy. A detection threshold is set, and only objects with a confidence level higher than 0.6 are considered to be effectively detected. The position information of each dynamic object in consecutive frames is recorded to generate the motion trajectory data of the object. A Kalman filter is used to smooth the trajectory and reduce the influence of noise. A time interval (such as 30 frames per second) is set for data collection to ensure the accuracy of motion trend analysis. A time series analysis is performed on the position information of each dynamic object, and its speed, acceleration and motion direction changes are calculated to generate the motion trend characteristics of the object. The average and standard deviation of the speed, acceleration and direction changes are recorded, and a suitable trajectory prediction model is selected, such as a linear regression model, LSTM (long short-term memory network) or Kalman filter. LSTM is suitable for processing time series data. , which can capture the motion pattern of dynamic objects. If LSTM is used, a training data set needs to be prepared so that the model can learn the motion laws of dynamic objects. The motion trend features extracted in the previous step are used as input to construct a feature vector, including information such as speed, acceleration, and direction. The prediction time window is set, such as predicting the trajectory within the next 3 seconds. Data of the corresponding time step is generated according to the sampling frequency. The feature vector is input into the selected prediction model to generate the predicted movement trajectory of each dynamic object, record the predicted trajectory coordinates of each object, generate time series data of future positions, and store the predicted movement trajectory of each dynamic object in the database to ensure that the latest trajectory information can be obtained during subsequent path intersection recognition. Select a suitable path intersection recognition algorithm, such as a geometric collision detection algorithm or a distance-based intersection detection method. Use space partitioning technology (such as quadtree) to optimize detection efficiency. Set the threshold for intersection detection and set the safety distance to 0.5 meters. When the distance between the predicted trajectory of an object and the navigation path is less than this threshold, it is marked as a potential collision point. For each dynamically adjusted navigation path, the predicted movement trajectory of each dynamic object is traversed, and the distance between the predicted trajectory of each object and the navigation path is calculated. If the distance is less than the set safety distance, the point is marked as a potential collision path point. The marked potential collision path points are stored in the database, and the coordinates of the collision point and related dynamic object information are recorded for subsequent avoidance processing. Appropriate collision avoidance strategies are selected, such as speed adjustment, path replanning, or dynamic obstacle avoidance algorithm. The RRT* (Rapid Random Tree) algorithm is used for dynamic path adjustment. The avoidance priority is set, such as giving priority to the safety of the path, followed by the smoothness and efficiency of the path. When a potential collision path point is detected, the RRT* algorithm is used to generate a new avoidance path in combination with the current navigation path to ensure that the new path avoids all potential collision points while maintaining proximity to the user's preset destination. The new collision avoidance navigation path information is updated to the system, and the adjusted path coordinates and related parameters are recorded to ensure that the system can execute smoothly when the user operates. .

[0056] In this embodiment, the specific steps of step S6 are: Step S61: when the user's current navigation mode is the joystick control mode, the joystick control mode is personalized and evolved to generate user personalized features; Step S62: adjusting the dynamic wheelchair control parameters based on the user's personalized characteristics to generate dynamic control adjustment parameters; Step S63: Perform intelligent collaborative control optimization on the collision avoidance navigation path and dynamic control adjustment parameters to build an intelligent navigation control model.

[0057] In this embodiment, the user's operation behavior is monitored in real time through sensors and input devices (such as joysticks), including the displacement, pressing force and frequency of use of the joystick, and a monitoring time window is set. For example, the operation data is recorded every 100ms during the user operation to ensure the real-time and accuracy of the data. The collected user operation data is stored in a database to form a user operation data set. Data clustering and statistical analysis methods (such as K-means clustering and principal component analysis) are applied to identify the user's operation mode and extract personalized features, such as operation sensitivity, reaction time and common operation methods. According to the analysis results of the user's operation behavior, personalized feature parameters are defined. Sensitivity parameter: reflects the user's reaction sensitivity to the joystick operation. The user's preferred direction of movement during operation (such as forward and backward), use machine learning models (such as decision trees) to model the user's personalized characteristics for subsequent use, determine the key parameters of wheelchair control, including speed, acceleration, steering sensitivity, and brake response time, use personalized characteristics as input, set the adjustment range of each parameter, and for the sensitivity parameter, set the initial value to 0.5, with a range of 0.1 to 1.0. Based on the user's personalized characteristics, use PID control algorithm (proportional-integral-differential control) or fuzzy control algorithm to perform dynamic parameter adjustment and set control rules. For example, when the user's operation sensitivity increases, the speed and acceleration of the wheelchair are increased accordingly, and the steering radius is reduced to improve the flexibility of operation. A real-time monitoring mechanism is implemented to dynamically adjust the control parameters according to the user's operation feedback. If the user frequently adjusts the joystick within a certain period of time, the system should increase its sensitivity parameters accordingly. Select a suitable control optimization model, such as Reinforcement learning models (such as Q-learning) or multi-agent systems (MAS) are used to achieve collaborative optimization of paths and control parameters, set optimization goals, such as minimizing travel time, avoiding collisions, and improving user comfort, integrate the generated collision avoidance navigation path and dynamic control adjustment parameters into the control model to achieve intelligent collaborative control, design decision algorithms, analyze the user's current operating status and environmental information in real time, dynamically adjust the navigation path and control parameters, use reinforcement learning algorithms, continuously optimize path selection and control parameters based on environmental feedback (such as collision risk, user comfort), guide the algorithm to learn to select the best combination of paths and control parameters through reward mechanisms, simulate different scenarios for testing, analyze performance and user feedback in different environments, adjust control strategies, and store the optimized navigation control model and parameters in the system for future use. Provide a user feedback mechanism to allow users to evaluate the navigation experience and further optimize the control model.

[0058] In this embodiment, a multi-mode navigation control system of an electric wheelchair controller is provided, which is used to execute the multi-mode navigation control method of the electric wheelchair controller as described above, including: The video enhancement module is used to obtain real-time environmental monitoring video based on the high-definition camera of the electric wheelchair; perform video jitter distortion compensation and frame-by-frame continuous optical flow tracking of dynamic objects on the real-time environmental monitoring video to generate the movement trajectory of each dynamic object; A dynamic environment modeling module is used to accurately identify static obstacles in the environment based on the real-time environment monitoring video, and to perform dynamic environment perception modeling based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map; A navigation mode decision module is used to identify the user's real-time wheelchair operation instructions; based on the user's real-time wheelchair operation instructions, a navigation mode switching decision is made to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode; The path planning module is used to perform dynamic feasible path planning and path-by-path slope constraint analysis based on the dynamic environment map when the user's current navigation mode is the intelligent navigation mode, so as to generate a dynamically adjusted navigation path; The collision avoidance module is used to predict the moving trajectory of dynamic objects in the dynamic adjustment navigation path based on the dynamic environment map, and to perform adaptive collision path avoidance to construct a collision avoidance navigation path; The intelligent navigation control module is used to adjust the dynamic wheelchair control parameters of the joystick control mode when the user's current navigation mode is the joystick control mode, and to perform intelligent collaborative control optimization based on the collision avoidance navigation path to build an intelligent navigation control model.

[0059] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0060] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A multi-mode navigation control method for an electric wheelchair controller, characterized in that: The following steps are involved: Step S1: acquiring a real-time environment monitoring video based on a high-definition camera of the electric wheelchair; performing video jitter distortion compensation and frame-by-frame continuous optical flow tracking of dynamic objects on the real-time environment monitoring video to generate a moving trajectory of each dynamic object; Step S2: accurately identifying static obstacles in the environment on the real-time environment monitoring video, and performing dynamic environment perception modeling based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map; Step S3: Identifying the user's real-time wheelchair operation instructions; Making a navigation mode switching decision based on the user's real-time wheelchair operation command to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode; Step S4: When the user's current navigation mode is the intelligent navigation mode, dynamic feasible path planning and path-by-path slope restriction constraint analysis are performed based on the dynamic environment map to generate a dynamically adjusted navigation path; Step S5: predicting the moving trajectory of the dynamic object in the dynamic adjustment navigation path based on the dynamic environment map, and performing adaptive collision path avoidance to construct a collision avoidance navigation path; Step S6: When the user's current navigation mode is the joystick control mode, the dynamic wheelchair control parameters of the joystick control mode are adjusted, and intelligent collaborative control optimization is performed based on the collision avoidance navigation path to build an intelligent navigation control model.

2. The multi-mode navigation control method of the electric wheelchair controller according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: Acquire real-time environmental monitoring video based on the electric wheelchair high-definition camera; Step S12: performing global contrast optimization on the real-time environment monitoring video to generate a globally optimized monitoring video; Step S13: performing video jitter distortion compensation on the global optimized monitoring video to construct a jitter distortion compensated monitoring video; Step S14: Performing dynamic object visual recognition on the jitter distortion compensation monitoring video, marking each dynamic object in the environment; Step S15: Perform frame-by-frame continuous optical flow tracking on each dynamic object in the environment to generate a moving trajectory of each dynamic object.

3. The multi-mode navigation control method of the electric wheelchair controller according to claim 2, characterized in that: The specific steps of step S12 are: Extracting timestamp information of the real-time environment monitoring video; Perform video frame synchronization optimization according to the timestamp information to obtain a time-synchronized monitoring video; Perform non-local mean denoising on the time-synchronized surveillance video to obtain the noise-filtered surveillance video; Divide the noise-filtered surveillance video into multiple regions to obtain multiple video regions; Performing color feature analysis on multiple video regions one by one to generate a color feature of each region; According to the color characteristics of each area, the color deviation between areas is identified to obtain the color deviation area; Performing local color correction processing on the color deviation area to obtain a color deviation correction area; The color deviation correction region is used to perform global contrast optimization on the noise-filtered surveillance video, thereby generating a globally optimized surveillance video.

4. The multi-mode navigation control method of the electric wheelchair controller according to claim 2, characterized in that: The specific steps of step S13 are: Decomposing the time-series image frames based on the global optimization surveillance video to obtain the image frame sequence; Label the environmental feature points of the global optimized surveillance video and extract multiple key environmental feature points; Calculating the relationship between feature points of multiple key environmental feature points according to the image frame sequence to generate a matching relationship between feature points of adjacent frames; The inter-frame global transformation parameters are estimated according to the feature point matching relationship between adjacent frames, thereby obtaining the image frame global transformation parameters; Based on the global transformation parameters of the image frames, the image frame sequence is subjected to frame-by-frame inverse transformation to obtain jitter distortion compensated image frames; The jitter-distortion-compensated image frames are temporally smoothed to construct jitter-distortion-compensated surveillance videos.

5. The multi-mode navigation control method of the electric wheelchair controller according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: performing a road condition deep semantic analysis on the jitter distortion compensation monitoring video to generate a road condition semantic feature; Step S22: accurately identifying environmental static obstacles based on the semantic features of the road conditions, and marking each environmental static obstacle; Step S23: performing geometric morphology mining on each static obstacle in the environment to generate geometric morphology information of each obstacle; Step S24: performing obstacle positioning calculation on the geometric information of each obstacle to obtain the environmental positioning coordinates of each obstacle; Step S25: performing an environmental three-dimensional structure analysis on the environmental positioning coordinates of each obstacle based on the semantic features of the road condition to generate an environmental three-dimensional structure diagram; Step S26: Dynamic environment perception modeling is performed on the three-dimensional structure diagram of the environment based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map.

6. The multi-mode navigation control method of the electric wheelchair controller according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: Identify the user's real-time wheelchair operation instructions; Step S32: analyzing the user's real-time wheelchair operation instructions and extracting user instruction intention features; Step S33: mining instant navigation needs according to the user's instruction intention characteristics, thereby generating the user's instant navigation needs data; Step S34: making a navigation mode switching decision based on the user's real-time navigation demand data to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode.

7. The multi-mode navigation control method of the electric wheelchair controller according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: when the user's current navigation mode is the intelligent navigation mode, locating the user's preset destination based on the user's real-time wheelchair operation instructions; Step S42: dynamically planning a feasible path for the user's preset destination based on the dynamic environment map, thereby generating multiple feasible navigation paths; Step S43: performing path-by-path slope limit constraint analysis on the dynamic environment map to generate a slope limit constraint feature for each path; Step S44: dynamically adjusting the multiple feasible navigation paths based on the slope limit constraint characteristics of each path to generate a dynamically adjusted navigation path.

8. The multi-mode navigation control method of the electric wheelchair controller according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: performing a dynamic object motion trend analysis on the dynamic environment map to generate a motion trend feature of each dynamic object; Step S52: performing dynamic movement trajectory prediction on the movement trend characteristics of each dynamic object to generate a movement prediction trajectory of each object; Step S53: performing path intersection identification on the dynamically adjusted navigation path according to the predicted movement trajectory of each object, and marking potential collision path points; Step S54: performing adaptive collision path avoidance based on potential collision path points to construct a collision avoidance navigation path.

9. The multi-mode navigation control method of the electric wheelchair controller according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: when the user's current navigation mode is the joystick control mode, the joystick control mode is personalized and evolved to generate user personalized features; Step S62: adjusting the dynamic wheelchair control parameters based on the user's personalized characteristics to generate dynamic control adjustment parameters; Step S63: Perform intelligent collaborative control optimization on the collision avoidance navigation path and dynamic control adjustment parameters to build an intelligent navigation control model.

10. A multi-mode navigation control system for an electric wheelchair controller, characterized in that: A multi-mode navigation control method for executing the electric wheelchair controller according to claim 1, comprising: The video enhancement module is used to obtain real-time environmental monitoring video based on the high-definition camera of the electric wheelchair; perform video jitter distortion compensation and frame-by-frame continuous optical flow tracking of dynamic objects on the real-time environmental monitoring video to generate the movement trajectory of each dynamic object; A dynamic environment modeling module is used to accurately identify static obstacles in the environment based on the real-time environment monitoring video, and to perform dynamic environment perception modeling based on the movement trajectory of each dynamic object, thereby generating a dynamic environment map; A navigation mode decision module is used to identify the user's real-time wheelchair operation instructions; based on the user's real-time wheelchair operation instructions, a navigation mode switching decision is made to obtain the user's current navigation mode, wherein the user's current navigation mode includes a joystick control mode and an intelligent navigation mode; The path planning module is used to perform dynamic feasible path planning and path-by-path slope constraint analysis based on the dynamic environment map when the user's current navigation mode is the intelligent navigation mode, so as to generate a dynamically adjusted navigation path; The collision avoidance module is used to predict the moving trajectory of dynamic objects in the dynamic adjustment navigation path based on the dynamic environment map, and to perform adaptive collision path avoidance to construct a collision avoidance navigation path; The intelligent navigation control module is used to adjust the dynamic wheelchair control parameters of the joystick control mode when the user's current navigation mode is the joystick control mode, and to perform intelligent collaborative control optimization based on the collision avoidance navigation path to build an intelligent navigation control model.

Images