Autonomous driving method of intelligent manned mobile robot capable of efficiently avoiding obstacles

Abstract

The invention discloses an intelligent manned mobile robot autonomous driving method capable of efficiently avoiding obstacles, belongs to the technical field of intelligent robot navigation control, and aims to solve the problems of lack of pedestrian intention pre-judgment ability, insufficient ground friction characteristic perception and lack of a control parameter dynamic adjustment mechanism in the prior art. Comprising the following steps: S1, synchronously acquiring video stream data of a pedestrian face area and three-dimensional point cloud data output by a depth sensor; s2, extracting eyelid movement frequency features and mouth angle displacement gradient features from the video stream data; according to the invention, through collecting pedestrian face video stream data, extracting eyelid motion frequency features and mouth angle displacement gradient features, and combining three-dimensional point cloud data, a deviation angle between a sight line direction and a head orientation is calculated, and a multi-feature fusion intention prediction model is constructed. And through collaborative analysis of micro-expressions and attitude deviations, a probability distribution matrix of pedestrian moving directions is output in advance.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent robot navigation and control, and in particular relates to an autonomous driving method for an intelligent manned mobile robot with efficient obstacle avoidance. Background Art

[0002] With the rapid development of intelligent technology and robotics, intelligent manned mobile robots are increasingly used in medical transportation, elderly care assistance, commercial services and other fields. Their core functional requirement is to achieve safe and efficient autonomous obstacle avoidance in complex dynamic environments. How to ensure that robots avoid obstacles during autonomous driving and ensure driving safety has become an urgent problem to be solved.

[0003] However, in high-density crowd flow environments such as hospital corridors and nursing home activity areas, existing technologies have the following significant drawbacks: 1. Lack of ability to predict pedestrian intentions: Traditional obstacle avoidance technologies primarily rely on distance detection devices such as lidar and ultrasonic sensors, combined with visual SLAM technology for obstacle location and path planning. These solutions rely solely on physical distance detection and are unable to predict pedestrian movement intentions without significant displacement. When a pedestrian suddenly turns, the robot can only initiate obstacle avoidance after detecting the position change, resulting in a delayed response. More critically, existing systems fail to model the correlation between pedestrian micro-expressions, such as eyelid movement and mouth corner changes, and movement direction, making them unable to capture early biometric signals of turning intentions. Second, insufficient perception of ground friction characteristics: In environments with mixed materials, such as areas with alternating marble and anti-slip mats, different materials have significant impacts on the robot's braking performance. Traditional solutions lack real-time perception of the ground's friction coefficient, resulting in fixed braking parameters and an inability to adapt to sudden material changes, leading to unstable braking and even skidding. 3. Lack of dynamic adjustment mechanism for control parameters: The existing system suffers from a broken control loop. The path planning module does not integrate pedestrian intention prediction data, and the motion controller (such as PID parameters) is not linked to real-time adjustment of the ground friction coefficient. For example, using a high integral coefficient on low-friction ground will cause path tracking overshoot. This static control strategy is very likely to cause low obstacle avoidance efficiency and safety hazards in mixed-material ground scenarios.

[0004] Therefore, an efficient obstacle avoidance and autonomous driving method for intelligent manned mobile robots is needed to solve the problems existing in the existing technology, such as the lack of ability to predict pedestrian intentions, insufficient perception of ground friction characteristics, and lack of a dynamic adjustment mechanism for control parameters. Summary of the Invention

[0005] The purpose of the present invention is to provide an efficient obstacle avoidance method for autonomous driving of an intelligent manned mobile robot, so as to solve the problems raised in the above background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance, comprising the following steps: S1, synchronously collects the video stream data of the pedestrian's facial area and the three-dimensional point cloud data output by the depth sensor; S2. extracting eyelid movement frequency features and mouth corner displacement gradient features from the video stream data, and calculating the deviation angle between the line of sight direction and the head orientation based on the three-dimensional point cloud data; S3, inputting the eyelid movement frequency characteristics, mouth corner displacement gradient characteristics and deviation angle into a pre-trained intention prediction model, and outputting a probability distribution matrix of the pedestrian's movement direction; S4. Obtain the ground reflection intensity attenuation curve and simultaneously collect the high-frequency vibration spectrum of the robot chassis; S5. Calculating a surface roughness index according to the reflection intensity attenuation curve, and extracting an energy proportion value in a frequency band of 500 to 1500 Hz from the high-frequency vibration spectrum; S6. Querying a material feature database based on the surface roughness index and the energy proportion value to determine the dynamic friction coefficient μ of the current ground surface; S7. Generate a pre-obstacle avoidance path according to the probability distribution matrix, and adjust the PID parameters of the path tracking controller based on the dynamic friction coefficient μ; S8. When it is detected that the pedestrian turning probability is greater than a preset threshold and the dynamic friction coefficient μ is lower than a safe value, the graded braking strategy is triggered to control the robot to decelerate.

[0007] It should be noted that the extraction of eyelid movement frequency features in step S2 includes: Perform adaptive histogram equalization on the eye area of ​​video stream data; The eyelid opening and closing state is detected through the first convolutional layer of the convolutional neural network, and an 8×8×16 feature map is output; The displacement speed of the pupil center coordinates within 5 consecutive frames was calculated, and when the speed exceeded 3 pixels / frame, it was marked as a rapid eye movement event.

[0008] It is worth noting that the calculation of the mouth corner displacement gradient feature in step S2 includes: Track pixel changes in the nasolabial fold area through the second convolutional layer of the convolutional neural network; Establish a polar coordinate system with the nose tip as the origin and calculate the radial displacement of the mouth corner key points between each frame; When the displacement gradient of the left mouth corner is greater than 0.15 pixels / frame for 3 frames, a left turn intention marker is generated.

[0009] It should be further explained that the construction of the intention prediction model in step S2 satisfies: If the eyelid flutter frequency is >3 Hz, the mouth corner displacement gradient is >0.15 pixels / frame, and the deviation angle is >15°, the turning probability in the corresponding direction is increased by 0.6; The probability values ​​of turning left, going straight, and turning right are normalized using the Softmax function.

[0010] As a preferred embodiment, the calculation of the surface roughness index in step S5 includes: Perform first-order differential processing on the reflection intensity attenuation curve and take the absolute value of the slope in the 0-100 nanosecond time window; When the absolute value of the slope is <0.2 / nanosecond, it is judged as a highly smooth material and assigned a roughness index of 0.2-0.4.

[0011] As a preferred embodiment, the processing of the high-frequency vibration spectrum in step S5 includes: Perform fast Fourier transform on the vibration signal and calculate the ratio of the energy in the 500-1500 Hz frequency band to the total energy; When the ratio is between 60% and 65% and the surface roughness index is 0.2-0.4, it is judged to be terrazzo floor.

[0012] As a preferred embodiment, the adjustment rule of the PID parameters in step S7 is: If μ<0.3, set Kp=120, Ki=0, Kd=30; If μ ≥ 0.3, set Kp = 80, Ki = 10, and Kd = 40.

[0013] As a preferred embodiment, the hierarchical braking strategy of step S8 includes: Phase 1: When the steering probability is greater than 0.6 and μ is less than 0.4, the motor torque is linearly reduced to 50% of the rated value; Phase 2: When the distance between the robot and the pedestrian is less than 1 meter, the magnetorheological brake is activated to apply reverse damping force.

[0014] As a preferred embodiment, it also includes an exception handling mechanism: when the intention prediction model output confidence is <50% and contradictory features are detected in three consecutive frames, a stop command is sent to the motion controller and the 360° circular warning light strip is activated.

[0015] As a preferred implementation, the generation of the stop instruction includes: Get the historical path data within the last 0.5 seconds; Calculate the optimal retreat vector based on historical path data; Generate a temporary path with 5 waypoints in a zigzag trajectory, with the distance between waypoints ≤ 30 cm.

[0016] Compared with the prior art, the present invention provides an efficient obstacle avoidance method for autonomous driving of an intelligent manned mobile robot, which has at least the following beneficial effects: (1) By collecting pedestrian facial video stream data, extracting eyelid movement frequency features and mouth corner displacement gradient features, and combining them with three-dimensional point cloud data to calculate the deviation angle between the gaze direction and the head orientation, a multi-feature fusion intention prediction model was constructed. When the pedestrian has not undergone obvious displacement, the probability distribution matrix of the pedestrian's movement direction can be output in advance through the collaborative analysis of micro-expressions and posture deviations. This mechanism breaks through the limitation of traditional detection that only relies on physical distance, establishes a correlation model between pedestrian micro-expressions and movement direction, and advances the reaction time of obstacle avoidance decision-making to the stage when the pedestrian intends to turn, effectively solving the collision risk caused by delayed reaction in high-density crowd flow scenes.

[0017] (2) By synchronously acquiring the ground reflection intensity attenuation curve and the chassis high-frequency vibration spectrum, calculating the surface roughness index and the energy proportion of the 500-1500Hz frequency band, and combining the material feature database to accurately determine the dynamic friction coefficient, the real-time identification of the ground material and the quantification of the friction characteristics are realized, which solves the problem of insufficient perception of the friction coefficient of mixed multi-material ground by traditional solutions. Through real-time feedback of the dynamic friction coefficient, the braking risk of low-friction roads can be predicted in advance, avoiding braking skidding or instability caused by material mutation, and significantly improving driving safety in complex ground environments.

[0018] (3) By generating a pre-obstacle avoidance path based on the probability distribution matrix of the pedestrian's movement direction and adaptively adjusting the PID parameters of the path tracking controller according to the real-time dynamic friction coefficient, when a high turning probability and low friction coefficient of the pedestrian are detected, a graded braking strategy is triggered, opening up the full closed loop of "pedestrian intention prediction-ground characteristic perception-control parameter adjustment", solving the problems of static control parameters and disconnection between path planning and environmental dynamics in traditional systems, and deeply integrating pedestrian intention data with ground friction characteristics to match the path tracking accuracy and braking response with the real-time environment, avoiding path overshoot on low-friction ground and achieving smooth deceleration in high-risk scenarios, which greatly improves obstacle avoidance efficiency and passenger safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of the steps provided by the present invention; Figure 2 This is a flow chart of the hierarchical braking strategy provided by the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] The present invention will be further described below with reference to the embodiments.

[0022] See also Figure 1-2 The present invention provides an efficient obstacle avoidance method for autonomous driving of an intelligent manned mobile robot, comprising the following steps: S1, synchronously collects the video stream data of the pedestrian's facial area and the three-dimensional point cloud data output by the depth sensor; S2, extracting eyelid movement frequency features and mouth corner displacement gradient features from the video stream data, and calculating the deviation angle between the gaze direction and the head orientation based on the 3D point cloud data; S3: Input the eyelid movement frequency features, mouth corner displacement gradient features, and deviation angle into the pre-trained intention prediction model and output the probability distribution matrix of the pedestrian's movement direction; S4. Obtain the ground reflection intensity attenuation curve and simultaneously collect the high-frequency vibration spectrum of the robot chassis; S5. Calculate the surface roughness index based on the reflection intensity attenuation curve and extract the energy proportion value of the 500-1500 Hz frequency band from the high-frequency vibration spectrum; S6. Querying a material feature database based on the surface roughness index and the energy proportion value to determine the dynamic friction coefficient μ of the current ground surface; S7, generating a pre-obstacle avoidance path according to the probability distribution matrix, and adjusting the PID parameters of the path tracking controller based on the dynamic friction coefficient μ; S8. When it is detected that the pedestrian turning probability is greater than a preset threshold and the dynamic friction coefficient μ is lower than a safe value, the graded braking strategy is triggered to control the robot to decelerate.

[0023] Further as Figure 1 As shown, it is worth noting that the extraction of eyelid movement frequency features in step S2 includes: Perform adaptive histogram equalization on the eye area of ​​video stream data; The eyelid opening and closing state is detected through the first convolutional layer of the convolutional neural network, and an 8×8×16 feature map is output; The displacement speed of the pupil center coordinates within 5 consecutive frames was calculated, and when the speed exceeded 3 pixels / frame, it was marked as a rapid eye movement event.

[0024] Among them, adaptive histogram equalization processing refers to improving the image quality of the eye area through local contrast enhancement technology. Specifically, it can be implemented using a block histogram equalization algorithm to eliminate the interference of uneven lighting on eyelid status detection.

[0025] Among them, the first convolutional layer of the convolutional neural network detects the eyelid opening and closing state, which means extracting the spatial features of the eye area through the trained convolution kernel. Specifically, a 3×3 convolution kernel can be used for edge detection to identify the geometric changes of the eyelid contour.

[0026] Among them, rapid eye movement event marking refers to establishing a motion trajectory analysis model based on the dynamic changes of pupil center coordinates. Specifically, the optical flow method can be used to calculate the displacement vector between consecutive frames to capture the instantaneous behavioral characteristics of pedestrian attention shifting.

[0027] Specifically, after the eye area is processed with histogram equalization, the contrast of the eyelid edges in the image is significantly enhanced, allowing the subsequent convolutional layers to more accurately capture the morphological changes during the eyelid opening and closing process; the feature map output by the first convolutional layer retains the temporal characteristics of eyelid movement through spatial dimensionality reduction, and combined with the displacement speed analysis of the pupil center between consecutive frames, it can effectively distinguish between normal blinking and intentional rapid eye movement; when a rapid eye movement exceeding the threshold is detected, the feature is associated with the pedestrian's possible intention to change direction.

[0028] Compared with existing technologies, traditional methods usually only detect eye closure status through a single frame image, without considering the motion speed characteristics in dynamic sequences; this solution, by fusing image enhancement, multi-frame motion analysis and neural network feature extraction, can accurately identify rapid eye movement events under complex lighting conditions, providing high-confidence biometric data for subsequent pedestrian intention prediction.

[0029] Through the above technical solution, the present application solves the problem of prediction delay caused by insufficient accuracy of eye micro-movement detection in the existing technology, can capture subtle signs of pedestrians' attention shift in advance, and reduce the risk of sudden stops caused by misjudging the direction of pedestrians' movement.

[0030] Further as Figure 1 As shown, it is worth noting that the calculation of the mouth corner displacement gradient feature in step S2 includes: Track pixel changes in the nasolabial fold area through the second convolutional layer of the convolutional neural network; Establish a polar coordinate system with the nose tip as the origin and calculate the radial displacement of the mouth corner key points between each frame; When the displacement gradient of the left mouth corner is greater than 0.15 pixels / frame for 3 frames, a left turn intention marker is generated.

[0031] Pixel change tracking in the nasolabial fold area monitors the dynamic changes in the skin texture around the mouth. Specifically, a convolution kernel with 32 filters is used for feature extraction to capture subtle facial muscle movements. The establishment of a polar coordinate system means setting angle and distance parameters with the nose tip as the origin. Specifically, the position of the mouth corner can be determined by the key point detection algorithm, and the two-dimensional plane coordinates can be converted into polar coordinate parameters. The radial displacement calculation refers to analyzing the position differences of the key points of the mouth corners between consecutive frames. It can be implemented by using the optical flow method or the difference algorithm to quantify the amplitude of the mouth movement. Generating a left-turn intention mark refers to judging the movement trend based on a preset threshold. For example, when the displacement in a specific direction is detected to continuously reach a threshold, a logical judgment is triggered to characterize the pedestrian's potential turning intention.

[0032] Specifically, subtle facial movement features are extracted through convolution operations on the nasolabial groove area, and the movement trajectory of the mouth corners is mapped to the polar coordinate system to eliminate the errors caused by changes in head position; the inter-frame calculation of radial displacement can effectively distinguish between natural expressions and muscle movements related to turning intentions. When the displacement gradient of the left corner of the mouth continuously exceeds the set threshold, it indicates that the pedestrian may be preparing to turn left; this judgment mechanism can predict the changes in the pedestrian's movement direction in advance, providing earlier warning signals for the robot's obstacle avoidance decisions.

[0033] Compared with existing technologies, traditional methods rely solely on head orientation or body posture for direction prediction and are unable to capture turning intentions conveyed by micro-expressions. This solution analyzes the biomechanical characteristics of the mouth corner displacement gradient and combines it with continuity judgment in time series to identify pedestrians' turning trends earlier, effectively compensating for the lag defect of traditional visual detection methods.

[0034] Through the above technical solution, the present application realizes the early recognition of pedestrians' turning intentions, enabling the robot to adjust the obstacle avoidance path before the pedestrians actually change the direction of movement, significantly improving the safety of human-machine collaboration in high-density pedestrian flow environments.

[0035] Further as Figure 1 As shown, it is worth noting that the construction of the intention prediction model in step S2 satisfies: If the eyelid flutter frequency is >3 Hz, the mouth corner displacement gradient is >0.15 pixels / frame, and the deviation angle is >15°, the turning probability in the corresponding direction is increased by 0.6; The probability values ​​of turning left, going straight, and turning right are normalized using the Softmax function.

[0036] The eyelid flutter frequency refers to the number of times the eyelids open and close per unit time. It can be achieved by calculating the area change rate of the pupil region in consecutive video frames, and is used to reflect the pedestrian's attention shift state. The mouth corner displacement gradient refers to the rate of movement of the mouth corner during facial expression changes. This can be achieved by tracking the position offset of the key points of the nasolabial groove using the optical flow method. It is used to characterize the pedestrian's potential turning intention. The deviation angle refers to the spatial angle between the gaze direction and the actual head orientation. It can be calculated based on the relative position relationship between the eye center and the head center of mass in a 3D point cloud coordinate system and is used to identify the direction of pedestrian attention deviation. The Softmax function is a multi-class probability normalization method that converts the original prediction value into a mutually exclusive distribution with a total probability of 1, which is used to generate interpretable pedestrian movement direction prediction results.

[0037] Specifically, during video stream processing, if the eyelid twitching frequency is detected to exceed 3Hz continuously, it indicates that the pedestrian may be in a state of high concentration. If the displacement gradient of the mouth corner continuously exceeds 0.15 pixels per frame and the spatial deviation angle between the head and the line of sight exceeds 15 degrees, the pedestrian is judged to have a clear tendency to change direction. This conditional trigger mechanism uses a weighted operation to increase the turning probability of the corresponding direction by 0.6, forming an initial probability matrix containing three dimensions: left turn, straight ahead, and right turn. Further exponential operation and normalization processing using the Softmax function generate a final decision matrix that conforms to the probability distribution characteristics, providing a quantitative basis for subsequent path planning.

[0038] Compared to existing technologies, traditional pedestrian intention prediction methods typically rely solely on single-modal data, such as analyzing head posture or body movements, while ignoring the synergistic effects of eye micro-expressions and facial expressions. Existing solutions often use linear weighting to calculate probabilities, which struggles to handle the nonlinear relationships between multiple features, resulting in insufficient confidence in prediction results.

[0039] Through the above technical solution, this application realizes the collaborative detection and probabilistic fusion of multi-dimensional biometric features, effectively improving the recognition accuracy of pedestrian turning intentions; by setting multiple dynamic threshold conditions, it can accurately capture the key turning points in pedestrian behavior patterns, and combined with the nonlinear probability normalization method, significantly enhance the robustness of the prediction model in complex scenarios, providing robots with more reliable prediction basis.

[0040] Further as Figure 1 As shown, it is worth noting that the calculation of the surface roughness index in step S5 includes: Perform first-order differential processing on the reflection intensity attenuation curve and take the absolute value of the slope in the 0-100 nanosecond time window; When the absolute value of the slope is <0.2 / nanosecond, it is judged as a highly smooth material and assigned a roughness index of 0.2-0.4.

[0041] The first-order differential processing of the lidar echo intensity curve refers to the differential operation of the curve of the reflection intensity changing with time. Specifically, it can be implemented using a numerical differential algorithm to capture the differences in the reflection intensity attenuation characteristics of different material surfaces. The absolute value of the slope within the time window refers to the maximum rate of change of the differential curve within the first 100 nanoseconds after the laser pulse is emitted. This can be achieved through sliding window difference calculation. This time window covers the effective detection range of the lidar for ground materials. The judgment condition for highly smooth materials is to use the absolute value of the slope of the differential curve less than 0.2 per nanosecond as the classification threshold. Specifically, this can be achieved using a supervised learning method based on a material sample library to distinguish between low-friction floor types such as tiles and marble.

[0042] Specifically, the laser radar collects raw data on the change of ground reflection intensity over time, and performs first-order differential processing on the echo signal to eliminate ambient light interference and enhance material characteristics; within the effective detection time window of 0-100 nanoseconds, the maximum absolute value of the slope of the differential curve is calculated. This parameter can characterize the difference in reflection characteristics of the ground microstructure; when the absolute value of the slope is detected to be less than the set threshold, it indicates that the ground reflection intensity changes smoothly, which is consistent with the physical properties of highly smooth materials. At this time, the corresponding low roughness index is assigned; this index is used as the basis for calculating the ground friction coefficient and is called by the subsequent path tracking control module.

[0043] Compared with existing technologies, traditional methods only detect obstacle distances through lidar point clouds, without considering the impact of ground material differences on the robot's braking performance. This solution analyzes the differential characteristics of laser echo characteristics and establishes a correlation model between ground material and friction coefficient, solving the problem of improper braking parameter setting in existing technologies due to ignoring ground material.

[0044] Through the above technical solution, this application can accurately identify highly smooth ground materials, provide a reliable basis for the robot to dynamically adjust braking parameters, avoid slipping on low-friction roads such as tiles and marble, and improve driving safety and control stability in complex ground environments.

[0045] Further as Figure 1 As shown, it is worth noting that the processing of the high-frequency vibration spectrum in step S5 includes: Perform fast Fourier transform on the vibration signal and calculate the ratio of the energy in the 500-1500 Hz frequency band to the total energy; When the ratio is between 60% and 65% and the surface roughness index is 0.2-0.4, it is judged to be terrazzo floor.

[0046] Among them, fast Fourier transform refers to a processing method for converting time-domain vibration signals into frequency-domain signals. It can be specifically implemented using a digital signal processing chip to extract the energy distribution characteristics of different frequency components.

[0047] The ratio of the energy in the 500-1500 Hz frequency band to the total energy refers to the proportion of the signal energy in a specific frequency range to the energy of the entire spectrum. Specifically, the sum of the energy of each frequency band can be calculated through integration to characterize the vibration response characteristics of the ground material to the robot chassis.

[0048] Among them, the surface roughness index refers to a quantitative parameter obtained by differential processing of the lidar echo intensity curve. Specifically, it can be associated with pre-stored data in the material feature database to collaboratively judge the ground friction characteristics.

[0049] Specifically, the time domain signal collected by the vibration sensor is processed by fast Fourier transform to generate a spectrum diagram containing the energy of different frequency components; by setting the target frequency band of 500-1500Hz, the ratio of the total energy in this interval to the total energy of the entire spectrum is calculated; when this ratio reaches a specific range of 60%-65%, combined with the surface roughness index calculated by the lidar data, by querying the pre-established material feature correspondence, the type of ground material with similar vibration response characteristics can be accurately identified; for example, when the energy ratio of this frequency band is detected at 62% and the surface roughness index is 0.35, it can be determined that the current ground is made of terrazzo.

[0050] Compared with existing technologies, traditional methods only rely on single-modal lidar data to determine ground type, which is prone to misjudgment due to interference from ambient light or surface reflective properties. This solution, by fusing vibration spectrum features with lidar data, constructs a multi-modal feature fusion ground recognition mechanism that can effectively distinguish ground materials with similar visual features but large differences in friction coefficients.

[0051] Through the above technical solution, this application can accurately identify the vibration response characteristics of special materials such as terrazzo, and dynamically calibrate the floor friction coefficient by combining surface roughness parameters. This dual-mode detection mechanism can avoid incorrect braking parameter settings caused by misjudgment by a single sensor, ensuring that the robot can adjust control parameters based on accurate floor characteristic data when driving on different floor materials, significantly improving the stability and safety during emergency braking.

[0052] Further as Figure 1 As shown, it is worth noting that the adjustment rules of the PID parameters in step S7 are: If μ<0.3, set Kp=120, Ki=0, Kd=30; If μ ≥ 0.3, set Kp = 80, Ki = 10, and Kd = 40.

[0053] The dynamic friction coefficient refers to the characteristic parameter of the sliding friction between the ground material and the robot's tires. It can be achieved by analyzing the laser radar echo intensity curve and matching the high-frequency vibration spectrum characteristics to characterize the degree of influence of different ground materials on the robot's motion control.

[0054] The proportional coefficient Kp refers to the gain parameter in the path tracking controller that is linearly related to the path deviation. This can be achieved by dynamically adjusting this parameter to change the system response speed, and is used to quickly correct the lateral deviation between the robot's movement direction and the planned path.

[0055] The integral coefficient Ki refers to the adjustment parameter in the controller for compensating the cumulative error. It can be achieved by eliminating the integral link of the steady-state error and is used to continuously correct the long-term path tracking deviation caused by the difference in ground friction.

[0056] The differential coefficient Kd refers to a parameter for predictive adjustment based on the error change rate. It can be achieved by performing a differential operation to suppress system overshoot, and is used to proactively respond to robot posture fluctuations caused by sudden changes in ground material.

[0057] Specifically, when the lidar and vibration sensor jointly detect that the dynamic friction coefficient of the ground is lower than 0.3, the path tracking controller automatically switches to a high-proportional, low-integral parameter combination, compensating for the risk of slipping on low-friction ground by enhancing the rapid response capability; when the friction coefficient rises to 0.3 or above, a parameter combination that balances proportion and integration is adopted to introduce an error correction function while ensuring the response speed; this parameter adjustment process is implemented through an online table lookup, dynamically matching the preset optimization parameter group according to the real-time calculated friction coefficient value.

[0058] Compared with existing technologies, traditional methods use fixed PID parameters for motion control, without considering the changes in friction coefficient caused by different ground materials, which leads to path tracking overshoot or braking lag on low-friction ground. This solution establishes a mapping relationship between friction coefficient and controller parameters, enabling the robot to automatically optimize the control strategy according to the real-time ground status, solving the technical defect that a single parameter cannot adapt to mixed ground materials.

[0059] Through the above technical solution, the present application realizes adaptive motion control under different ground friction conditions, effectively reduces the path tracking deviation caused by sudden changes in ground material, improves the robot's trajectory keeping ability on low-friction ground, and avoids excessive braking on high-friction ground, thereby ensuring the driving smoothness and safety during manned movement.

[0060] Further as Figure 2As shown, it is worth noting that the hierarchical braking strategy of step S8 includes: Phase 1: When the steering probability is greater than 0.6 and μ is less than 0.4, the motor torque is linearly reduced to 50% of the rated value; Phase 2: When the distance between the robot and the pedestrian is less than 1 meter, the magnetorheological brake is activated to apply reverse damping force.

[0061] The graded braking strategy refers to a control method that implements different braking intensities in stages. Specifically, it can be implemented using a dual-threshold trigger mechanism, which achieves progressive adjustment of the braking torque by setting trigger conditions at different stages. Motor torque linear reduction refers to the proportional reduction of the output torque of the drive system. This can be achieved by using a PID controller working in conjunction with the motor driver to achieve initial deceleration by reducing power output. The reverse damping force applied by the magnetorheological brake refers to the use of the viscosity change characteristics of the magnetorheological fluid to generate controllable resistance. Specifically, this can be achieved by adjusting the magnetic field strength using an electromagnetic coil, and the braking efficiency can be enhanced by adjusting the damping coefficient in real time.

[0062] Specifically, when it is detected that the probability of the pedestrian turning left exceeds the set threshold and the ground friction coefficient is insufficient, the robot first enters the first braking stage. At this time, the control signal of the drive motor is adjusted to reduce its output torque at a predetermined slope, for example, reducing the torque to half of the initial value within 0.5 seconds; when the robot continues to approach the pedestrian and the distance is shortened to the safety critical value, the second braking stage is automatically started. At this time, the electromagnetic coil of the magnetorheological brake adjusts the current intensity in real time according to the feedback from the distance sensor, generating a damping force field opposite to the direction of movement. The magnitude of the damping force is positively correlated with the current speed of the robot.

[0063] Compared with existing technologies, traditional braking schemes usually use a single threshold to trigger emergency stop operations, which can easily cause manned mobile robots to shake or slide violently; this scheme, on the other hand, uses staged braking control to achieve a step-by-step speed reduction while maintaining posture stability. Especially under low-friction ground conditions, combined with precise control of the reverse damping force, it effectively avoids lateral displacement caused by inertia.

[0064] Through the above technical solution, this application solves the technical problem in the prior art that slipping or excessive impact is prone to occur during emergency braking. It can achieve safe and smooth deceleration control under complex ground conditions, ensure that the robot maintains operational stability when avoiding and turning towards pedestrians in time, and reduce the risk of equipment wear and passenger discomfort caused by emergency braking.

[0065] Further as Figure 2As shown, it is worth mentioning that it also includes an exception handling mechanism: when the confidence of the intention prediction model output is less than 50% and contradictory features are detected in three consecutive frames, a stop command is sent to the motion controller and the 360° circular warning light strip is activated.

[0066] Among them, the output confidence refers to the quantitative value of the credibility of the model prediction result, which can be achieved by calculating the entropy value of the probability distribution. The higher the entropy value, the greater the uncertainty of the prediction result. Contradictory features refer to logical conflicts between different sensor data. Specifically, this can be achieved by the Euclidean distance of the feature vectors exceeding a preset threshold. For example, it is triggered when the eyelid movement feature and the mouth corner displacement feature point in different directions. A stop command is a control signal that immediately terminates the current motion command. This can be achieved by sending an emergency stop message via the CAN bus. A 360° circular warning light strip is a visual warning device arranged around the circumference of the robot. This can be implemented using an RGB LED array, for example, using PWM dimming control to produce a red, breathing effect.

[0067] Specifically, when the confidence level in pedestrian intention prediction is insufficient and contradictory features are detected, the system will enter safety protection mode; when the entropy value of the output probability distribution of the prediction model exceeds the preset threshold, combined with the feature difference analysis of continuous multi-frame sensor data, it is judged that unreliable detection conditions exist in the current environment. At this time, a shutdown command is sent through the emergency communication interface of the motion controller, and the ring light strip is driven into a warning state through the lighting control module. For example, in the nursing home corridor scenario, when the pedestrian's facial expression features and body posture features are detected at the same time pointing in opposite directions, the system actively stops moving to avoid the risk of misjudgment.

[0068] Compared with existing technologies, traditional obstacle avoidance systems continue to execute the original path planning when sensor data is abnormal, while this solution actively triggers safety protection by establishing a dual detection mechanism; existing technologies lack an evaluation system for the confidence of prediction models and are unable to identify contradictions in sensor data in complex environments, which can easily lead to misjudgments; this solution introduces a combined judgment condition of confidence threshold and contradiction feature detection to effectively improve the safety of the system in uncertain environments.

[0069] Through the above technical solution, this application can stop movement in time when the pedestrian intention prediction is unreliable or the sensor data is abnormal, avoiding the risk of collision due to misjudgment; through the instant feedback of the ring warning light, it can remind surrounding personnel to pay attention to the robot status and gain response time for the system to restore to a safe state; this mechanism is particularly suitable for scenes with complex reflective surfaces and dense personnel flow, such as hospital corridors, to ensure that manned robots can still maintain safe operation in uncertain environments.

[0070] Further as Figure 2As shown, it is worth noting that the generation of the stop instruction includes: Get the historical path data within the last 0.5 seconds; Calculate the optimal retreat vector based on historical path data; The specific calculation formula is as follows: ; Where, represents the optimal retreat vector, and represents the time step, Represents the time step The location coordinates of Indicates the preset minimum safety distance. Indicates the distance between the current robot and the nearest obstacle. Indicates the basic retreat range, It represents the risk sensitivity coefficient; Generate a temporary path with 5 waypoints in a zigzag trajectory, with the distance between waypoints ≤ 30 cm.

[0071] The optimal retreat vector refers to the best evacuation direction derived from the spatiotemporal continuity analysis of the robot's motion trajectory. This can be achieved by using a sliding window method to perform a weighted average calculation of historical paths. The vector generation process incorporates the obstacle distribution density and the robot's kinematic constraints into the calculation dimension. A zigzag trajectory refers to an obstacle avoidance path composed of continuous broken line segments. Specifically, it can be achieved by generating a waypoint sequence using an improved artificial potential field method. The spacing between adjacent waypoints is constrained by a path smoothness optimization algorithm.

[0072] Specifically, when the system detects a decrease in the confidence level of pedestrian behavior prediction and contradictory characteristics, it first extracts the time series data of the historical path and uses the Kalman filter algorithm to predict the movement trend within the next 0.2 seconds; combined with the real-time distribution of environmental obstacles, the azimuth and velocity components of the retreat vector are calculated through the quadratic programming algorithm; the path generation module uses the dynamic window method to generate a temporary path containing turning points based on the retreat vector parameters. The position error of each waypoint is controlled within the accuracy range of the positioning system to ensure that the robot can complete the avoidance action in a small space.

[0073] Compared with existing technologies, traditional exception handling solutions often use emergency braking or straight-line reversing strategies, which easily cause the risk of secondary collisions. This solution dynamically generates a retreat path through historical path data analysis, and combined with a Z-shaped trajectory design, it can effectively avoid sudden obstacles while maintaining the stability of the fuselage. The fixed retreat path generation method in the existing technology cannot adapt to dynamic environmental changes, while the waypoint sequence generation mechanism of this application can adjust the trajectory curvature in real time according to the ground friction coefficient.

[0074] Through the above technical solution, this application effectively solves the problem of rigid path planning during emergency braking, and can achieve safe evacuation in complex human flow environments; through the calculation of dynamic retreat vectors, it ensures that the robot chooses the optimal avoidance direction in emergency situations; the design of the Z-shaped waypoint sequence improves the maneuverability and obstacle avoidance capability in a small space, avoiding the risk of lateral collision caused by straight-line retreat.

[0075] This program has the following working process: Step 1: Synchronously collect the video stream data of the pedestrian's facial area and the 3D point cloud data output by the depth sensor; Step 2: extracting eyelid movement frequency features and mouth corner displacement gradient features from the video stream data, and calculating the deviation angle between the gaze direction and the head orientation based on the three-dimensional point cloud data; Step 3: Input the eyelid movement frequency characteristics, mouth corner displacement gradient characteristics and deviation angle into the pre-trained intention prediction model, and output the probability distribution matrix of the pedestrian's movement direction; Step 4: Obtain the ground reflection intensity attenuation curve and simultaneously collect the high-frequency vibration spectrum of the robot chassis; Step 5: Calculating the surface roughness index according to the reflection intensity attenuation curve, and extracting the energy proportion value of the 500-1500 Hz frequency band from the high-frequency vibration spectrum; Step 6: Query the material feature database based on the surface roughness index and energy ratio value to determine the dynamic friction coefficient μ of the current ground surface; Step 7: Generate a pre-obstacle avoidance path according to the probability distribution matrix, and adjust the PID parameters of the path tracking controller based on the dynamic friction coefficient μ; Step 8: When it is detected that the pedestrian turning probability is greater than the preset threshold and the dynamic friction coefficient μ is lower than the safe value, the graded braking strategy is triggered to control the robot to decelerate.

[0076] In summary, by collecting facial video stream data from pedestrians, extracting eyelid movement frequency features and mouth corner displacement gradient features, and combining them with 3D point cloud data to calculate the deviation angle between gaze direction and head orientation, a multi-feature fusion intention prediction model was constructed. This model can output a probability distribution matrix of the pedestrian's movement direction in advance by collaboratively analyzing micro-expressions and posture deviations, even when the pedestrian has not significantly moved. This mechanism breaks through the limitations of traditional detection that relies solely on physical distance, establishes a correlation model between pedestrian micro-expressions and movement direction, and advances the reaction time of obstacle avoidance decisions to the stage when the pedestrian's turning intention begins, effectively eliminating the collision risk caused by delayed reaction in high-density pedestrian flow scenarios. By synchronously acquiring the ground reflection intensity attenuation curve and the chassis' high-frequency vibration spectrum, calculating the surface roughness index and the energy proportion in the 500-1500Hz frequency band, and combining it with a material feature database to accurately determine the dynamic friction coefficient, this system achieves real-time identification of ground materials and quantification of friction characteristics. This addresses the problem of traditional solutions' insufficient perception of the friction coefficient of mixed-material surfaces. Through real-time feedback on the dynamic friction coefficient, braking risks on low-friction roads can be predicted in advance, avoiding skidding or instability caused by sudden material changes, significantly improving driving safety in complex road environments. By generating a pre-obstacle avoidance path based on the probability distribution matrix of the pedestrian's movement direction and adaptively adjusting the PID parameters of the path tracking controller according to the real-time dynamic friction coefficient, a graded braking strategy is triggered when a high turning probability and low friction coefficient of the pedestrian are detected, opening up a full closed loop of "pedestrian intention prediction - ground characteristic perception - control parameter adjustment", solving the problems of static control parameters and disconnection between path planning and environmental dynamics in traditional systems, and deeply integrating pedestrian intention data with ground friction characteristics to match path tracking accuracy and braking response with the real-time environment, avoiding path overshoot on low-friction ground and achieving smooth deceleration in high-risk scenarios, greatly improving obstacle avoidance efficiency and passenger safety.

[0077] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance, characterized in that: The following steps are involved: S1, synchronously collects the video stream data of the pedestrian's facial area and the three-dimensional point cloud data output by the depth sensor; S2. extracting eyelid movement frequency features and mouth corner displacement gradient features from the video stream data, and calculating the deviation angle between the line of sight direction and the head orientation based on the three-dimensional point cloud data; S3, inputting the eyelid movement frequency characteristics, mouth corner displacement gradient characteristics and deviation angle into a pre-trained intention prediction model, and outputting a probability distribution matrix of the pedestrian's movement direction; S4. Obtain the ground reflection intensity attenuation curve and simultaneously collect the high-frequency vibration spectrum of the robot chassis; S5. Calculating a surface roughness index according to the reflection intensity attenuation curve, and extracting an energy proportion value in a frequency band of 500 to 1500 Hz from the high-frequency vibration spectrum; S6. Querying a material feature database based on the surface roughness index and the energy proportion value to determine the dynamic friction coefficient μ of the current ground surface; S7. Generate a pre-obstacle avoidance path according to the probability distribution matrix, and adjust the PID parameters of the path tracking controller based on the dynamic friction coefficient μ; S8. When it is detected that the pedestrian turning probability is greater than a preset threshold and the dynamic friction coefficient μ is lower than a safe value, the graded braking strategy is triggered to control the robot to decelerate.

2. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: The extraction of eyelid movement frequency features in step S2 includes: Perform adaptive histogram equalization on the eye area of ​​video stream data; The eyelid opening and closing state is detected through the first convolutional layer of the convolutional neural network, and an 8×8×16 feature map is output; The displacement speed of the pupil center coordinates within 5 consecutive frames was calculated, and when the speed exceeded 3 pixels / frame, it was marked as a rapid eye movement event.

3. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: The calculation of the mouth corner displacement gradient feature in step S2 includes: Track pixel changes in the nasolabial fold area through the second convolutional layer of the convolutional neural network; Establish a polar coordinate system with the nose tip as the origin and calculate the radial displacement of the mouth corner key points between each frame; When the displacement gradient of the left mouth corner is greater than 0.15 pixels / frame for 3 frames, a left turn intention marker is generated.

4. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: The construction of the intention prediction model in step S2 satisfies: If the eyelid flutter frequency is >3 Hz, the mouth corner displacement gradient is >0.15 pixels / frame, and the deviation angle is >15°, the turning probability in the corresponding direction is increased by 0.6; The probability values ​​of turning left, going straight, and turning right are normalized using the Softmax function.

5. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: The calculation of the surface roughness index in step S5 includes: Perform first-order differential processing on the reflection intensity attenuation curve and take the absolute value of the slope in the 0-100 nanosecond time window; When the absolute value of the slope is <0.2 / nanosecond, it is judged as a highly smooth material and assigned a roughness index of 0.2-0.

4.

6. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: The processing of the high-frequency vibration spectrum in step S5 includes: Perform fast Fourier transform on the vibration signal and calculate the ratio of the energy in the 500-1500 Hz frequency band to the total energy; When the ratio is between 60% and 65% and the surface roughness index is 0.2-0.4, it is judged to be terrazzo floor.

7. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: The adjustment rules of the PID parameters in step S7 are: If μ<0.3, set Kp=120, Ki=0, Kd=30; If μ ≥ 0.3, set Kp = 80, Ki = 10, and Kd = 40.

8. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: The hierarchical braking strategy of step S8 includes: Phase 1: When the steering probability is greater than 0.6 and μ is less than 0.4, the motor torque is linearly reduced to 50% of the rated value; Phase 2: When the distance between the robot and the pedestrian is less than 1 meter, the magnetorheological brake is activated to apply reverse damping force.

9. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 1, characterized in that: It also includes an exception handling mechanism: when the intent prediction model output confidence is less than 50% and contradictory features are detected in three consecutive frames, a stop command is sent to the motion controller and the 360° circular warning light strip is activated.

10. The method for autonomous driving of an intelligent manned mobile robot with efficient obstacle avoidance according to claim 9, characterized in that: The generation of the stop instruction includes: Get the historical path data within the last 0.5 seconds; Calculate the optimal retreat vector based on historical path data; Generate a temporary path with 5 waypoints in a zigzag trajectory, with the distance between waypoints ≤ 30 cm.

Images