An autonomous driving method of an intelligent manned mobile robot with high obstacle avoidance efficiency

Abstract

The application discloses a kind of high-efficiency obstacle avoidance intelligent manned mobile robot autonomous driving method, belong to intelligent robot navigation control technical field, for the existing technology in the presence of pedestrian intention prediction ability loss, ground friction characteristic is not enough and the problem of control parameter dynamic adjustment mechanism loss, including S1, the video stream data of pedestrian face area and the three-dimensional point cloud data of depth sensor output are synchronously collected;S2, extract eyelid movement frequency feature and mouth corner displacement gradient feature from the video stream data;The application extracts eyelid movement frequency feature, mouth corner displacement gradient feature by collecting pedestrian face video stream data, and combines three-dimensional point cloud data to calculate the deviation angle of line of sight direction and head orientation, constructs the intention prediction model of multi-feature fusion, can in the probability distribution matrix of pedestrian moving direction is output in advance by micro-expression and the collaborative analysis of attitude deviation when pedestrian does not occur obvious displacement.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent robot navigation control, and particularly relates to an efficient obstacle avoidance intelligent manned mobile robot autonomous driving method. BACKGROUND

[0002] With the rapid development of intelligent technology and robot technology, intelligent manned mobile robots are increasingly widely used in medical transfer, old-age walking assistance, commercial services and other fields. The core functional requirement is to achieve safe and efficient autonomous obstacle avoidance in complex dynamic environments. How to ensure that the robot avoids obstacles during autonomous driving and ensures driving safety has become a problem to be solved.

[0003] However, in high-density pedestrian flow environments such as hospital corridors and nursing home activity areas, the existing technology has the following significant defects:

[0004] 1. Lack of pedestrian intention prediction ability: traditional obstacle avoidance technology mainly relies on laser radar, ultrasonic sensor and other distance detection devices, combined with visual SLAM technology for obstacle positioning and path planning. This scheme is based only on physical distance detection and cannot predict the movement intention of pedestrians when they have not moved significantly. When a pedestrian suddenly turns, the robot can only start obstacle avoidance after detecting the position change, resulting in a delayed response. More importantly, the existing system fails to establish a correlation model between pedestrian micro-expression features such as eyelid movement and mouth corner changes and movement direction, and cannot capture early biological feature signals of the turning intention.

[0005] 2. Lack of ground friction characteristic perception: In a mixed ground environment with different materials such as marble and non-slip mats, the influence of different materials on the braking performance of the robot is significantly different. The traditional scheme lacks real-time perception ability of the ground friction coefficient, and the braking parameters are fixed and unchanged, which cannot adapt to the material mutation scene, resulting in unstable braking process or even side slipping.

[0006] 3. Lack of dynamic adjustment mechanism for control parameters: The existing system has a control closed loop breakage problem. The path planning module does not fuse pedestrian intention prediction data, and the motion controller (such as PID parameters) is not associated with real-time adjustment of the ground friction coefficient. For example, a high integral coefficient is still used on low-friction ground, which can cause path tracking overshoot. This static control strategy in a mixed material ground scene can easily cause low obstacle avoidance efficiency and safety hazards.

[0007] Therefore, an efficient obstacle avoidance intelligent manned mobile robot autonomous driving method is needed to solve the problems of lack of pedestrian intention prediction ability, lack of ground friction characteristic perception and lack of dynamic adjustment mechanism for control parameters in the existing technology. SUMMARY

[0008] The present application aims to provide an intelligent manned mobile robot autonomous driving method with high obstacle avoidance efficiency to solve the problems in the background art.

[0009] To achieve the above object, the present application provides the following technical scheme: an intelligent manned mobile robot autonomous driving method with high obstacle avoidance efficiency, comprising the following steps:

[0010] S1, synchronously collecting video stream data of a pedestrian face region and three-dimensional point cloud data output by a depth sensor;

[0011] S2, extracting eyelid movement frequency features and mouth corner displacement gradient features from the video stream data, and calculating a deviation angle between a line of sight direction and a head orientation based on the three-dimensional point cloud data;

[0012] S3, inputting the eyelid movement frequency features, mouth corner displacement gradient features and deviation angle into a pre-trained intention prediction model to output a probability distribution matrix of a pedestrian movement direction;

[0013] S4, acquiring a ground reflection intensity decay curve, and synchronously collecting a high-frequency vibration spectrum of a robot chassis;

[0014] S5, calculating a surface roughness index according to the reflection intensity decay curve, and extracting an energy proportion value of a 500-1500 Hz frequency band from the high-frequency vibration spectrum;

[0015] S6, querying a material feature database based on the surface roughness index and energy proportion value to determine a dynamic friction coefficient μ of a current ground;

[0016] S7, generating a pre-obstacle avoidance path according to the probability distribution matrix, and adjusting PID parameters of a path tracking controller based on the dynamic friction coefficient μ;

[0017] S8, when a pedestrian turning probability is greater than a preset threshold value and the dynamic friction coefficient μ is lower than a safety value, triggering a hierarchical braking strategy to control the robot to slow down.

[0018] It should be noted in the scheme that the extraction of the eyelid movement frequency features in the S2 step comprises:

[0019] Adaptive histogram equalization processing is performed on the eye region of the video stream data;

[0020] The first convolutional layer of a convolutional neural network is used to detect the eyelid opening and closing state, and an 8x8x16 feature map is output;

[0021] The offset speed of the pupil center coordinates in consecutive 5 frames is calculated, and when the speed exceeds 3 pixels / frame, it is marked as a rapid eye movement event.

[0022] It is further worth mentioning that the calculation of the mouth corner displacement gradient feature in the S2 step includes:

[0023] Tracking the pixel changes in the nasolabial fold area through the second convolutional layer of the convolutional neural network;

[0024] Establishing a polar coordinate system with the tip of the nose as the origin and calculating the radial displacement of the mouth corner key point between frames;

[0025] When the left mouth corner displacement gradient is greater than 0.15 pixels / frame for 3 consecutive frames, a left turn intention label is generated.

[0026] It is further worth mentioning that the construction of the intention prediction model in the S2 step satisfies:

[0027] If the eyelid flutter frequency is > 3Hz, the mouth corner displacement gradient is > 0.15 pixels / frame, and the deviation angle is > 15°, the turning probability in the corresponding direction increases by 0.6;

[0028] The left turn, straight, and right turn probability values are normalized by the Softmax function.

[0029] As a preferred embodiment, the calculation of the surface roughness index in the S5 step includes:

[0030] First-order differential processing is performed on the reflection intensity decay curve, and the absolute value of the slope within the 0-100 nanosecond time window is taken;

[0031] When the absolute value of the slope is < 0.2 / nanosecond, it is determined to be a high-smooth material and the roughness index is assigned to 0.2-0.4.

[0032] As a preferred embodiment, the processing of the high-frequency vibration spectrum in the S5 step includes:

[0033] Fast Fourier transform is performed on the vibration signal, and the ratio of the energy in the 500-1500Hz band to the total energy is calculated;

[0034] When the ratio is between 60% and 65% and the surface roughness index is 0.2-0.4, it is determined to be a terrazzo floor.

[0035] As a preferred embodiment, the adjustment rule of the PID parameters in the S7 step is:

[0036] If μ < 0.3, set Kp = 120, Ki = 0, and Kd = 30;

[0037] If μ ≥ 0.3, set Kp = 80, Ki = 10, and Kd = 40.

[0038] As a preferred embodiment, the hierarchical braking strategy of the S8 step includes:

[0039] First stage: when the steering probability > 0.6 and μ < 0.4, control the motor torque to linearly decrease to 50% of the rated value;

[0040] Second stage: when the robot is less than 1 meter away from the pedestrian, activate the magneto-rheological brake to apply a reverse damping force.

[0041] As a preferred embodiment, an abnormal handling mechanism is also included: when the intention prediction model output confidence is < 50% and contradictory features are detected for 3 consecutive frames, send a stop instruction to the motion controller and activate the 360° ring warning light belt.

[0042] As a preferred embodiment, the generation of the stop instruction includes:

[0043] Obtain historical path data in the last 0.5 seconds;

[0044] Calculate the optimal retreat vector based on the historical path data;

[0045] Generate a temporary path with 5 waypoints in a zigzag trajectory, with a waypoint spacing of ≤ 30 cm.

[0046] Compared with the prior art, the intelligent manned mobile robot autonomous driving method provided by the application at least has the following beneficial effects:

[0047] (1) By collecting pedestrian face video stream data, extracting eyelid movement frequency features and mouth corner displacement gradient features, and combining three-dimensional point cloud data to calculate the deviation angle of the line of sight direction and the head orientation, a multi-feature fusion intention prediction model is constructed. The probability distribution matrix of the pedestrian movement direction can be output in advance through the cooperative analysis of micro-expression and posture deviation when the pedestrian has not moved significantly. This mechanism breaks through the limitations of traditional physical distance detection, establishes a correlation model between pedestrian micro-expression and movement direction, and advances the reaction time of the obstacle avoidance decision to the pedestrian turning intention generation stage, effectively solving the collision risk caused by reaction lag in high-density crowd scenes.

[0048] (2) By synchronously acquiring the ground reflection intensity decay curve and the chassis high-frequency vibration spectrum, calculating the surface roughness index and the energy proportion in the 500-1500Hz frequency band, and combining the material feature database to accurately determine the dynamic friction coefficient, real-time identification and friction characteristic quantization of the ground material are realized, solving the problem of insufficient perception of the friction coefficient of mixed ground materials in traditional schemes. Through real-time feedback of the dynamic friction coefficient, the braking risk of low-friction road surfaces can be predicted in advance, avoiding braking side slip or instability caused by sudden changes in material, and significantly improving the driving safety in complex ground environments.

[0049] (3) By generating a pre-obstacle avoidance path based on the pedestrian moving direction probability distribution matrix, and adaptively adjusting the PID parameters of the path tracking controller according to the real-time dynamic friction coefficient, when the high steering probability and low friction coefficient of the pedestrian are detected, the hierarchical braking strategy is triggered, which realizes the full closed loop of "pedestrian intention prediction-ground feature perception-control parameter adjustment", solves the problems of static control parameters of traditional system, disconnection between path planning and environment dynamics, and deeply integrates the pedestrian intention data and ground friction characteristics, so that the path tracking accuracy and braking response are matched with the real-time environment, the path overshoot is avoided on the low friction ground, and smooth deceleration is realized in the high-risk scene, which greatly improves the obstacle avoidance efficiency and the safety of carrying people. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The step flowchart provided by the present application is provided.

[0051] Figure 2 The flowchart of the hierarchical braking strategy provided by the present application is provided. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0053] The present application will be further described below with reference to the embodiments.

[0054] Please refer to Figures 1-2 The present application provides an intelligent carrying mobile robot autonomous driving method for efficient obstacle avoidance, comprising the following steps:

[0055] S1, synchronously collecting video stream data of a pedestrian face region and three-dimensional point cloud data output by a depth sensor;

[0056] S2, extracting eyelid motion frequency features and mouth corner displacement gradient features from the video stream data, and calculating a deviation angle between a line of sight direction and a head orientation based on the three-dimensional point cloud data;

[0057] S3, inputting the eyelid motion frequency features, the mouth corner displacement gradient features and the deviation angle into a pre-trained intention prediction model to output a probability distribution matrix of a pedestrian moving direction;

[0058] S4, obtaining a ground reflection intensity decay curve, and synchronously collecting a high-frequency vibration spectrum of a robot chassis;

[0059] S5, calculating the surface roughness index according to the reflection intensity attenuation curve, and extracting the energy proportion value of the 500-1500 Hz frequency band from the high-frequency vibration spectrum;

[0060] S6, querying the material characteristic database based on the surface roughness index and the energy proportion value to determine the dynamic friction coefficient μ of the current ground;

[0061] S7, generating a pre-obstacle avoidance path according to the probability distribution matrix, and adjusting the PID parameters of the path tracking controller based on the dynamic friction coefficient μ;

[0062] S8, when it is detected that the pedestrian turning probability is greater than a preset threshold and the dynamic friction coefficient μ is lower than a safety value, triggering a hierarchical braking strategy to control the robot to slow down.

[0063] Further as Figure 1 illustrated, it is worth noting that the extraction of the eyelid movement frequency feature in the S2 step includes:

[0064] Adaptive histogram equalization processing is performed on the eye region of the video stream data;

[0065] The first convolutional layer of the convolutional neural network detects the eyelid opening and closing state, and outputs a feature map of 8x8x16;

[0066] The offset speed of the pupil center coordinates in the continuous 5 frames is calculated, and when the speed exceeds 3 pixels / frame, it is marked as a rapid eye movement event.

[0067] The adaptive histogram equalization processing refers to improving the image quality of the eye region through local contrast enhancement technology, which can be realized by using a block histogram equalization algorithm, and is used to eliminate the interference of uneven illumination on the eyelid state detection.

[0068] The first convolutional layer of the convolutional neural network detects the eyelid opening and closing state, which refers to extracting the spatial features of the eye region through the trained convolution kernel, and specifically can use a 3x3 convolution kernel for edge detection, which is used to identify the geometric changes of the eyelid contour.

[0069] The rapid eye movement event marking refers to establishing a motion trajectory analysis model based on the dynamic changes of the pupil center coordinates, which can specifically use the optical flow method to calculate the displacement vector between the continuous frames, which is used to capture the instantaneous behavior characteristics of the pedestrian attention shift.

[0070] Specifically, after the eye region is processed by histogram equalization, the contrast of the eyelid edge in the image is significantly enhanced, so that the subsequent convolution layer can more accurately capture the morphological changes in the opening and closing process of the eyelid; the feature map output by the first convolution layer retains the time sequence characteristics of the eyelid movement through spatial dimension reduction, and combined with the displacement speed analysis between consecutive frames, the normal blinking and the intentional saccade can be effectively distinguished; when the saccade exceeding the threshold is detected, the feature is associated with the possible direction change intention of the pedestrian.

[0071] Compared with the prior art, the traditional method usually detects the eye closure state only through a single frame image, without considering the motion speed characteristics in the dynamic sequence; the present scheme can accurately identify the saccade event under complex lighting conditions by fusing image enhancement, multi-frame motion analysis and neural network feature extraction, and provide high-confidence biological feature data for subsequent pedestrian intention prediction.

[0072] Through the above technical solutions, the present application solves the prediction delay problem caused by insufficient detection accuracy of eye micro-movement in the prior art, can capture the subtle signs of attention shift of pedestrians in advance, and reduce the risk of emergency stop of robots due to misjudgment of the motion direction of pedestrians.

[0073] Further as shown in Figure 1 It is worth noting that the calculation of the mouth corner displacement gradient feature in the S2 step includes:

[0074] The second convolution layer of the convolutional neural network tracks the pixel changes in the nasolabial fold region;

[0075] The polar coordinate system is established with the tip of the nose as the origin, and the radial displacement amount of the mouth corner key point between each frame is calculated;

[0076] When the left mouth corner displacement gradient is greater than 0.15 pixels / frame for 3 consecutive frames, a left turn intention label is generated.

[0077] The pixel change tracking of the nasolabial fold region refers to monitoring the dynamic changes of the skin texture around the mouth, which can specifically use a convolution kernel with 32 filters for feature extraction to capture subtle facial muscle movements;

[0078] The establishment of the polar coordinate system refers to setting the angle and distance parameters with the tip of the nose as the origin, which can specifically determine the position of the mouth corner through the key point detection algorithm, and convert the two-dimensional plane coordinates into polar coordinate parameters;

[0079] The radial displacement amount calculation refers to analyzing the position difference of the mouth corner key point between consecutive frames, which can specifically use the optical flow method or the difference algorithm to realize, for quantifying the amplitude of mouth movement;

[0080] The left-turn intention mark generation refers to judging the motion trend according to a preset threshold, for example, triggering a logical judgment when the displacement amount in a specific direction is detected to continuously reach the threshold, for representing the potential turning intention of the pedestrian.

[0081] Specifically, the facial subtle motion features are extracted by the convolution operation on the nasolabial sulcus area, and the error caused by the head position change is eliminated by mapping the lip corner motion trajectory into the polar coordinate system; the inter-frame calculation of the radial displacement amount can effectively distinguish the muscle movements related to the natural expression and the turning intention, and when the left lip corner displacement gradient continuously exceeds the set threshold, it indicates that the pedestrian may be preparing to turn left; the judgment mechanism can predict the change of the motion direction of the pedestrian in advance, and provide an earlier warning signal for the robot obstacle avoidance decision.

[0082] Compared with the prior art, the traditional method only relies on the head orientation or body posture for direction prediction, and cannot capture the turning intention conveyed by micro-expression; by analyzing the lip corner displacement gradient, a biomechanical feature, and combining the continuity judgment on the time sequence, the turning trend of the pedestrian can be identified earlier, and the lag defect of the traditional visual detection method is effectively made up.

[0083] Through the above technical solutions, the pedestrian turning intention can be identified early, so that the robot can adjust the obstacle avoidance path before the pedestrian actually changes the motion direction, and the safety of human-robot cooperation in high-density crowd environment is significantly improved.

[0084] Further as shown in Figure 1 It is worth noting that the construction of the intention prediction model in the S2 step satisfies:

[0085] If the eyelid tremor frequency is >3Hz, the lip corner displacement gradient is >0.15 pixels / frame, and the deviation angle is >15°, the turning probability in the corresponding direction is increased by 0.6;

[0086] The left-turn, straight, and right-turn probability values are normalized by the Softmax function.

[0087] The eyelid tremor frequency refers to the number of times of opening and closing of the eyelid per unit time, which can be realized by calculating the area change rate of the pupil region in the continuous video frames, and is used to reflect the attention shift state of the pedestrian.

[0088] The lip corner displacement gradient refers to the moving rate of the lip corner position during the facial expression change, which can be realized by tracking the position offset of the nasolabial sulcus key point by using the optical flow method, and is used to represent the potential turning intention of the pedestrian.

[0089] The deviation angle refers to the spatial included angle between the line of sight direction and the actual orientation of the head, which can be calculated by the relative position relationship between the eyeball center and the head centroid in the three-dimensional point cloud coordinate system, and is used to identify the attention shift direction of the pedestrian.

[0090] The Softmax function refers to a multi-classification probability normalization method, which can specifically convert the original prediction value into a distribution form that is mutually exclusive and the sum of probabilities is 1, and is used to generate an interpretable pedestrian moving direction prediction result.

[0091] Specifically, in the video stream processing process, when the eyelid tremor frequency is detected to be continuously more than 3Hz, it indicates that the pedestrian may be in a state of high concentration of attention; if the mouth corner displacement gradient is continuously more than 0.15 pixels per frame and the spatial deviation angle of the head and the line of sight reaches more than 15 degrees at this time, it is determined that the pedestrian has a clear tendency of changing the moving direction. The condition triggering mechanism increases the turning probability of the corresponding direction by 0.6 through weighted operation, forming an initial probability matrix including three dimensions of left turn, straight, and right turn. Further, through the exponential operation and normalization processing of the Softmax function, a final decision matrix conforming to the probability distribution characteristics is generated, providing a quantitative basis for subsequent path planning.

[0092] Compared with the prior art, the traditional pedestrian intention prediction method usually only relies on single modal data, such as analyzing the head posture or limb action alone, while ignoring the synergistic effect of eye micro-expression and facial expression. The existing scheme adopts a linear weighting method in probability calculation, which is difficult to handle the non-linear relationship between multiple features, resulting in insufficient confidence of the prediction result.

[0093] Through the above technical solutions, the present application realizes the cooperative detection and probability fusion of multi-dimensional biological features, effectively improves the recognition accuracy of the pedestrian turning intention; by setting multiple dynamic threshold conditions, the key turning points in the pedestrian behavior pattern can be accurately captured, and combined with the non-linear probability normalization method, the robustness of the prediction model in complex scenes is significantly enhanced, providing more reliable prediction basis for the robot.

[0094] Further as shown in Figure 1 It is worth noting that the calculation of the surface roughness index in the S5 step includes:

[0095] First-order differential processing is performed on the reflection intensity decay curve, and the absolute value of the slope in the 0-100 nanosecond time window is taken;

[0096] When the absolute value of the slope is <0.2 / nanosecond, it is determined to be a high smooth material and the roughness index is assigned to 0.2-0.4.

[0097] The first-order differential processing of the laser radar echo intensity curve refers to the differential operation on the curve of the reflection intensity changing with time, which can be specifically implemented by using a numerical differentiation algorithm, and is used to capture the difference in reflection intensity decay characteristics of different material surfaces;

[0098] The absolute value of the slope in the time window refers to selecting the maximum change rate of the differential curve in the first 100 nanosecond time interval after the laser pulse is emitted, which can be specifically realized by sliding window difference calculation. The time window covers the effective detection range of the laser radar on the ground material;

[0099] The determination condition of the high smooth material refers to taking the absolute value of the differential curve slope less than 0.2 per nanosecond as the classification threshold, which can be specifically realized by a supervised learning method based on a material sample library, and is used to distinguish low friction coefficient ground types such as ceramic tiles and marbles.

[0100] Specifically, the original data of the ground reflection intensity changing with time is collected by the laser radar, the first-order differential processing is performed on the echo signal, the environmental light interference is eliminated, and the material characteristics are enhanced. In the effective detection time window of 0-100 nanoseconds, the maximum absolute value of the slope of the differential curve is calculated, which can represent the difference in the reflection characteristics of the ground microstructure. When the absolute value of the slope is less than the set threshold, it indicates that the ground reflection intensity changes gently, which meets the physical characteristics of the high smooth material, and the corresponding low roughness index is assigned. The index is used as the basis for calculating the ground friction coefficient and is called by the subsequent path tracking control module.

[0101] Compared with the prior art, the traditional method only detects the distance of obstacles through the laser radar point cloud, without considering the influence of the ground material difference on the braking performance of the robot. The present scheme establishes an association model between the ground material and the friction coefficient by analyzing the differential characteristics of the laser echo characteristics, solving the problem of improper setting of the braking parameters caused by ignoring the ground material in the prior art.

[0102] Through the above technical scheme, the high smooth ground material can be accurately identified, providing a reliable basis for dynamically adjusting the braking parameters of the robot, avoiding the slipping phenomenon on low friction roads such as ceramic tiles and marbles, and improving the driving safety and control stability in complex ground environments.

[0103] Further as shown in Figure 1 It is worth noting that the processing of the high-frequency vibration spectrum in the S5 step includes:

[0104] Performing fast Fourier transform on the vibration signal to calculate the ratio of the energy in the 500-1500 Hz band to the total energy;

[0105] When the ratio is between 60% and 65% and the surface roughness index is between 0.2 and 0.4, it is determined as terrazzo ground.

[0106] The fast Fourier transform refers to a processing method for converting time domain vibration signals into frequency domain signals, which can be specifically realized by a digital signal processing chip, and is used to extract the energy distribution characteristics of different frequency components.

[0107] The ratio of the energy of the 500-1500 Hz frequency band to the total energy refers to the proportion of the signal energy in a specific frequency range to the entire spectrum energy, and the energy sum of each frequency band can be calculated by integral operation to represent the vibration response characteristics of the ground material to the robot chassis.

[0108] The surface roughness index refers to a quantitative parameter obtained by differential processing of the laser radar return intensity curve, and can be associated with the pre-stored data in the material characteristic database to cooperatively determine the ground friction characteristics.

[0109] Specifically, the time domain signal collected by the vibration sensor is processed by fast Fourier transform to generate a frequency spectrum containing energy of different frequency components; by setting the target frequency band of 500-1500 Hz, the proportion of the energy sum in this interval to the total energy sum of the entire spectrum is calculated; when the ratio reaches a specific interval of 60%-65%, the surface roughness index calculated from the laser radar data is combined to accurately identify the ground material type with similar vibration response characteristics by querying the pre-established material characteristic corresponding relationship; for example, when the energy ratio of this frequency band is 62% and the surface roughness index is 0.35, it is determined that the current ground is terrazzo material.

[0110] Compared with the prior art, the traditional method only relies on laser radar single modal data to determine the ground type, which is easy to cause misjudgment due to environmental light interference or surface reflection characteristics; and the present scheme fuses the vibration spectrum characteristics and laser radar data to construct a multi-modal feature fusion ground recognition mechanism, which can effectively distinguish ground materials with similar visual characteristics but large friction coefficient differences.

[0111] Through the above technical scheme, the vibration response characteristics of special material ground such as terrazzo can be accurately identified, and the surface roughness parameter is combined to realize dynamic calibration of the ground friction coefficient. This dual-mode detection mechanism can avoid the error of single sensor misjudgment, ensure that the robot can adjust the control parameters based on accurate ground characteristic data when driving on different material ground, and significantly improve the stability and safety in the emergency braking process.

[0112] Further as shown in Figure 1 It is worth noting that the adjustment rule of the PID parameters in the S7 step is:

[0113] If μ < 0.3, set Kp = 120, Ki = 0, and Kd = 30;

[0114] If μ ≥ 0.3, set Kp = 80, Ki = 10, and Kd = 40.

[0115] The dynamic friction coefficient refers to the sliding friction characteristic parameter between the ground material and the robot tire, and can be specifically realized by a method of laser radar echo intensity curve analysis and high-frequency vibration spectrum feature matching, and is used to represent the influence degree of different ground materials on the robot motion control.

[0116] The proportional coefficient Kp refers to the gain parameter in the path tracking controller that is linearly related to the path deviation, and can be specifically realized by dynamically adjusting the parameter to change the system response speed, and is used to quickly correct the lateral deviation between the robot motion direction and the planned path.

[0117] The integral coefficient Ki refers to the adjustment parameter in the controller for compensating the accumulated error, and can be specifically realized by an integral link for eliminating the steady-state error, and is used to continuously correct the long-term path tracking deviation caused by the difference in ground friction.

[0118] The differential coefficient Kd refers to the parameter for predictive adjustment based on the error change rate, and can be specifically realized by a differential operation for suppressing system overshoot, and is used to respond in advance to the robot attitude fluctuation caused by sudden changes in ground material.

[0119] Specifically, when the laser radar and the vibration sensor jointly detect that the ground dynamic friction coefficient is lower than 0.3, the path tracking controller automatically switches to a high-proportion low-integral parameter combination to compensate for the risk of slipping on a low-friction ground by enhancing the rapid response capability; when the friction coefficient rises to 0.3 and above, a parameter combination balancing the proportion and the integral is adopted to introduce error correction function while ensuring response speed; the parameter adjustment process is realized by online table lookup, and the friction coefficient value calculated in real time is dynamically matched with the preset optimized parameter group.

[0120] Compared with the prior art, the traditional method uses fixed PID parameters for motion control, without considering the change in friction coefficient caused by different ground materials, resulting in path tracking overshoot or brake lag on low-friction ground; the present scheme establishes a mapping relationship between the friction coefficient and the controller parameters, so that the robot can automatically optimize the control strategy according to the real-time ground state, solving the technical defect that a single parameter cannot adapt to a mixed ground of multiple materials.

[0121] Through the above technical scheme, the present application realizes adaptive motion control under different ground friction conditions, effectively reduces the path tracking deviation caused by sudden changes in ground material, improves the trajectory keeping ability of the robot on low-friction ground, and at the same time avoids excessive braking on high-friction ground, ensuring the driving stability and safety during manned movement.

[0122] Further as shown in Figure 2 It is worth noting that the step S8 of the hierarchical braking strategy includes:

[0123] First stage: when the steering probability > 0.6 and μ < 0.4, the motor torque is linearly reduced to 50% of the rated value;

[0124] Second stage: when the robot is less than 1 meter away from the pedestrian, the magneto-rheological brake is activated to apply a reverse damping force.

[0125] Among them, the staged braking strategy refers to a control method of implementing different braking intensities in stages, which can be realized by using a double-threshold triggering mechanism, and the trigger conditions of different stages are set to realize progressive adjustment of the braking torque;

[0126] The linear reduction of the motor torque refers to the proportional reduction of the output torque of the driving system, which can be realized by using a PID controller and a motor driver to work together to reduce the power output and achieve preliminary deceleration;

[0127] The magneto-rheological brake applies a reverse damping force, which refers to using the viscosity change characteristic of magneto-rheological fluid to generate a controllable resistance, which can be realized by adjusting the magnetic field strength of the electromagnetic coil to enhance the braking efficiency by adjusting the damping coefficient in real time.

[0128] Specifically, when the left-turn probability of the pedestrian is detected to exceed the set threshold and the ground friction coefficient is insufficient, the first braking stage is entered first, at this time the output torque of the driving motor is adjusted by adjusting the control signal to decrease at a predetermined slope, for example, the torque is reduced to half of the initial value within 0.5 seconds; when the robot continues to approach the pedestrian and the distance is shortened to a safety threshold, the second braking stage is automatically started, at this time the electromagnetic coil of the magneto-rheological brake adjusts the current intensity in real time according to the distance sensor feedback, generates a damping force field opposite to the direction of motion, and the size of the damping force is positively related to the current speed of the robot.

[0129] Compared with the prior art, the traditional braking scheme usually uses a single threshold to trigger an emergency stop operation, which is easy to cause the manned mobile robot to shake or slide violently; and the present scheme realizes stepwise attenuation of speed under the premise of maintaining stable posture through staged braking control, especially under low-friction ground conditions, combined with accurate regulation of the reverse damping force, effectively avoids lateral displacement caused by inertia.

[0130] Through the above technical scheme, the technical problem of easy sliding or excessive impact during emergency braking in the prior art is solved, safe and stable deceleration control can be realized under complex ground conditions, the robot can maintain stable operation when avoiding turning pedestrians in time, and the risk of equipment wear and passenger discomfort caused by emergency braking is reduced.

[0131] Further as Figure 2As shown, it is worth noting that the abnormal handling mechanism is also included: when the intention prediction model outputs a confidence of <50% and contradictory features are detected for 3 consecutive frames, a stop instruction is sent to the motion controller, and a 360° ring-shaped warning light belt is activated.

[0132] wherein the output confidence refers to the credibility of the model prediction result, which can be quantified by entropy value calculation of the probability distribution, and the higher the entropy value, the greater the uncertainty of the prediction result;

[0133] Contradictory features refer to logical conflicts between different sensor data, which can be realized by exceeding a pre-set threshold value of the Euclidean distance of the feature vector, for example, when the eyelid movement feature and the mouth corner displacement feature point in different directions;

[0134] The stop instruction refers to a control signal that immediately terminates the current motion instruction, which can be realized by sending an emergency stop message through the CAN bus. The 360° ring-shaped warning light belt refers to a visual warning device arranged circumferentially around the robot, which can be realized by an array of RGB LED light beads, for example, by PWM dimming control to produce a red breathing effect.

[0135] Specifically, when the pedestrian intention prediction confidence is insufficient and contradictory features are detected, the system will enter a safety protection mode; when the output probability distribution entropy value of the prediction model exceeds a pre-set threshold value, combined with the feature difference analysis of consecutive multiple frames of sensor data, it is determined that the current environment has unreliable detection conditions, at which time a stop command is sent through the emergency communication interface of the motion controller, and the ring-shaped light belt is driven into a warning state through the light control module, for example, in the corridor scene of a nursing home, when the pedestrian facial expression feature and the body posture feature are detected to point in opposite directions, the system actively stops advancing to avoid the risk of misjudgment.

[0136] Compared with the prior art, the traditional obstacle avoidance system still continues to execute the original path planning when the sensor data is abnormal, while the present scheme actively triggers safety protection by establishing a double detection mechanism; the prior art lacks an evaluation system for the confidence of the prediction model, which cannot identify contradictory sensor data in complex environments, and is prone to misjudgment; the present scheme introduces a combination of confidence threshold and contradictory feature detection conditions, effectively improving the safety of the system in uncertain environments.

[0137] Through the above technical scheme, the present application can stop moving in time when the pedestrian intention prediction is unreliable or the sensor data is abnormal, avoiding the risk of collision due to false judgment; through the immediate feedback of the ring-shaped warning light, the surrounding personnel can be reminded to pay attention to the state of the robot, so as to gain response time for the system to recover to a safe state; this mechanism is particularly suitable for scenes such as hospital corridors where there are complex reflective ground and dense personnel flow, ensuring that the manned robot can still operate safely in uncertain environments.

[0138] Further as Figure 2 illustrated, it is worth noting that the generation of the stop instruction includes:

[0139] acquiring historical path data in the last 0.5 seconds;

[0140] calculating an optimal retreat vector based on the historical path data;

[0141] The specific calculation formula is as follows:

[0142] ;

[0143] In the formula, represents the optimal retreat vector, and represents the time step, represents the position coordinates of the time step , represents a preset minimum safety distance, represents the distance between the current robot and the nearest obstacle, represents a basic retreat amplitude, represents a hazard sensitivity coefficient;

[0144] generating a temporary path of 5 waypoints in a zigzag trajectory, with a waypoint spacing ≤ 30 cm.

[0145] The optimal retreat vector refers to the best evacuation direction based on the spatiotemporal continuity analysis of the robot motion trajectory. Specifically, a sliding window method can be used to calculate the weighted average of the historical path to achieve this. The obstacle distribution density and robot kinematic constraints are included in the calculation dimension during the generation of this vector.

[0146] The zigzag trajectory refers to an obstacle avoidance path composed of consecutive polyline segments. An improved artificial potential field method can be used to generate a waypoint sequence to achieve this. The spacing between adjacent waypoints is constrained by a path smoothness optimization algorithm.

[0147] Specifically, when detecting a decrease in pedestrian behavior prediction confidence and feature contradictions, the system first extracts the time series data of the historical path, predicts the motion trend in the next 0.2 seconds using the Kalman filter algorithm, combines the real-time distribution of environmental obstacles, calculates the azimuth angle and velocity component of the retreat vector through the quadratic programming algorithm, and generates a temporary path containing turning points using the dynamic window method according to the retreat vector parameters. The position error of each waypoint is controlled within the accuracy range of the positioning system, ensuring that the robot can complete the avoidance action in a small space.

[0148] Compared with the prior art, the traditional abnormal handling scheme mostly adopts emergency braking or straight-line reversing strategy, which is easy to cause secondary collision risk; the scheme generates a retreat path dynamically through historical path data analysis, and combines with Z-shaped trajectory design, so that the sudden obstacle can be effectively avoided under the premise of keeping the fuselage stable; the generation method of the fixed retreat path in the prior art cannot adapt to dynamic environmental changes, and the waypoint sequence generation mechanism of the application can adjust the trajectory curvature in real time according to the ground friction coefficient.

[0149] Through the above technical scheme, the application effectively solves the problem of rigid path planning in the emergency braking process, and can realize safe evacuation in a complex pedestrian flow environment; through the calculation of the dynamic retreat vector, the optimal avoidance direction of the robot in the sudden situation is ensured; the design of the Z-shaped waypoint sequence improves the maneuvering obstacle avoidance ability in the narrow space, and avoids the lateral collision risk caused by straight-line retreat.

[0150] The scheme has the following working process:

[0151] Step one: synchronously collecting video stream data of a pedestrian face region and three-dimensional point cloud data output by a depth sensor;

[0152] Step two: extracting eyelid motion frequency features and mouth corner displacement gradient features from the video stream data, and calculating a deviation angle between a line of sight direction and a head orientation based on the three-dimensional point cloud data;

[0153] Step three: inputting the eyelid motion frequency features, the mouth corner displacement gradient features and the deviation angle into a pre-trained intention prediction model to output a probability distribution matrix of a pedestrian moving direction;

[0154] Step four: acquiring a ground reflection intensity decay curve, and synchronously collecting a high-frequency vibration spectrum of a robot chassis;

[0155] Step five: calculating a surface roughness index according to the reflection intensity decay curve, and extracting an energy proportion value of a 500-1500Hz frequency band from the high-frequency vibration spectrum;

[0156] Step six: querying a material feature database based on the surface roughness index and the energy proportion value to determine a dynamic friction coefficient μ of the current ground;

[0157] Step seven: generating a pre-obstacle avoidance path according to the probability distribution matrix, and adjusting PID parameters of a path tracking controller based on the dynamic friction coefficient μ;

[0158] Step eight: when it is detected that a pedestrian turning probability is greater than a preset threshold and the dynamic friction coefficient μ is lower than a safety value, triggering a hierarchical braking strategy to control the robot to slow down.

[0159] In summary: by collecting pedestrian face video stream data, extracting eyelid movement frequency features, mouth corner displacement gradient features, and combining three-dimensional point cloud data to calculate the deviation angle of the line of sight direction and the head orientation, a multi-feature fusion intention prediction model is constructed, which can output the probability distribution matrix of the pedestrian moving direction in advance through the cooperative analysis of micro-expression and posture deviation when the pedestrian has not moved obviously. This mechanism breaks through the limitations of traditional physical distance detection, establishes the association model between pedestrian micro-expression and movement direction, and advances the reaction time of obstacle avoidance decision to the pedestrian turning intention generation stage, effectively solving the collision risk caused by reaction lag in high-density crowd scenes.

[0160] By synchronously acquiring the ground reflection intensity decay curve and the chassis high-frequency vibration spectrum, calculating the surface roughness index and the energy ratio in the 500-1500Hz frequency band, and combining the material feature database to accurately determine the dynamic friction coefficient, real-time recognition and friction characteristic quantization of the ground material are realized, solving the problem of insufficient friction coefficient perception of traditional schemes for mixed ground of multiple materials. Through real-time feedback of the dynamic friction coefficient, the braking risk of low-friction road surface can be predicted in advance to avoid braking side slip or unstable phenomenon caused by sudden change of material, significantly improving the driving safety in complex ground environment.

[0161] By generating a pre-obstacle avoidance path based on the pedestrian moving direction probability distribution matrix, and adaptively adjusting the PID parameters of the path tracking controller according to the real-time dynamic friction coefficient, when a high turning probability of pedestrians and a low friction coefficient are detected, a hierarchical braking strategy is triggered, which opens a full closed loop of "pedestrian intention prediction-ground characteristic perception-control parameter adjustment", solving the problem of static control parameters, disconnection between path planning and dynamic environment in traditional systems. By deeply integrating pedestrian intention data and ground friction characteristics, the path tracking accuracy and braking response are matched with the real-time environment, avoiding path overshoot on low-friction ground and achieving smooth deceleration in high-risk scenarios, greatly improving the obstacle avoidance efficiency and passenger safety.

[0162] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A high-efficiency obstacle-avoiding intelligent manned mobile robot autonomous driving method, characterized in that, The method comprises the following steps: S1, synchronously collecting video stream data of a pedestrian face region and three-dimensional point cloud data output by a depth sensor; S2, extracting eyelid movement frequency features and mouth corner displacement gradient features from the video stream data, and calculating a deviation angle between a line of sight direction and a head orientation based on the three-dimensional point cloud data; S3, inputting the eyelid movement frequency features, mouth corner displacement gradient features and deviation angle into a pre-trained intention prediction model to output a probability distribution matrix of a pedestrian moving direction; S4, acquiring a ground reflection intensity decay curve, and synchronously collecting a high-frequency vibration spectrum of a robot chassis; S5, calculating a surface roughness index according to the reflection intensity decay curve, and extracting an energy proportion value of a 500-1500 Hz frequency band from the high-frequency vibration spectrum; S6, querying a material feature database based on the surface roughness index and the energy proportion value to determine a dynamic friction coefficient μ of a current ground; S7, generating a pre-obstacle avoidance path according to the probability distribution matrix, and adjusting PID parameters of a path tracking controller based on the dynamic friction coefficient μ; S8, when it is detected that a pedestrian turning probability is greater than a preset threshold value and the dynamic friction coefficient μ is lower than a safety value, triggering a hierarchical braking strategy to control the robot to slow down. 2.The autonomous driving method of the intelligent manned mobile robot with high obstacle avoidance efficiency according to claim 1, wherein: The extraction of the eyelid movement frequency features in the S2 step comprises: performing adaptive histogram equalization processing on an eye region of the video stream data; detecting eyelid opening and closing states through a first convolutional layer of a convolutional neural network to output a 8×8×16 feature map; calculating a displacement speed of pupil center coordinates in five continuous frames, and marking a rapid eye movement event when the speed exceeds 3 pixels / frame. 3.The autonomous driving method of the intelligent manned mobile robot with high obstacle avoidance efficiency according to claim 1, characterized in that: The calculation of the mouth corner displacement gradient features in the S2 step comprises: tracking pixel changes in a nasolabial sulcus region through a second convolutional layer of the convolutional neural network; establishing a polar coordinate system with a tip of a nose as an origin, and calculating a radial displacement amount of a mouth corner key point between frames; generating a left turning intention mark when a left mouth corner displacement gradient is greater than 0.15 pixels / frame for three frames. 4.The autonomous driving method of the intelligent manned mobile robot with high obstacle avoidance efficiency according to claim 1, wherein: The construction of the intention prediction model in the S2 step satisfies: if an eyelid fluttering frequency > 3 Hz, a mouth corner displacement gradient > 0.15 pixels / frame and a deviation angle > 15°, a turning probability of a corresponding direction is increased by 0.6; performing normalization processing on left turning, straight going and right turning probability values through a Softmax function.

5. The autonomous traveling method of the intelligent manned mobile robot with high obstacle avoidance efficiency according to claim 1, characterized in that: The calculation of the surface roughness index in the S5 step comprises: performing first-order differential processing on the reflection intensity decay curve, and taking an absolute value of a slope in a 0-100 nanosecond time window; when the absolute value of the slope < 0.2 / nanosecond, determining that the material is high-smooth and assigning a roughness index of 0.2-0.

4. 6.The autonomous traveling method of a high-efficiency obstacle-avoiding intelligent manned mobile robot according to claim 1, characterized in that: The processing of the high-frequency vibration spectrum in the S5 step comprises: performing fast Fourier transform on the vibration signal to calculate a ratio of energy in a 500-1500 Hz frequency band to total energy; when the ratio is between 60% and 65% and the surface roughness index is 0.2-0.4, determining that the ground is terrazzo.

7. The autonomous traveling method of the intelligent manned mobile robot with high obstacle avoidance efficiency according to claim 1, characterized in that: The adjustment rule of the PID parameters in the S7 step is: if μ < 0.3, setting Kp = 120, Ki = 0 and Kd = 30; if μ ≥ 0.3, setting Kp = 80, Ki = 10 and Kd = 40. 8.The autonomous traveling method of a high-efficiency obstacle-avoiding intelligent manned mobile robot according to claim 1, characterized in that: The hierarchical braking strategy of the S8 step includes: First stage: when the steering probability > 0.6 and μ < 0.4, control the motor torque to linearly decrease to 50% of the rated value; Second stage: when the distance between the robot and the pedestrian < 1 meter, activate the magnetorheological brake to apply a reverse damping force. 9.The autonomous traveling method of a high-efficiency obstacle-avoiding intelligent manned mobile robot according to claim 1, characterized in that: An abnormal handling mechanism is also included: when the intention prediction model output confidence < 50% and contradictory features are detected for 3 consecutive frames, send a stop instruction to the motion controller and activate the 360° ring warning light belt. 10.The autonomous traveling method of the intelligent manned mobile robot with high obstacle avoidance efficiency according to claim 9, characterized in that: The generation of the stop instruction includes: Obtain historical path data within the last 0.5 seconds; Calculate the optimal retreat vector based on the historical path data; Generate a temporary path with 5 waypoints in a zigzag trajectory, with a waypoint spacing ≤ 30 cm.

Images