Intelligent blind guiding helmet system

Abstract

The invention provides an intelligent blind guiding helmet system. A helmet comprises a camera, a radar, an ultrasonic sensor, an image sensor, an inertial measurement unit, a satellite signal receiving unit GPS, a control panel and an output module. The control panel comprises an Orange Pi development board and a control circuit. The output module comprises a loudspeaker and a vibration feedback device; the control panel serves as a processing unit and is used for data processing. According to the EITYOLOV8 model, on the basis of YOLOV8, a convolutional layer is reconstructed, deep separable convolution and dynamic convolution kernel technologies are adopted, the feature extraction efficiency is improved, ResNet and MobileNet backbone networks are combined, and the identification degree and generalization ability of the model are enhanced; by introducing MaskFusion semantics and a dynamic RGB-D SLAM system and combining an unscented Kalman filter optimization SLAM algorithm, multi-sensor data fusion is realized, the navigation precision and safety are improved, and the blind person is assisted in daily travel.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an intelligent blind guide helmet system. Background Art

[0002] In recent years, scholars have developed various intelligent guide stick devices, usually adding sensors to traditional guide sticks to provide road information. However, these devices have single functions and are limited in handling complex environments. Although they are cheap and easy to carry, their stability and applicable scenarios are limited. Wearable guide devices integrate navigation systems, but they are insufficient in processing real-time scenarios, such as zebra crossing judgment, vehicle distance and pedestrian avoidance. In general, research on guide devices is still limited to safe obstacle avoidance, and there is insufficient research on algorithms combining target detection and ranging. With the development of science and technology, we should use technology to realize the desire of guide devices to replace visual functions while ensuring the safety of the blind.

[0003] The object recognition technology used in the prior art: the camera is installed in front of the helmet to complete the image acquisition work, collect object pictures at different angles, and input the video stream into the YOLO target detection model to scale, normalize, extract features, and recognize and output the pictures, and convert the original image into a black and white image. Different thresholds are set by threshold segmentation, and the colors in the image similar to the color of the blind path are "corroded" by "corrosion operation", and the required object information is retained. According to the code, the pattern features extracted from the picture are matched with the trained model features, and finally the results are output to detect the target object. However, the Darknet target detection model is run on YOLO. Darknet is a relatively niche deep learning framework with no community. It is mainly maintained by the author team, so the promotion is weak and the number of users is small. In addition, due to limited maintenance personnel, the use of functions is relatively small compared to the SSD model, which is not conducive to the construction and use of the target detection model. Existing guide devices such as guide sticks and smart glasses have played a role in helping the blind avoid obstacles, but they usually have a single function and are difficult to adapt to complex indoor and outdoor environments. The navigation accuracy is not high enough. The development of modern science and technology, such as visual SLAM (Simultaneous Localization and Mapping) technology and deep learning target detection technology (such as YOLO), has made it possible to further improve guide devices. YOLO_V8 is the latest generation of target detection algorithm, which uses convolutional neural network (CNN) for fast and efficient object detection. SLAM technology can build environmental maps and locate in real time through image data. Combining these two technologies can effectively improve the perception ability of guide devices in complex dynamic environments, especially enhance the ability to detect and avoid obstacles. Through intelligent technology, the blind can get more accurate navigation services. Summary of the invention

[0004] In order to solve the above problems, the present invention provides an intelligent guide helmet system.

[0005] To implement the above technical solution, the details are as follows:

[0006] An intelligent guide helmet system is provided with: a camera 1, a radar 2, an ultrasonic sensor 3, an image sensor 4, an inertial measurement unit (IMU) 5, a satellite signal receiving unit (GPS) 6, a mobile network signal transceiver unit 7, a control board 8 and an output module 9;

[0007] Camera 1 is installed at the front of the helmet facing the direction of walking; Camera 1 is used to obtain image information around the blind person, and combines the EITYOLO_V8 model to perform target detection and obstacle recognition; Camera 1 can independently obtain image information of the surrounding environment and transmit it to the processor for object detection and environmental perception;

[0008] The radar sensor 2 is installed on the top of the helmet to detect long-distance obstacle information;

[0009] Ultrasonic sensor 3 is used to assist in detecting close-range obstacles;

[0010] The image sensor 4 is installed inside the helmet and is used to transmit the image data collected by the camera 1;

[0011] The inertial measurement unit (IMU) 5 captures the acceleration and angular velocity of the helmet, updates the posture information of the helmet in real time, directly interacts with the optimized SLAM system and jointly optimizes it to achieve accurate spatial positioning; the present invention adopts a 9-axis inertial measurement unit (IMU) posture sensor, which directly interacts with the dynamic RGB-D SLAM system and jointly optimizes it to minimize the motion and measurement information related to the posture estimation error modeling;

[0012] The satellite signal receiving unit (GPS) 6 obtains the current position of the helmet by receiving signals from multiple GPS satellites;

[0013] The present invention adopts a high-precision SLAM algorithm based on unscented Kalman filtering and combined with IMU, GPS, radar and ultrasonic sensing signals to improve the positioning accuracy and navigation stability of the system; the algorithm can effectively fuse multiple sensor data (radar sensor 2, ultrasonic sensor 3, image sensor 4, 9-axis inertial measurement unit (IMU) attitude sensor) to provide high-precision positioning and navigation services even in complex dynamic environments;

[0014] The mobile network signal transceiver unit 7 is responsible for receiving real-time map data and uploading information obtained by various sensors to ensure that the map data in the Orange Pi development board is always up to date;

[0015] Control board 8 uses Orange Pi as a development board, which is responsible for processing the data collected from camera 1 and various sensors; the processor of the development board combines the EITYOLO_V8 model to detect targets in the image and identify obstacles and road information;

[0016] The output module 9 includes: a speaker and a vibration feedback device; the processor of the development board outputs navigation instructions through the speaker according to the detected path and obstacle information, and helps the blind avoid obstacles through vibration feedback;

[0017] The navigation method includes the following steps:

[0018] S1, collecting images of the surrounding environment while walking through camera 1, and transmitting the images of the surrounding environment while walking to the Orange Pi development board through image sensor 4;

[0019] S2, obtain obstacle recognition results by using a weighted fusion strategy for radar sensor 2 and ultrasonic sensor 3;

[0020] Radar data weight W r and ultrasonic data weight W u Calculated by the following formula:

[0021]

[0022] Where, d r and d u are radar adjustment parameters and ultrasonic adjustment parameters, representing the distance information detected by radar and ultrasonic respectively; μ r and μ u is the perception threshold of radar and ultrasonic wave, which can be obtained by preset;

[0023] S3, inputting the obstacle recognition result, the posture information of the inertial measurement unit (IMU) 5 and the location information of the satellite signal receiving unit (GPS) 6 into the mobile network signal transceiver unit 7;

[0024] S4, input the results of S1 and S3 into the control board (8) using Orange Pi as the development board for analysis, and obtain a map with obstacle information;

[0025] The analysis includes: using the EITYOLO_V8 model to detect targets, identify obstacles and road information, optimize the SLAM algorithm for semantic segmentation, and build a three-dimensional point cloud map; the road information recognition is specifically: inputting the collected image into the Orange Pi development board in the control board 8, and detecting the road information through the EIT YOLO_V8 model in the development board;

[0026] The analysis steps are as follows:

[0027] S4.1. Construct the EITYOLO_V8 model by optimizing the convolutional layers and backbone network.

[0028] The construction steps are as follows:

[0029] S4.1.1. Perform a depthwise separable convolution (DSC) operation.

[0030] Depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution;

[0031] By decomposing traditional convolution into two steps, depth-wise separable convolution significantly reduces the amount of computation and the number of parameters while retaining the ability to effectively extract target features.

[0032] The depth-wise separable convolution operation is expressed as follows:

[0033]

[0034] In the formula, w i is the depth convolution kernel, represents an element-by-element multiplication operation, x i and x j is an element in the road feature map actually collected in this application, and h' is the processed road feature map;

[0035] As can be seen from the expression, the depth-wise separable convolution applies different convolution kernels on each channel and combines the outputs together through point-by-point convolution;

[0036] S4.1.2. Use dynamic convolution kernel (DCK) for feature extraction;

[0037] S4.1.3, perform Multi-Backbone Fusion (MBF) operations;

[0038] The multi-backbone network includes: ResNet network and MobileNet network;

[0039] Through multi-backbone network fusion, the EITYOLO_V8 model can simultaneously utilize the deep features of ResNet and the lightweight features of MobileNet; the feature maps of the two networks are combined through weighted fusion, which enhances the model's adaptability to different types of targets and scenarios; the fusion formula is as follows:

[0040] F final (x) = αF1(x) + (1-α)F2(x)

[0041] In the formula, F1(x) and F2(x) are feature maps from two different backbone networks, ResNet and MobileNet, respectively; α is a weighting coefficient, which indicates the contribution of the two feature maps in the final fusion result. According to the performance of different networks, the value of α can be dynamically adjusted and varies between 0≤α≤1. The weighted fusion process enables the EITYOLO_V8 model to provide a more powerful feature extraction capability by combining the advantages of two different networks, thereby improving the accuracy and generalization ability of target detection.

[0042] S4.2, using MaskFusion semantics and dynamic RGB-D SLAM system to optimize the SLAM algorithm and obtain an optimized SLAM algorithm;

[0043] The MaskFusion semantic and dynamic RGB-D SLAM system can achieve object recognition, semantic segmentation, and tracking and reconstruction of the environment in a scene during continuous autonomous motion;

[0044] The optimization process is as follows:

[0045] The specific implementation method is to perform semantic segmentation on the road feature map actually collected in the present invention through a neural network model, generate an object mask, and build a three-dimensional point cloud map with semantic labels in combination with depth information (the distance from each pixel to the camera);

[0046] Furthermore, the road feature image actually collected is I RGB , the depth information image is I D , the semantic mask is M, then the output of the 3D point cloud map can be expressed as:

[0047] P = f(I RGB ,I D ,M)

[0048] In the formula, f represents the fusion function, which is responsible for integrating multi-source information into a unified three-dimensional map;

[0049] S4.3, input the results of S1 and S3 into the EITYOLO_V8 model and the optimized SLAM algorithm to obtain a map with obstacle information, and use the unscented Kalman filter algorithm to optimize the accuracy of the map;

[0050] Furthermore, in order to improve the positioning accuracy and robustness of the optimized SLAM algorithm, a high-precision SLAM algorithm based on the Unscented Kalman Filter (UKF) is used in the Orange Pi development board; this algorithm combines IMU, GPS, radar and ultrasonic sensing signals to accurately estimate the system state;

[0051] Specifically, define the system state vector X = [x, y, z, v x ,vy ,v z ,θ,φ,ψ] T , including: position [x, y, z]; velocity [v x ,v y ,v z ]; attitude information [θ, φ, ψ], which represent the yaw angle, pitch angle and roll angle under Euler angle respectively; through the unscented Kalman filter algorithm, the Orange Pi development board performs two steps of update, including: time update operation and measurement update operation; the state vector is continuously corrected to achieve accurate state estimation; the time update process is used to predict the current state according to the dynamic model of the system; the expression of time update is:

[0052] X k|k-1 =FX k-1|k-1 +Q

[0053] Where, X k|k-1 is the predicted system state vector, which represents the state estimate at time k; F is the state transfer matrix, which is used to describe the relationship between the state and time; X k-1|k-1 is the state updated at the last moment k-1; Q is the process noise covariance matrix, which is used to describe the noise introduced by uncertainty;

[0054] During the time update process, the acceleration and angular velocity data provided by the inertial measurement unit (IMU) 5 are used to update the position and attitude estimates; the IMU data is combined with the dynamic model in the Orange Pi development board to predict the system state through the state transfer matrix F; the measurement update process is used to correct the predicted state in combination with the data of external sensors (such as GPS, radar, ultrasound); the measurement update formula is:

[0055] K k =P k|k-1 H T (HP k|k-1 H T +R) -1

[0056] In the formula, K k is the Kalman gain, which determines how to balance between the predicted value and the measured value; P k|k-1 is the covariance matrix of the predicted state; H is the observation matrix, which maps the state variables to the measurement space; R is the observation noise covariance matrix, which describes the uncertainty in the sensor measurement;

[0057] Then the state estimation formula for measurement update is:

[0058] X k|k =X k|k-1 +K k (Zk -HX k|k-1 )

[0059] In the formula, X k|k is the system state after measurement and correction; Z k Observation data (such as location obtained by GPS, obstacle distance measured by radar and ultrasound); HX k|k-1 is the measured value calculated based on the predicted state;

[0060] Furthermore, the covariance update formula is:

[0061] P k|k =(IK k H)P k|k-1

[0062] This formula is used to update the uncertainty of the state and is combined with sensor information for correction, making the estimation of the system state more accurate.

[0063] Furthermore, the Kalman gain (K k ) determines the weighting between predicted and observed values, which is calculated using the predicted covariance matrix and the measurement noise;

[0064] Furthermore, the status update (X k|k ) corrects the state estimate by combining the predicted state and the observed data. The role of the Kalman gain in this process is to balance the weights of prediction and observation;

[0065] Furthermore, the covariance update (P k|k ) updates the uncertainty of the system's state estimation, reflecting the credibility of the new state estimation in the future. Through these three steps, UKF can effectively fuse information from different sensors and improve the accuracy and robustness of state estimation;

[0066] S5. Output the result of S4 through the output module 9 to obtain the final navigation result; through the speaker and the vibration device, ensure that the user can receive the navigation instructions and obstacle avoidance reminders in time.

[0067] Beneficial effects of the present invention

[0068] 1. The EITYOLO_V8 model algorithm used in the present invention improves the efficiency and accuracy of target detection by optimizing the design of the convolutional layer and the backbone network, and enhances the adaptability of the model, so that it can more accurately identify and detect targets in a variety of complex environments;

[0069] 2. The present invention realizes the recognition and semantic segmentation of objects in the scene by introducing MaskFusion semantics and dynamic RGB-D SLAM system, and improves the system's understanding of the environment and the accuracy of navigation;

[0070] 3. The present invention adopts a high-precision SLAM algorithm based on unscented Kalman filtering, combined with IMU and GPS sensing signals, to further improve the positioning accuracy and navigation stability of the system;

[0071] 4. The present invention ensures that the user can receive navigation instructions and obstacle avoidance reminders in a timely manner through the cooperation of the speaker and the vibration device. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A schematic diagram of the intelligent blind guide helmet of the present invention;

[0073] Figure 2 It is a schematic diagram of the navigation method of the present invention. DETAILED DESCRIPTION

[0074] The present invention is further described in detail below in conjunction with specific embodiments.

[0075] An intelligent guide helmet system is provided with: a camera 1, a radar 2, an ultrasonic sensor 3, an image sensor 4, an inertial measurement unit (IMU) 5, a satellite signal receiving unit (GPS) 6, a mobile network signal transceiver unit 7, a control board 8 and an output module 9;

[0076] Camera 1 is installed at the front of the helmet facing the direction of walking; Camera 1 is used to obtain image information around the blind person, and combines the EITYOLO_V8 model to perform target detection and obstacle recognition; Camera 1 can independently obtain image information of the surrounding environment and transmit it to the processor for object detection and environmental perception;

[0077] The radar sensor 2 is installed on the top of the helmet to detect long-distance obstacle information;

[0078] Ultrasonic sensor 3 is used to assist in detecting close-range obstacles;

[0079] The image sensor 4 is installed inside the helmet and is used to transmit the image data collected by the camera 1;

[0080] The inertial measurement unit (IMU) 5 captures the acceleration and angular velocity of the helmet, updates the posture information of the helmet in real time, directly interacts with the optimized SLAM system and jointly optimizes it to achieve accurate spatial positioning; the present invention adopts a 9-axis inertial measurement unit (IMU) posture sensor, which directly interacts with the dynamic RGB-D SLAM system and jointly optimizes it to minimize the motion and measurement information related to the posture estimation error modeling;

[0081] The satellite signal receiving unit (GPS) 6 obtains the current position of the helmet by receiving signals from multiple GPS satellites;

[0082] The present invention adopts a high-precision SLAM algorithm based on unscented Kalman filtering and combined with IMU, GPS, radar and ultrasonic sensing signals to improve the positioning accuracy and navigation stability of the system; the algorithm can effectively fuse multiple sensor data (radar sensor 2, ultrasonic sensor 3, image sensor 4, 9-axis inertial measurement unit (IMU) attitude sensor) to provide high-precision positioning and navigation services even in complex dynamic environments;

[0083] The mobile network signal transceiver unit 7 is responsible for receiving real-time map data and uploading information obtained by various sensors to ensure that the map data in the Orange Pi development board is always up to date;

[0084] Control board 8 uses Orange Pi as a development board, which is responsible for processing the data collected from camera 1 and various sensors; the processor of the development board combines the EITYOLO_V8 model to detect targets in the image and identify obstacles and road information;

[0085] The output module 9 includes: a speaker and a vibration feedback device; the processor of the development board outputs navigation instructions through the speaker according to the detected path and obstacle information, and helps the blind avoid obstacles through vibration feedback;

[0086] The navigation method includes the following steps:

[0087] S1, collecting images of the surrounding environment while walking through camera 1, and transmitting the images of the surrounding environment while walking to the Orange Pi development board through image sensor 4;

[0088] S2, obtain obstacle recognition results by using a weighted fusion strategy for radar sensor 2 and ultrasonic sensor 3;

[0089] Obstacle identification is as follows: radar sensors are used to capture long-distance obstacle information and provide high-precision environmental perception. They have a wide detection range and are suitable for detecting large obstacles in the distance. Ultrasonic sensors are mainly responsible for detecting close-range obstacles, especially in crowded environments. Ultrasonic waves can sense dynamic obstacles around in real time. Through the fusion of radar and ultrasonic sensors, the Orange Pi development board can achieve multi-level obstacle detection at both near and far distances, and combine image sensor data to achieve accurate modeling in three-dimensional space. The fusion process adopts a weighted fusion strategy, and the radar data weight W is used as the weight of the radar data. r and ultrasonic data weight W u Calculated by the following formula:

[0090]

[0091] Where, d r and d uare radar adjustment parameters and ultrasonic adjustment parameters, representing the distance information detected by radar and ultrasonic respectively; μ r and μ u is the perception threshold of radar and ultrasonic wave, which can be obtained by preset;

[0092] For radar μ r , according to experience, it is set between 50% and 90% of the maximum effective detection distance (that is, the maximum detection distance of the radar is 100 meters, μ r Can be set to an effective detection distance between 50 meters and 90 meters);

[0093] For ultrasonic μ u According to experience, it is set between 1 meter and 5 meters;

[0094] For radar d r , based on experience r ∈(1, 10), which can adjust the sensitivity appropriately for targets with large distance changes;

[0095] For ultrasonic d u , because ultrasound is more sensitive to changes in close-range targets, we choose d based on experience. u ∈(1,20);

[0096] S3, inputting the obstacle recognition result, the posture information of the inertial measurement unit (IMU) 5 and the location information of the satellite signal receiving unit (GPS) 6 into the mobile network signal transceiver unit 7;

[0097] S4, input the results of S1 and S3 into the control board (8) using Orange Pi as the development board for analysis, and obtain a map with obstacle information;

[0098] The analysis includes: using the EITYOLO_V8 model for target detection, identifying obstacles and road information; using the optimized SLAM algorithm for semantic segmentation and building a three-dimensional point cloud map; the road information recognition is specifically: inputting the collected image into the Orange Pi development board in the control board 8, and detecting the road information through the EIT YOLO_V8 model in the development board;

[0099] The analysis steps are as follows:

[0100] S4.1. Construct the EITYOLO_V8 model by optimizing the convolutional layers and backbone network.

[0101] The construction steps are as follows:

[0102] S4.1.1. Perform a depthwise separable convolution (DSC) operation.

[0103] Depthwise separable convolution decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution;

[0104] For each input channel, the convolution kernel only operates on the data of that channel without involving other channels. Cross-channel convolution is no longer required, thus reducing the amount of calculation.

[0105] Pointwise convolution (PC) uses a 1×1 convolution kernel to convolve all channels; pointwise convolution integrates each output channel generated by the depth convolution to generate a new output feature map;

[0106] By decomposing traditional convolution into two steps, depth-wise separable convolution significantly reduces the amount of computation and the number of parameters while retaining the ability to effectively extract target features.

[0107] Furthermore, the traditional convolution operation is expressed as follows:

[0108]

[0109] In the formula, w ij is the weight of the convolution kernel, x ij is an element in the road feature map actually collected in this application, and h is the processed road feature map;

[0110] The depth-wise separable convolution operation is expressed as follows:

[0111]

[0112] Among them, w i is the depth convolution kernel, represents an element-by-element multiplication operation, x i and x j is an element in the road feature map actually collected in this application, and h' is the processed road feature map;

[0113] As can be seen from the expression, the depth-wise separable convolution applies different convolution kernels on each channel and combines the outputs together through point-by-point convolution;

[0114] S4.1.2. Use dynamic convolution kernel (DCK) for feature extraction;

[0115] The introduction of dynamic convolution kernels improves the flexibility of the model; the dynamic convolution kernel dynamically adjusts the shape and size of the convolution kernel according to different areas of the input image, so that the model can adapt to different visual scenes more accurately; when the edges or contours of objects in the image require a small convolution kernel size (3x3, 5x5, 1x1) to effectively extract features, the dynamic convolution kernel can be adaptively adjusted according to local information, improving detection accuracy and the adaptability of the model;

[0116] S4.1.3, perform Multi-Backbone Fusion (MBF) operations;

[0117] The multi-backbone network includes: ResNet network and MobileNet network;

[0118] Multi-Backbone Fusion improves the model's feature extraction capability by combining the advantages of ResNet and MobileNet networks. Each backbone network has different advantages at different feature levels, and fusing them can bring richer feature representations. ResNet is a deep convolutional network that solves the training problem of deep networks through residual connections and is good at capturing deep features in images, while ResNet focuses more on capturing details from low-level to high-level. MobileNet is a lightweight convolutional network that focuses on computational efficiency and real-time performance of the model. It is suitable for running in resource-constrained environments, used to extract shallower features, and can achieve better performance with lower computational complexity.

[0119] Through multi-backbone network fusion, the EITYOLO_V8 model can simultaneously utilize the deep features of ResNet and the lightweight features of MobileNet; the feature maps of the two networks are combined through weighted fusion, which enhances the model's adaptability to different types of targets and scenarios; the fusion formula is as follows:

[0120] F final (x) = αF1(x) + (1-α)F2(x)

[0121] In the formula, F1(x) and F2(x) are feature maps from two different backbone networks, ResNet and MobileNet, respectively; α is a weighting coefficient, which indicates the contribution of the two feature maps in the final fusion result. According to the performance of different networks, the value of α can be dynamically adjusted and varies between 0≤α≤1. The weighted fusion process enables the EITYOLO_V8 model to provide a more powerful feature extraction capability by combining the advantages of two different networks, thereby improving the accuracy and generalization ability of target detection.

[0122] S4.2, using MaskFusion semantics and dynamic RGB-D SLAM system to optimize the SLAM algorithm and obtain an optimized SLAM algorithm;

[0123] The MaskFusion semantic and dynamic RGB-D SLAM system can achieve object recognition, semantic segmentation, and tracking and reconstruction of the environment in a scene during continuous autonomous motion;

[0124] The optimization process is as follows:

[0125] The specific implementation method is to perform semantic segmentation on the road feature map actually collected in the present invention through a neural network model, generate an object mask, and build a three-dimensional point cloud map with semantic labels in combination with depth information (the distance from each pixel to the camera);

[0126] Furthermore, the road feature image actually collected is I RGB , the depth information image is I D , the semantic mask is M, then the output of the 3D point cloud map can be expressed as:

[0127] P = f(I RGB ,I D ,M)

[0128] In the formula, f represents the fusion function, which is responsible for integrating multi-source information into a unified three-dimensional map;

[0129] S4.3, input the results of S1 and S3 into the EITYOLO_V8 model and the optimized SLAM algorithm to obtain a map with obstacle information, and use the unscented Kalman filter algorithm to optimize the accuracy of the map;

[0130] Furthermore, in order to improve the positioning accuracy and robustness of the optimized SLAM algorithm, a high-precision SLAM algorithm based on the Unscented Kalman Filter (UKF) is used in the Orange Pi development board; this algorithm combines IMU, GPS, radar and ultrasonic sensing signals to accurately estimate the system state;

[0131] Specifically, define the system state vector X = [x, y, z, v x ,v y ,v z ,θ,φ,ψ] T , including: position [x, y, z]; velocity [v x ,v y ,v z]; attitude information [θ, φ, ψ], which represent the yaw angle, pitch angle and roll angle under Euler angle respectively; through the unscented Kalman filter algorithm, the Orange Pi development board performs two steps of update, including: time update operation and measurement update operation; the state vector is continuously corrected to achieve accurate state estimation; the time update process is used to predict the current state according to the dynamic model of the system; the expression of time update is:

[0132] X k|k-1 =FX k-1|k-1 +Q

[0133] In the formula, X k|k-1 is the predicted system state vector, which represents the state estimate at time k; F is the state transfer matrix, which is used to describe the relationship between the state and time; X k-1|k-1 is the state updated at the last moment k-1; Q is the process noise covariance matrix, which is used to describe the noise introduced by uncertainty;

[0134] During the time update process, the acceleration and angular velocity data provided by the inertial measurement unit (IMU) 5 are used to update the position and attitude estimates; the IMU data is combined with the dynamic model in the Orange Pi development board to predict the system state through the state transfer matrix F; the measurement update process is used to correct the predicted state in combination with the data of external sensors (such as GPS, radar, ultrasound); the measurement update formula is:

[0135] K k =P k|k-1 H T (HP k|k-1 H T +R) -1

[0136] In the formula, K k is the Kalman gain, which determines how to balance between the predicted value and the measured value; P k|k-1 is the covariance matrix of the predicted state; H is the observation matrix, which maps the state variables to the measurement space; R is the observation noise covariance matrix, which describes the uncertainty in the sensor measurement;

[0137] Then the state estimation formula for measurement update is:

[0138] X k|k =X k|k-1 +K k (Z k -HX k|k-1 )

[0139] In the formula, X k|k is the system state after measurement and correction; Z kObservation data (such as location obtained by GPS, obstacle distance measured by radar and ultrasound); HX k|k-1 is the measured value calculated based on the predicted state;

[0140] Furthermore, the covariance update formula is:

[0141] P k|k =(IK k H)P k|k-1

[0142] This formula is used to update the uncertainty of the state and is combined with sensor information for correction, making the estimation of the system state more accurate.

[0143] Furthermore, the Kalman gain (K k ) determines the weighting between predicted and observed values, which is calculated using the predicted covariance matrix and the measurement noise;

[0144] Furthermore, the status update (X k|k ) corrects the state estimate by combining the predicted state and the observed data. The role of the Kalman gain in this process is to balance the weights of prediction and observation;

[0145] Furthermore, the covariance update (P k|k ) updates the uncertainty of the system's state estimation, reflecting the credibility of the new state estimation in the future. Through these three steps, UKF can effectively fuse information from different sensors and improve the accuracy and robustness of state estimation.

[0146] Compared with the traditional extended Kalman filter (EKF), the unscented Kalman filter (UKF) used in the present invention has higher accuracy, especially when dealing with nonlinear systems. It calculates the mean and covariance of state estimation through unscented transform (UT), avoiding the error caused by the linearization process. The unscented transform transfers the distribution of state variables to the next moment through a set of determined sampling points, thereby more accurately capturing the nonlinear characteristics of the system. The processor performs more intelligent path planning based on the semantic map generated by the optimized SLAM system and the detection results of the EITYOLO_V8 model, combined with obstacle information. The system uses the detected object information and three-dimensional map to select the optimal route and effectively avoid obstacles.

[0147] S5, output the result of S4 through the output module 9 to obtain the final navigation result; through the speaker and the vibration device, ensure that the user can receive the navigation instructions and obstacle avoidance reminders in time;

[0148] The battery of the present invention adopts a model aircraft battery, and its main function is to provide a stable power supply for the equipment. At the same time, a step-down module is used to provide a stable power supply for the development board to ensure the stability of the equipment during operation. The present invention adopts all-optical devices for navigation and recognition, and has faster and more accurate image detection and classification processing capabilities. The control board receives real-time map data through a mobile network signal transceiver unit and uploads the information obtained by the device to ensure that the map data in the system is always kept up to date.

Claims

1. An intelligent guide helmet system, characterized in that: The system comprises: a camera (1), a radar (2), an ultrasonic sensor (3), an image sensor (4), an inertial measurement unit (5), a satellite signal receiving unit (6), a mobile network signal transceiver unit (7), a control panel (8) and an output module (9); The camera (1) is mounted on the front end of the helmet facing the walking direction; the radar sensor (2) is mounted on the top of the helmet. The navigation method of the system comprises the following steps: S1, collecting images of the surrounding environment while walking through a camera (1), and transmitting the images of the surrounding environment while walking to an Orange Pi development board through an image sensor (4); S2, obtaining an obstacle recognition result by using a weighted fusion strategy for the radar sensor (2) and the ultrasonic sensor (3); S3, inputting the obstacle recognition result, the posture information of the inertial measurement unit (5) and the position information of the satellite signal receiving unit (6) into the mobile network signal transceiver unit (7); S4, input the results of S1 and S3 into the control board (8) using Orange Pi as the development board for analysis, and obtain a map with obstacle information; S5. Output the result of S4 through the output module (9) to obtain the final navigation result; and ensure that the user can receive the navigation instruction and obstacle avoidance reminder in time through the speaker and the vibration device.

2. The intelligent blind guide helmet system according to claim 1, characterized in that: The expression for obtaining the obstacle recognition result by using the radar sensor (2) and the ultrasonic sensor (3) through a weighted fusion strategy is as follows: Where, d r and d u are radar adjustment parameters and ultrasonic adjustment parameters, representing the distance information detected by radar and ultrasonic respectively; μ r and μ u is the perception threshold of radar and ultrasonic wave; W r Represents radar data weight; W u Indicates the ultrasonic data weight.

3. The intelligent blind guide helmet system according to claim 1, characterized in that: The results of S1 and S4 are input into a control board (8) using Orange Pi as a development board for analysis, and finally a map with obstacle information is obtained; The analysis includes: using the EITYOLO_V8 model to detect targets and identify obstacles and road information; using an optimized SLAM algorithm to perform semantic segmentation and construct a three-dimensional point cloud map; wherein the road information recognition specifically includes: inputting the collected image into the Orange Pi development board in the control board (8), and detecting the road information through the EITYOLO_V8 model in the development board; Here are the steps: S4.

1. Construct the EITYOLO_V8 model by optimizing the convolutional layers and backbone network. S4.2, using MaskFusion semantics and dynamic RGB-D SLAM system to optimize the SLAM algorithm and obtain an optimized SLAM algorithm; S4.3, input the results of S1 and S3 into the EITYOLO_V8 model and the optimized SLAM algorithm to obtain a map with obstacle information, and use the unscented Kalman filter algorithm to optimize the accuracy of the map; The precision optimization includes: time update operation and measurement update operation; The expression for time update is: X k|k-1 =FX k-1|k-1 +Q Where, X k|k-1 is the predicted system state vector; F is the state transfer matrix; X k-1|k-1 is the state updated at the last moment k-1; Q is the process noise covariance matrix; The measurement update expression is: K k =P k|k-1 H T (HP k|k-1 H T +R) -1 In the formula, K k is the Kalman gain; P k|k-1 is the covariance matrix of the predicted state; H is the observation matrix; R is the observation noise covariance matrix.

Images