Novel blind guiding system based on image semantic segmentation network

Abstract

The invention discloses a novel blind guiding system based on an image semantic segmentation network, and relates to the technical field of blind guiding path planning, the blind guiding system uses an OCRNet image semantic segmentation algorithm under a MindSpore framework, realizes accurate perception and understanding of a complex environment of urban streetscape, has excellent calculation performance and strong distributed training ability, and can be applied to the field of blind guiding path planning. According to the method, mass image data collected by a binocular camera can be deeply analyzed, various objects in the image are accurately classified and segmented, the conditions of roads, buildings, pedestrians, traffic lights and vehicles are accurately distinguished, higher accuracy and higher robustness are shown in a complex scene, and the method is suitable for popularization and application. Even under the conditions of dim light, object shielding or complex and changeable environment, the environment information can still be accurately recognized, reliable data support is provided for intelligent path planning and safe travel of blind users, and the blind are helped to see the surrounding world clearly.

Description

Technical Field

[0001] The present invention relates to the technical field of blind guide path planning, and in particular to a novel blind guide system based on an image semantic segmentation network. Background Art

[0002] Current urban road conditions are far from ideal, primarily due to incomplete blind path infrastructure, frequent obstruction, and damage. Coupled with complex urban traffic conditions, these issues pose significant safety risks for visually impaired individuals, even posing life-threatening risks. People with visual impairments face significant travel challenges. Traditional guidance methods, such as guide dogs and standard canes, have significant limitations in the complex and ever-changing modern environment. Guide dogs are expensive, difficult to train, and limited in number, hindering widespread adoption. Standard canes provide only basic detection and are unable to help blind people accurately navigate complex environments or navigate complex situations such as traffic. This often leaves blind people feeling helpless and lost, significantly limiting their range and independence. Traditional guidance systems, such as the Bat Obstacle Avoidance system, rely on continuous scanning with an Apple LiDAR phone. While incorporating high-precision positioning and obstacle detection technology, these systems suffer from high equipment dependency and a heavy operational burden. The positioning signal is easily obstructed by various obstacles, resulting in low positioning accuracy and challenges in navigation accuracy.

[0003] Image semantic segmentation is a key branch of computer vision, providing effective solutions to computer vision problems across many fields. It aims to classify each pixel in an image into a predefined semantic category and label each category with a different color, thereby achieving pixel-level semantic understanding and segmentation of the image. Therefore, fully leveraging image semantic segmentation networks is crucial for improving navigation for the blind.

[0004] In summary, in order to help blind people perceive the roads and obstacles in their surrounding environment and provide accurate navigation and obstacle avoidance information, it is urgent to provide a new blind guide system based on image semantic segmentation network. Summary of the Invention

[0005] The purpose of the present invention is to provide a new blind guide system based on an image semantic segmentation network to solve the problem raised in the above background technology of helping blind people perceive the roads and obstacles in their surrounding environment and provide accurate navigation and obstacle avoidance information.

[0006] To achieve the above-mentioned object, the present invention provides the following technical solutions: a new blind guide system based on an image semantic segmentation network, comprising a wearable blind guide device, a mobile terminal APP, and an online data service layer;

[0007] The wearable guide device includes a binocular camera, a voice navigation module, an image processing module, a wireless communication module, a headset module, a power module, an early warning module, and a main control module; the online data service layer includes a data cleaning module, a data storage module, and a data transmission module;

[0008] The binocular camera is electrically connected to the image processing module and is used to capture real-time images of the road and environment along the blind person's route. The captured image data is then transmitted to the image processing module for image analysis to obtain road condition information along the route. The voice navigation module is used to plan the blind person's route and make real-time corrections to the navigation route based on the blind person's actual route. The headset module is electrically connected to the wireless communication module and is used for online communication between the guardian and the blind user to obtain real-time status information of the blind person. The power supply module adopts a dual power supply mode, namely a large-capacity battery and a small hand-cranked generator. The early warning module is used to warn in emergency situations.

[0009] The online data service layer is used to perform noise reduction filtering, cache backup and real-time online transmission on the collected environmental image data during navigation.

[0010] Preferably, the image processing module includes an image input unit, a feature extraction unit, a feature enhancement unit and a fusion feature analysis unit;

[0011] The image input unit is used to convert the images collected by the binocular camera in a unit time into a feature image in a unified format. The input format is H×W×F, where H and W represent the height and width of the feature map, respectively, and F represents the input feature map set.

[0012] The feature extraction unit is used to extract features from the input feature map set. The extracted image features include roads, buildings, pedestrians, vehicles, traffic lights, green vegetation and obstacles, and the extracted feature data is input into the feature enhancement unit.

[0013] The feature enhancement unit is used to capture continuous feature information in time and space, establish a cross-attention mechanism, transfer and exchange information between different locations, and fuse features from different images through the cross-attention mechanism to finally generate a feature fusion map;

[0014] The fusion feature analysis unit is used to analyze the feature fusion graph, identify the type of features in the graph, and pass the analyzed data to the main control module.

[0015] Preferably, the specific steps of converting the image into an input feature image of a unified format are:

[0016] (1) Image preprocessing: load n input images in parallel, convert them into the same color space, and generate a metadata index table to record the shooting time, size, format and image quality score of each image;

[0017] (2) Clarity assessment: Perform image quality assessment on the preprocessed image and give an image quality score;

[0018] (3) Image arrangement and classification: Arrange the processed images in the order of shooting time and divide the images into 5 equal parts. In each image classification, discard the images with low image quality scores. Finally, reintegrate and arrange the discarded images to form a data set F of valid images per unit time, F = {F1, F2, ... F n};

[0019] (4) Output image data: Output the processed image data set for image feature extraction.

[0020] Preferably, the feature enhancement unit extracts the feature information F C The images are processed by maximum pooling and average pooling respectively. Maximum pooling is used to extract the most significant features of each sub-region and highlight the main features of the image. Average pooling is used to calculate the average value from each sub-region of the feature map, retaining the overall feature distribution information and the global structure of the image. The two pooling features are spliced with the original input features in the channel dimension to obtain the feature map F. A , F A The expression is: F A =f c (f Avgpool (F C ), f maxpool =(F C ), F C );

[0021] The splicing feature F A The data are fed into three branches respectively. Each branch is transformed nonlinearly at the beginning by point convolution with the ReLU nonlinear activation function to capture complex features and distinguish different semantic categories. The features of the first branch are then multiplied pixel by pixel with the feature map of the second branch after maximum pooling and feature conversion, thereby aggregating feature information from different dimensions and associating pixel-level features with multiple categories to generate a weight vector. The features of the third branch are multiplied with the weight vector generated by the second branch after average pooling and feature conversion to obtain the enhanced feature representation F. B ;

[0022] The original concatenated features and the enhanced feature representation are added pixel by pixel, and then the dimensionality is reduced and integrated through 1×1 convolution to generate the final enhanced feature map FMBF , as input for further processing to generate the final segmentation result, which is used to enhance the performance of semantic segmentation tasks.

[0023] Preferably, the voice navigation module includes an environment perception system unit, a voice interaction system unit, an offline map system unit and a custom input unit;

[0024] The environmental perception system unit includes an air pressure sensor, a temperature and humidity sensor, and an ultrasonic module. The air pressure sensor is used to sense the blind person's altitude. Every 0.1hPa pressure difference is equivalent to a vertical drop of 1 meter. The temperature and humidity sensor is used to identify the slipperiness of the road surface. The ultrasonic module is used to emit 40-60kHz ultrasonic waves and construct a 1-meter accurate spatial map based on the echo time difference to alert the blind person to any obstacles ahead.

[0025] The voice interaction system unit includes bone conduction headphones and an intelligent AI system. Voice is transmitted through the bone conduction headphones to retain ambient sound perception. The intelligent AI system is used to compress navigation instructions into three key elements: direction of travel, distance to travel, and type of hazard.

[0026] The offline map system unit is used to generate barrier-free path heat maps after desensitizing user trajectory data, realizing navigation for the blind in an offline state. The custom input unit is used for guardians to customize the blind's forward route and destination location.

[0027] Preferably, the specific algorithm process of the ultrasonic module to construct a 1m precision spatial map is as follows:

[0028] Calculate the distance d of the obstacle ahead once 测量 , Where v represents the speed of sound of the ultrasonic wave in the current environment, and Δt represents the time difference between the ultrasonic wave emission and the echo reception;

[0029] At this ambient temperature, dynamic compensation of sound velocity is performed, i.e. v = 331.4 + 0.6 × T, where T is the current ambient temperature;

[0030] Posture compensation: When a blind user tilts the device, the actual projection distance must be compensated for, i.e., d 实际 =d 测量 ×cosθ, where θ is the angle between the ultrasonic module and the horizontal plane;

[0031] The calculation of the distance after correction compensation is:

[0032] Preferably, the early warning module includes a Beidou positioning unit, an IMU sensor, a remote positioning and monitoring unit, and an electronic fence unit;

[0033] The Beidou positioning unit is used to locate the position of blind users in real time;

[0034] The IMU sensor is used to monitor the blind person's body posture in real time. When the IMU sensor detects a fall, it transmits the collected signal data to the main control module, which controls the wireless communication module to promptly send the fall information and the blind person's location to the guardian's mobile phone via text message.

[0035] The remote positioning and monitoring unit is used to bind with the mobile terminal APP, so that the guardian can check the location and movement trajectory of the blind user and keep track of the blind user's whereabouts at any time;

[0036] The electronic fence unit is used by the guardian to set up a safe geographical area. When the blind person goes beyond this area, an alarm will be sounded immediately to alert the guardian.

[0037] Preferably, the mobile terminal APP is a HarmonyOS-based APP, which allows the guardian to view the map location of the blind person in real time on the mobile phone through the APP. The APP includes a real-time video function. The guardian can open the real-time video interface by tapping "real-time video" on the screen and directly watch the environment in front of the blind person.

[0038] Preferably, the data cleaning module is used to eliminate the multipath reflection interference of the ultrasonic module, detect and repair sensor burst errors through a sliding window algorithm, the data storage module is used to store the environmental data of the last 1 minute in RAM, store historical path data in column format, and when a fall event is triggered, save the original sensor data of the previous 30 seconds to a tamper-proof storage area, and all location data are homomorphically encrypted when stored; the data transmission module dynamically selects the transmission protocol according to the network quality, automatically matches the network transmission under 4G / 5G / Wi-Fi / Bluetooth conditions, enables the LoRa emergency communication mode in the signal blind spot, and maintains basic navigation at a rate of 1kbps.

[0039] Preferably, the guide system also includes an external connection device, which is a monitor for monitoring human life and health information, and is used to obtain the human life and health information of blind users and the collected human life and health information main control module, so that the monitoring person can access and view the human life and health information of the blind user through the terminal device.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This guidance system uses the OCRNet image semantic segmentation algorithm within the MindSpore framework to achieve precise perception and understanding of complex urban streetscape environments. With excellent computing performance and powerful distributed training capabilities, it can conduct in-depth analysis of massive image data collected by binocular cameras, accurately classifying and segmenting various objects in the images, and accurately distinguishing between roads, buildings, pedestrians, traffic lights, and vehicles. It demonstrates higher accuracy and greater robustness in complex scenarios. Even in dim light, obstructed objects, or complex and changing environments, it can still accurately identify environmental information, providing reliable data support for intelligent route planning and safe travel for blind users, helping them "see" the world around them clearly.

[0042] The binocular camera extracts depth information. Combined with the acoustic obstacle avoidance function of the voice navigation module, it uses the principle of parallax to accurately calculate the distance between surrounding objects and the device. Using the OCRNet model within the Mindspore framework, pixel-by-pixel image semantic classification is performed, classifying different objects and scene elements into semantic categories such as roads, obstacles, and pedestrians. Finally, the depth and pixel semantic information are integrated to complete semantic environment reconstruction. In complex city streets, blind people can avoid surrounding obstacles through voice prompts.

[0043] With comprehensive functions, guardians can link the Ascend Navigator device through the Hongmeng app to achieve remote positioning and monitoring, greatly improving the difficulty of blind people in traveling. Through intelligent means, it ensures the safety of blind people's travel and enables them to participate in social activities more independently, greatly improving the quality of life of blind people, helping them better integrate into society, promoting society's understanding and acceptance of the blind community, and promoting social integration.

[0044] This guide system will play an important role in promoting the construction of a barrier-free society in my country. On the one hand, it provides the blind with advanced travel assistance tools, allowing them to feel the convenience brought by technology during their travels, reflecting the society's care and respect for special groups, and helping to create a good social atmosphere of caring for people with disabilities; on the other hand, with the popularization of this equipment, all sectors of society will pay more attention to the travel safety of the blind and the construction of barrier-free facilities, prompting the government and relevant departments to increase investment in the maintenance and construction of barrier-free facilities such as urban blind paths, and improve the city's barrier-free travel system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 1 is a system module diagram of the blind guiding system of the present invention;

[0046] Figure 2 This is a module structure diagram of the feature enhancement unit of the present invention;

[0047] Figure 3 This is the logic diagram of the fall detection algorithm of the present invention;

[0048] Figure 4 This is a display diagram of the mobile terminal APP of the present invention. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0050] See also Figure 1-4 ,The present invention provides a new blind guide system based on image semantic segmentation network, including a wearable blind guide device, a mobile terminal APP and an online data service layer;

[0051] The wearable guide device includes a binocular camera, a voice navigation module, an image processing module, a wireless communication module, a headset module, a power module, an early warning module, and a main control module; the online data service layer includes a data cleaning module, a data storage module, and a data transmission module;

[0052] The binocular camera is electrically connected to the image processing module and is used to capture real-time images of the road and environment along the blind person's route. The captured image data is then transmitted to the image processing module for image analysis to obtain road condition information along the route. The voice navigation module is used to plan the blind person's route and make real-time corrections to the navigation route based on the blind person's actual route. The headset module is electrically connected to the wireless communication module and is used for online communication between the guardian and the blind user to obtain real-time status information of the blind person. The power supply module adopts a dual power supply mode, namely a large-capacity battery and a small hand-cranked generator. The early warning module is used to warn in emergency situations.

[0053] The online data service layer is used to perform noise reduction filtering, cache backup and real-time online transmission on the collected environmental image data during navigation.

[0054] See Figure 1 and Figure 2At the hardware level, the system is equipped with an Orange Pi AIpro main control board and a binocular camera. The Orange Pi AIpro main control board, with its powerful AI computing power, provides a solid guarantee for running complex algorithms and processing large amounts of data. The binocular camera is used to collect accurate depth information and build a three-dimensional model of the surrounding environment. Various sensors are closely integrated to jointly detect real-time environmental information. The OCRNet image semantic segmentation algorithm under the MindSpore framework is used to achieve accurate perception and understanding of the complex environment of urban street scenes. The Hongmeng APP is closely connected with these hardware devices and can obtain various data collected by the hardware in real time, such as the real-time location of the blind person, surrounding environment information, and the working status of the equipment, and present this information intuitively to the guardian. The guardian can clearly see the map location of the blind person's location and the real-time video image of the camera on the mobile phone through the APP, so as to understand the blind person's travel status at any time.

[0055] Orange Pi AIpro 8T 16G adopts the Ascend AI technology route, highly integrating CPU, AI computing, ISP, graphics output and other functions. It has 8TOPS AI computing power, which can effectively realize the acceleration of AI applications such as target recognition and image classification. Orange Pi AIpro is a quad-core 64-bit processor and AI processor with an integrated graphics processor. Orange Pi AIpro contains a fairly rich interface, including GPIO interface and two USB3.0, which can meet all the requirements of this project, including connecting 3 button sensors, 1 IMU inertial navigation sensor, 1 Beidou positioning sensor, 1 USB headset and 1 binocular camera. Huawei's GPIO interface is used to connect to the button sensor, transmit high and low levels to control the device to enable different functions, and connect multiple external sensors through the USB expansion dock. Overall, Orange Pi AIpro can meet the needs of this project;

[0056] The OCRNet model under the MindSpore framework is used as the image analysis module. OCRNet uses a new object context information, explicitly enhancing the contribution of pixels from the same type of object when constructing context information. OCRNet proposes an efficient staggered sparse self-attention scheme, which simulates the dense relationship between any two pixels in all pixels through the combination of two sparse relationship matrices. To further improve the performance and adaptability of the model, the project team trained the Cityspaces dataset on Huawei's AI development platform Modelarts. After parameter adjustment and optimization, they finally obtained the model ckpt file with the best performance. Compared with image semantic segmentation algorithms such as PSPNet, ACFNet, and HRNet, OCRNet has better segmentation effect and higher accuracy.

[0057] See Figure 3 and Figure 4 , Figure 4 Figure a in the middle is the homepage of the APP function, Figure b is a schematic diagram of the electronic fence on the APP, Figure c is a schematic diagram of path planning, Figure d is a function display diagram of AI consultation, Figure e is a status diagram of adding equipment, and Figure f is a real-time video display diagram. The system is equipped with a high-precision IMU sensor that can monitor the body posture of the blind in real time. When the sensor detects that the blind person falls, it will promptly send the fall information to the guardian's mobile phone via text message. The guardian can know the blind person's fall at the first time and take timely measures to ensure the blind person's safety. The guardian can view the blind person's location and movement trajectory in real time on the map interface of the APP, and keep track of his whereabouts at any time. The APP is also equipped with a real-time video function. The guardian only needs to tap "real-time video" on the screen to open the real-time video interface and directly watch the environment in front of the blind person. Whether traveling during the day or at night, the guardian can rest assured, especially when the blind person goes out alone. This real-time monitoring function allows the guardian to understand his safety status in a timely manner.

[0058] Ascend Navigator relies on the Orange Pi AIpro main control board and binocular cameras to achieve precise environmental perception. The binocular cameras can extract depth information and use the principle of parallax to accurately calculate the distance between surrounding objects and the device. At the same time, with the help of the OCRNet model under the Mindspore framework, pixel semantic classification is performed, and the image is analyzed pixel by pixel to classify different objects and scene elements into semantic categories such as roads, obstacles, and pedestrians. Finally, the depth and pixel semantic information are integrated to complete the semantic environment reconstruction. In complex city streets, it can enable blind people to avoid surrounding obstacles through voice prompts.

[0059] Application direction: Blind users: As a core user group, they have a strong demand for travel safety and independence, and are eager to use advanced technology to overcome travel barriers and integrate into social life;

[0060] Guardians: Family members, friends or related care institutions of the blind, who are concerned about the safety of the blind and hope to understand the blind's dynamics in real time through convenient means and provide them with remote support and protection.

[0061] The results of the field test are shown in the table below: Table 1 shows the field test results of the path planning solution:

[0062]

[0063] The guidance system leverages the efficient computing power of the Orange Pi AIPro development board, the powerful AI algorithm support of the MindSpore framework, and the convenient interactive experience of the app, giving the Ascend Navigator precise environmental perception, intelligent path planning, and stable device connectivity.

[0064] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A new blind guidance system based on an image semantic segmentation network, characterized by: Including wearable guide devices, mobile terminal APP and online data service layer; The wearable guide device includes a binocular camera, a voice navigation module, an image processing module, a wireless communication module, a headset module, a power module, an early warning module, and a main control module; the online data service layer includes a data cleaning module, a data storage module, and a data transmission module; The binocular camera is electrically connected to the image processing module and is used to capture real-time images of the road and environment along the blind person's route, and transmit the captured image data to the image processing module for image analysis to obtain road condition information along the route. The voice navigation module is used to plan the blind person's route and to correct the navigation route in real time based on the blind person's actual route. The headset module is electrically connected to the wireless communication module, which is used for online communication between the guardian and the blind user to understand the real-time status information of the blind; The power module adopts dual power supply mode, namely large-capacity battery and small hand-cranked generator; The early warning module is used for early warning in emergency situations; The online data service layer is used to perform noise reduction filtering, cache backup and real-time online transmission on the collected environmental image data during navigation.

2. The novel blind-guiding system based on an image semantic segmentation network according to claim 1, characterized in that: The image processing module includes an image input unit, a feature extraction unit, a feature enhancement unit and a fusion feature analysis unit; The image input unit is used to convert the images collected by the binocular camera in a unit time into a feature image in a unified format. The input format is H×W×F, where H and W represent the height and width of the feature map, respectively, and F represents the input feature map set. The feature extraction unit is used to extract features from the input feature map set. The extracted image features include roads, buildings, pedestrians, vehicles, traffic lights, green vegetation and obstacles, and the extracted feature data is input into the feature enhancement unit. The feature enhancement unit is used to capture continuous feature information in time and space, establish a cross-attention mechanism, transfer and exchange information between different locations, and fuse features from different images through the cross-attention mechanism to finally generate a feature fusion map; The fusion feature analysis unit is used to analyze the feature fusion graph, identify the type of features in the graph, and pass the analyzed data to the main control module.

3. The novel blind-guiding system based on an image semantic segmentation network according to claim 2, characterized in that: The specific steps for converting images into input feature images in a unified format are: (1) Image preprocessing: load n input images in parallel, convert them into the same color space, and generate a metadata index table to record the shooting time, size, format and image quality score of each image; (2) Clarity assessment: Perform image quality assessment on the preprocessed image and give an image quality score; (3) Image arrangement and classification: Arrange the processed images in the order of shooting time and divide the images into 5 equal parts. In each image classification, discard the images with low image quality scores. Finally, reintegrate and arrange the discarded images to form a data set F of valid images per unit time, F = {F1, F2, ... F n }; (4) Output image data: Output the processed image data set for image feature extraction.

4. The novel blind-guiding system based on an image semantic segmentation network according to claim 2, characterized in that: The feature enhancement unit extracts the feature information F C The images are processed by maximum pooling and average pooling respectively. Maximum pooling is used to extract the most significant features of each sub-region and highlight the main features in the image. Average pooling is used to calculate the average value from each sub-region of the feature map, retaining the overall feature distribution information, and retaining the global structure of the image. The two pooled features are spliced with the original input features in the channel dimension to obtain the feature map F. A , F A The expression is: F A =f c (f Avgpool (F C ), f maxpool =(F C ), F C ); The splicing feature F A The data are fed into three branches respectively. Each branch is transformed nonlinearly at the beginning by point convolution with the ReLU nonlinear activation function to capture complex features and distinguish different semantic categories. The features of the first branch are then multiplied pixel by pixel with the feature map of the second branch after maximum pooling and feature conversion, thereby aggregating feature information from different dimensions and associating pixel-level features with multiple categories to generate a weight vector. The features of the third branch are multiplied with the weight vector generated by the second branch after average pooling and feature conversion to obtain the enhanced feature representation F. B ; The original concatenated features and the enhanced feature representation are added pixel by pixel, and then the dimensionality is reduced and integrated through 1×1 convolution to generate the final enhanced feature map F MBF , as input for further processing to generate the final segmentation result, which is used to enhance the performance of semantic segmentation tasks.

5. The novel blind-guiding system based on an image semantic segmentation network according to claim 2, characterized in that: The voice navigation module includes an environmental perception system unit, a voice interaction system unit, an offline map system unit, and a custom input unit; The environmental perception system unit includes an air pressure sensor, a temperature and humidity sensor, and an ultrasonic module. The air pressure sensor is used to sense the blind person's altitude. Every 0.1hPa pressure difference is equivalent to a vertical drop of 1 meter. The temperature and humidity sensor is used to identify the slipperiness of the road surface. The ultrasonic module is used to emit 40-60kHz ultrasonic waves and construct a 1-meter accurate spatial map based on the echo time difference to alert the blind person to any obstacles ahead. The voice interaction system unit includes bone conduction headphones and an intelligent AI system. Voice is transmitted through the bone conduction headphones to retain ambient sound perception. The intelligent AI system is used to compress navigation instructions into three key elements: direction of travel, distance to travel, and type of hazard. The offline map system unit is used to generate barrier-free path heat maps after desensitizing user trajectory data, realizing navigation for the blind in an offline state. The custom input unit is used for guardians to customize the blind's forward route and destination location.

6. The novel blind-guiding system based on an image semantic segmentation network according to claim 5, characterized in that: The specific algorithm process of the ultrasonic module to build a 1m precision spatial map is as follows: Calculate the distance d of the obstacle ahead once 测量 , Where v represents the speed of sound of the ultrasonic wave in the current environment, and Δt represents the time difference between the ultrasonic wave emission and the echo reception; At this ambient temperature, dynamic compensation of sound velocity is performed, i.e. v = 331.4 + 0.6 × T, where T is the current ambient temperature; Posture compensation: When a blind user tilts the device, the actual projection distance must be compensated for, i.e., d 实际 =d 测量 ×cosθ, where θ is the angle between the ultrasonic module and the horizontal plane; The calculation of the distance after correction compensation is:

7. The novel blind-guiding system based on an image semantic segmentation network according to claim 1, characterized in that: The early warning module includes a Beidou positioning unit, an IMU sensor, a remote positioning and monitoring unit, and an electronic fence unit; The Beidou positioning unit is used to locate the position of blind users in real time; The IMU sensor is used to monitor the blind person's body posture in real time. When the IMU sensor detects a fall, it transmits the collected signal data to the main control module, which controls the wireless communication module to promptly send the fall information and the blind person's location to the guardian's mobile phone via text message. The remote positioning and monitoring unit is used to bind with the mobile terminal APP, so that the guardian can check the location and movement trajectory of the blind user and keep track of the blind user's whereabouts at any time; The electronic fence unit is used by the guardian to set up a safe geographical area. When the blind person goes beyond this area, an alarm will be sounded immediately to alert the guardian.

8. The novel blind-guiding system based on image semantic segmentation network according to claim 7, characterized in that: The mobile terminal APP is based on HarmonyOS and is used by guardians to view the map location of the blind person in real time on their mobile phones. The APP includes a real-time video function. Guardians can open the real-time video interface by tapping "Live Video" on the screen and directly view the environment in front of the blind person.

9. The novel blind-guiding system based on image semantic segmentation network according to claim 5, characterized in that: The data cleaning module is used to eliminate multipath reflection interference from the ultrasonic module and detect and repair sensor burst errors through a sliding window algorithm. The data storage module is used to store the environmental data of the last minute in RAM, store historical path data in a columnar format, and save the previous 30 seconds of sensor raw data to a tamper-proof storage area when a fall event is triggered. All location data is dynamically encrypted during storage. The data transmission module dynamically selects the transmission protocol based on network quality, automatically matches network transmission under 4G / 5G / Wi-Fi / Bluetooth conditions, enables LoRa emergency communication mode in signal blind spots, and maintains basic navigation at a rate of 1kbps.

10. The novel blind-guiding system based on image semantic segmentation network according to claim 1, characterized in that: The guide system also includes an external connection device, which is a monitor for monitoring human life and health information, and is used to obtain the human life and health information of blind users and connect the collected human life and health information to the main control module. The monitoring person accesses and views the human life and health information of the blind user through the terminal device.

Images