Local vibration direction indication method and system based on blind guiding intelligent glasses and medium

Abstract

The invention provides a local vibration direction indication method and system based on blind guiding intelligent glasses, and a medium. The method comprises the following steps: acquiring depth data and image data of a surrounding environment based on the blind guiding intelligent glasses; performing synchronous calibration and feature fusion on the depth data and the image data to obtain fusion data; analyzing the fusion data based on the identification model to obtain obstacle distribution information; generating a vibration control instruction based on a set vibration coding rule, and analyzing the direction of the obstacle according to the vibration control instruction to obtain direction indication information; transmitting the azimuth indication information to the terminal according to a preset mode; by analyzing the depth data and the image data of the surrounding environment in real time, the obstacles are accurately analyzed, so that the distribution and azimuth information of the obstacles is obtained, and the detection precision is improved.

Description

Technical Field

[0001] This application relates to the field of orientation indication technology for head-mounted devices, and more specifically, to an orientation indication method, system, and medium based on local vibration of smart glasses for the blind. Background Technology

[0002] The safety and convenience of independent travel for visually impaired individuals depend on the accurate perception and efficient transmission of environmental information by guide devices. While current mainstream guide technologies and devices have achieved a certain level of application, numerous technical bottlenecks and experiential deficiencies remain in practical use, making it difficult to meet the comprehensive and highly safe travel needs of visually impaired individuals. The specific situation and problems are as follows: Traditional mechanical guide devices for the visually impaired: limited functionality and significant limitations in perception. Mechanical guide canes, representing the most widely used basic tools for the visually impaired, work by having the tip of the cane contact the ground, relying on the user's tactile sense to perceive the flatness of the ground and the presence of obstacles. However, this type of device has significant limitations: Limited sensing range: It can only detect ground obstacles (such as bumps and potholes) within the reach of the cane. It cannot detect low-lying obstacles in the air (such as tree branches and hanging objects) or dynamic obstacles at a distance (such as moving vehicles and running pedestrians), resulting in serious safety blind spots and risks such as head collisions and being scratched by vehicles.

[0003] Highly dependent on operation: It requires users to hold and actively wave the detector continuously. When users are carrying items, going up and down stairs, or when their hands are injured, the convenience of use is greatly reduced. At the same time, the detection accuracy is highly dependent on the user's experience and tactile sensitivity, making it difficult for the elderly, visually impaired people, or people with mobility difficulties to operate accurately.

[0004] The information delivery is limited: it can only provide basic information such as "whether there is an obstacle", and cannot distinguish the type of obstacle (such as pedestrians and walls, steps and potholes). Users find it difficult to make scientific avoidance decisions, which greatly reduces travel safety. Summary of the Invention

[0005] The purpose of this application is to provide a method, system, and medium for local vibration orientation indication based on smart glasses for the blind. By analyzing the depth data and image data of the surrounding environment in real time, it can accurately analyze obstacles, thereby obtaining the distribution and orientation information of obstacles and improving detection accuracy.

[0006] This application also provides a method for indicating the orientation of local vibrations based on smart glasses for the blind, including: Based on navigation smart glasses, depth data and image data of the surrounding environment are collected; Simultaneous calibration and feature fusion of depth data and image data are performed to obtain fused data; Based on the analysis and fusion of data using the recognition model, obstacle distribution information is obtained; Vibration control commands are generated based on the set vibration coding rules. The location of obstacles is analyzed based on the vibration control commands to obtain location indication information. The directional information is transmitted to the terminal in a preset manner.

[0007] Optionally, in the orientation indication method based on local vibration of smart glasses for the blind described in the embodiments of this application, the acquisition of depth data and image data of the surrounding environment based on the smart glasses specifically includes: The depth data is based on the 3D ToF sensor and includes the straight-line distance between the object and the wearer, the spatial contour of the object's surface, the slope variation parameters of the ground, and the diameter and depth geometric dimensions of the pit. The spatial contour shape includes the height of the protrusion and the depth of the indentation; Image data is obtained by acquiring the texture features, contour features, color features of objects, as well as the motion trajectory and speed change trend of dynamic objects using image sensors.

[0008] The texture features include the texture of steps and the texture of pedestrian clothing; the contour features include the shape of vehicles and the shape of tree branches; and the color features include the color of traffic lights and the color of warning signs.

[0009] Optionally, in the orientation indication method for local vibration based on smart glasses for the blind described in this application embodiment, depth data and image data are synchronously calibrated and feature-fused to obtain fused data, specifically including: Depth data and image data are acquired, and time difference caused by response speed and data transmission link differences is eliminated based on the time alignment algorithm to obtain depth data and image data at the same time point; Features are extracted from depth data and image data at the same time point to obtain depth features and image features; The cosine similarity algorithm is used to calculate the matching degree between the contour boundary in the depth features and the contour feature point set in the image features, and to filter out the feature subsets pointing to the same obstacle in the two types of data. We obtain fused data by weighting and fusing the feature subsets pointing to the same obstacle from the two types of data.

[0010] Optionally, in the orientation indication method for local vibration based on smart glasses for the blind described in this application embodiment, obstacle distribution information is obtained by analyzing and fusing data based on a recognition model, specifically including: Initialize the recognition model by inputting the fused data into the initial recognition model and performing feature parsing. The results of feature parsing are matched with an obstacle feature database to calculate feature similarity. Obstacles are classified based on feature similarity, and a local spatial coordinate system is constructed based on the classification results and the three-dimensional coordinate information in the fused data. Map the three-dimensional coordinates of each obstacle to the local spatial coordinate system. Clustering algorithms are used to aggregate feature points of obstacles of the same category and adjacent locations to form the spatial bounding box of a single obstacle, thereby determining the specific location range of the obstacle in the coordinate system and obtaining the spatial positioning information of the obstacle. Obstacle distribution information is generated based on the spatial positioning information of obstacles.

[0011] Optionally, in the orientation indication method for local vibration based on smart glasses for the blind described in this application embodiment, vibration control commands are generated based on set vibration coding rules, and the orientation of obstacles is analyzed according to the vibration control commands to obtain orientation indication information, specifically including: After the equipment is started, the processing unit automatically loads the pre-stored multi-dimensional vibration coding rule library and generates vibration coding rules. The rule library is the unified basis for instruction generation and orientation analysis. Using obstacle distribution information obtained from fused data analysis as input, vibration control commands are generated based on vibration coding rules; Based on the vibration control command, the obstacle's orientation information is analyzed to obtain orientation indication information.

[0012] Optionally, in the orientation indication method based on local vibration of smart glasses for the blind described in this application embodiment, the orientation indication information is transmitted to the terminal in a preset manner, specifically including: Obtain the directional indication information, encrypt the directional indication information, and obtain ciphertext data; Generate the corresponding encryption key based on the ciphertext data; Encrypt the ciphertext data with the encryption key and transmit it to generate transmission data; A decryption key is generated based on the transmitted data and ciphertext data and matched with the encryption key; The encryption key, decryption key, and corresponding ciphertext data are transmitted together to the terminal.

[0013] Secondly, embodiments of this application provide a orientation indication system based on local vibration of smart guide glasses. The system includes a memory and a processor. The memory includes a program for an orientation indication method based on local vibration of smart guide glasses. When the program for the orientation indication method based on local vibration of smart guide glasses is executed by the processor, it implements the following steps: Based on navigation smart glasses, depth data and image data of the surrounding environment are collected; Simultaneous calibration and feature fusion of depth data and image data are performed to obtain fused data; Based on the analysis and fusion of data using the recognition model, obstacle distribution information is obtained; Vibration control commands are generated based on the set vibration coding rules. The location of obstacles is analyzed based on the vibration control commands to obtain location indication information. The directional information is transmitted to the terminal in a preset manner.

[0014] Optionally, in the orientation indication system based on local vibration of smart glasses for the blind described in this application embodiment, the acquisition of depth data and image data of the surrounding environment based on the smart glasses specifically includes: The depth data is based on the 3D ToF sensor and includes the straight-line distance between the object and the wearer, the spatial contour of the object's surface, the slope variation parameters of the ground, and the diameter and depth geometric dimensions of the pit. The spatial contour shape includes the height of the protrusion and the depth of the indentation; Image data is obtained by acquiring the texture features, contour features, color features of objects, as well as the motion trajectory and speed change trend of dynamic objects using image sensors.

[0015] The texture features include the texture of steps and the texture of pedestrian clothing; the contour features include the shape of vehicles and the shape of tree branches; and the color features include the color of traffic lights and the color of warning signs.

[0016] Optionally, in the orientation indication system based on local vibration of smart glasses for the blind described in this application embodiment, depth data and image data are synchronously calibrated and feature-fused to obtain fused data, specifically including: Depth data and image data are acquired, and time difference caused by response speed and data transmission link differences is eliminated based on the time alignment algorithm to obtain depth data and image data at the same time point; Features are extracted from depth data and image data at the same time point to obtain depth features and image features; The cosine similarity algorithm is used to calculate the matching degree between the contour boundary in the depth features and the contour feature point set in the image features, and to filter out the feature subsets pointing to the same obstacle in the two types of data. We obtain fused data by weighting and fusing the feature subsets pointing to the same obstacle from the two types of data.

[0017] Thirdly, embodiments of this application also provide a computer-readable storage medium, which includes a method program for orientation indication based on local vibration of smart glasses for the blind. When the method program for orientation indication based on local vibration of smart glasses for the blind is executed by a processor, it implements the steps of the orientation indication method for local vibration of smart glasses for the blind as described in any of the preceding claims.

[0018] As can be seen from the above, the present application provides a method, system, and medium for directional indication based on local vibration of smart glasses for the blind. This involves collecting depth and image data of the surrounding environment using smart glasses; synchronously calibrating and fusing the depth and image data to obtain fused data; analyzing the fused data based on a recognition model to obtain obstacle distribution information; generating vibration control commands based on set vibration coding rules; analyzing the obstacle orientation according to the vibration control commands to obtain directional indication information; and transmitting the directional indication information to a terminal in a preset manner. By analyzing the depth and image data of the surrounding environment in real time, the system accurately analyzes obstacles, thereby obtaining the distribution and orientation information of obstacles and improving detection accuracy. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for indicating the orientation of local vibrations based on smart glasses for the visually impaired, provided in an embodiment of this application; Figure 2 A flowchart illustrating the depth data and image data acquisition process for a local vibration orientation indication method based on smart glasses for the visually impaired, as provided in this application embodiment. Figure 3 A flowchart illustrating the fusion data acquisition method for a local vibration orientation indication method based on smart glasses for the visually impaired, provided in this application embodiment. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for determining the orientation of a visually impaired person based on local vibrations in some embodiments of this application. This method, used in a terminal device, includes the following steps: S101 collects depth data and image data of the surrounding environment based on navigation smart glasses; S102, synchronously calibrate and fuse depth data and image data to obtain fused data; S103, based on the identification model, analyze and fuse data to obtain obstacle distribution information; S104, generate vibration control commands based on the set vibration coding rules, analyze the obstacle's orientation according to the vibration control commands, and obtain orientation indication information; S105, transmits the directional information to the terminal in a preset manner.

[0024] It's worth noting that combining 3D Time-of-Flight (ToF) with an image sensor is beneficial. ToF provides accurate depth and pothole recognition, while the image sensor assists in object recognition (such as steps and pedestrians). The synergy between the two greatly improves reliability.

[0025] Integrating this complex sensor system onto a lightweight, non-invasive device like glasses offers inherent advantages in terms of wearability and user experience compared to handheld canes or bulky head-mounted devices.

[0026] Specifically, the depth data collected by the 3D ToF sensor includes the straight-line distance between various objects in the environment and the wearer, the spatial contour of the object surface (such as the height of protrusions and the depth of depressions), the slope change parameters of the ground, and the geometric dimension information such as the diameter and depth of pits, which can accurately construct a three-dimensional spatial model of the environment. Image data acquired by image sensors includes texture features of objects (such as the texture of steps and the texture of pedestrian clothing), contour features (such as the shape of vehicles and the shape of tree branches), color features (such as the color of traffic lights and the color of warning signs), as well as the motion trajectory and speed change trend of dynamic objects, providing core basis for object category recognition. The data from both sensors complement each other, effectively compensating for the recognition blind spots of a single sensor, significantly improving the reliability of recognition in complex environments, and reducing the risk of misjudgment.

[0027] Environmental data acquisition steps: The 3D ToF sensor and image sensor integrated into the smart glasses for the visually impaired are used to simultaneously acquire depth data and image data of the surrounding environment; wherein, the 3D ToF sensor and image sensor are both set on the temples or frames of the glasses, and the 3D ToF sensor is used to acquire distance and spatial shape data of obstacles in the environment, and the image sensor is used to acquire category feature data of objects in the environment. Data fusion processing steps: The depth data and image data are received by the processing unit built into the glasses. The two types of data are simultaneously calibrated and feature fused. Based on the preset recognition model, the specific category, spatial orientation and distance information of the obstacle to the wearer are identified. The feature fusion includes associating and matching the depth dimension features collected by the 3D ToF sensor with the texture and contour features collected by the image sensor. Vibration command generation step: The processing unit generates corresponding vibration control commands according to the identified obstacle information and preset vibration coding rules; the vibration coding rules include the correspondence between the obstacle orientation and the position of the glasses vibration source, the correspondence between the obstacle type and the vibration mode, and the correspondence between the obstacle distance and the vibration frequency. Local vibration alert step: The processing unit sends a vibration control command to the corresponding electronically controlled vibration source, and controls the electronically controlled vibration source to generate vibrations that conform to the vibration control command, so as to indicate relevant information about obstacles to the wearer; the electronically controlled vibration source is set in the part of the glasses that contacts the wearer's face, including the front and back sides of the temples, the nose pads and the brow area, and each electronically controlled vibration source is independently controlled.

[0028] Please refer to Figure 2 , Figure 2This is a flowchart illustrating the acquisition of depth and image data in a method for orientation indication based on local vibration of smart glasses for the visually impaired, as described in some embodiments of this application. According to embodiments of the present invention, the acquisition of depth and image data of the surrounding environment based on navigation smart glasses specifically includes: S201 is based on depth data collected by a 3D ToF sensor. The depth data includes the straight-line distance between the object and the wearer, the spatial contour of the object's surface, the slope variation parameters of the ground, and the diameter and depth geometric dimensions of the pit. S202, the spatial contour shape includes the height of the protrusion and the depth of the depression; S203: Based on the image sensor, the texture features, contour features, color features of the object, as well as the motion trajectory and speed change trend of the dynamic object, are collected to obtain image data; S204, texture features include the texture of steps and the texture of pedestrian clothing; outline features include the shape of vehicles and the shape of tree branches; color features include the color of traffic lights and the color of warning signs.

[0029] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating a fusion data acquisition method for a local vibration orientation indication method based on smart glasses for the blind, as described in some embodiments of this application. According to embodiments of the present invention, depth data and image data are synchronously calibrated and feature-fused to obtain fused data, specifically including: S301, acquire depth data and image data, and eliminate the time difference caused by response speed and data transmission link differences based on the time alignment algorithm to obtain depth data and image data at the same time node; S302, extract the features of depth data and image data at the same time point to obtain depth features and image features; S303 uses the cosine similarity algorithm to calculate the matching degree between the contour boundary in the depth features and the contour feature point set in the image features, and filters out the feature subsets pointing to the same obstacle in the two types of data. S304. Weighted fusion of feature subsets pointing to the same obstacle from the two types of data to obtain fused data.

[0030] It should be noted that, for scenarios where traditional guide devices are prone to miss such as irregular potholes and low tree branches, a dedicated recognition model has been developed. This model combines 3D ToF depth data with feature data from image sensors to achieve rapid detection and classification of special obstacles.

[0031] Multi-dimensional vibration coding mapping algorithm: Constructs a dynamic mapping logic between sensor data and vibration commands. It not only triggers the corresponding vibration source based on the orientation, but also distinguishes the obstacle type (pedestrian, vehicle, static obstacle) through vibration mode (long vibration, short vibration, continuous vibration, pulse vibration). It uses vibration frequency to feed back the obstacle distance, forming a tactile coding system for multi-dimensional information transmission.

[0032] Depth and Image Data Fusion Analysis Algorithm: As a core component of data processing, it requires a four-step process of "synchronous calibration - feature extraction - fusion association - data optimization" to generate accurate fused data, providing a reliable basis for obstacle recognition. The specific steps are as follows: 1. Synchronous Calibration: Eliminating Spatiotemporal Deviation - Time Synchronization: Relying on the device's built-in high-precision timestamp module, a unified millisecond-level time stamp is added to the depth data frames of the 3D ToF sensor and the image data frames of the image sensor. A time alignment algorithm eliminates the time difference caused by differences in the start-up response speed and data transmission links between the two types of sensors, ensuring accurate matching of environmental information at the same moment. - Spatial Calibration: Based on parameters obtained from previous laboratory calibration (including sensor installation angle, positional offset on the frame, optical distortion coefficient, etc.), a mapping relationship is established between the three-dimensional spatial coordinates of the depth data and the two-dimensional pixel coordinates of the image data. A coordinate transformation algorithm corrects installation errors, ensuring that the corresponding positions of the same physical obstacle in the two types of data completely coincide, avoiding fusion deviations caused by spatial misalignment.

[0033] Feature Extraction: Precisely Deconstructing Core Information - Depth Data Feature Extraction: Employing a region-growing-based segmentation algorithm, depth data is divided into different regions based on distance differences. Depth feature parameters of obstacles are extracted, including the average distance of the target region (reflecting its proximity to the wearer), distance variance (judging the surface smoothness of the obstacle, such as the difference between steps and flat ground), and depth abrupt change points at the contour boundaries (distinguishing the boundary between the obstacle and the background environment), forming a structured depth feature vector. - Image Data Feature Extraction: Using a lightweight convolutional neural network (such as the MobileNet model), visual features of objects are extracted from image data, including texture features (such as the difference between the texture of pedestrian clothing and the smooth surface of a wall), contour feature point sets (such as the rectangular contour of a vehicle and the irregular contour of tree branches), and color feature matrices (such as the red, yellow, and green features of traffic lights), generating a high-dimensional visual feature vector to provide a basis for object category recognition.

[0034] Fusion and Association: Constructing a Unified Feature Description - Feature Matching: Using the cosine similarity algorithm, the matching degree between the contour boundary in the depth features and the contour feature point set in the image features is calculated. This filters out feature subsets pointing to the same obstacle from both data types, eliminating environmental noise (such as image pseudo-features caused by lighting changes and depth interference caused by ground water stains). - Weighted Fusion: Dynamically assigning feature weights based on obstacle type—for terrain obstacles such as potholes and ground protrusions, depth features account for 70% (depth data is more accurate for terrain perception), and visual features account for 30% (to assist in verifying whether it is a passable area); for dynamic obstacles such as pedestrians and vehicles, visual features account for 60% (visual data is more reliable for object category recognition), and depth features account for 40% (to assist in judging movement trends and distance). A unified obstacle feature description vector is generated through a weighted summation algorithm, completing the feature fusion of the two data types.

[0035] Data Optimization: Outputting High-Quality Fusion Data - Noise Reduction: The fused feature description vector is input into a Kalman filter model to filter dynamic interference (such as feature jitter caused by rapid pedestrian movement, image noise caused by strong light and rain, and depth data fluctuations caused by reflections from obstacle surfaces), smoothing feature parameters. - Data Standardization: The optimized feature vector is converted according to a preset format to output standardized fusion data containing information such as obstacle category (e.g., pedestrian, pothole, vehicle), precise spatial orientation (3D coordinate positioning), real-time distance to the wearer, motion state (static / dynamic), and risk level (determined based on distance and movement speed), providing accurate input for the subsequent vibration command generation module.

[0036] According to an embodiment of the present invention, obstacle distribution information is obtained by analyzing and fusing data based on a recognition model, specifically including: Initialize the recognition model by inputting the fused data into the initial recognition model and performing feature parsing. The results of feature parsing are matched with an obstacle feature database to calculate feature similarity. Obstacles are classified based on feature similarity, and a local spatial coordinate system is constructed based on the classification results and the three-dimensional coordinate information in the fused data. Map the three-dimensional coordinates of each obstacle to the local spatial coordinate system. Clustering algorithms are used to aggregate feature points of obstacles of the same category and adjacent locations to form the spatial bounding box of a single obstacle, thereby determining the specific location range of the obstacle in the coordinate system and obtaining the spatial positioning information of the obstacle. Obstacle distribution information is generated based on the spatial positioning information of obstacles.

[0037] According to an embodiment of the present invention, vibration control commands are generated based on a set vibration coding rule, and the obstacle's orientation is analyzed according to the vibration control commands to obtain orientation indication information, specifically including: After the equipment is started, the processing unit automatically loads the pre-stored multi-dimensional vibration coding rule library and generates vibration coding rules. The rule library is the unified basis for instruction generation and orientation analysis. Using obstacle distribution information obtained from fused data analysis as input, vibration control commands are generated based on vibration coding rules; Based on the vibration control command, the obstacle's orientation information is analyzed to obtain orientation indication information.

[0038] According to an embodiment of the present invention, transmitting directional information to a terminal in a preset manner specifically includes: Obtain the directional indication information, encrypt the directional indication information, and obtain ciphertext data; Generate the corresponding encryption key based on the ciphertext data; Encrypt the ciphertext data with the encryption key and transmit it to generate transmission data; A decryption key is generated based on the transmitted data and ciphertext data and matched with the encryption key; The encryption key, decryption key, and corresponding ciphertext data are transmitted together to the terminal.

[0039] Secondly, embodiments of this application provide a orientation indication system based on local vibration of smart guide glasses. The system includes a memory and a processor. The memory includes a program for an orientation indication method based on local vibration of smart guide glasses. When the processor executes the program for the orientation indication method based on local vibration of smart guide glasses, it implements the following steps: Based on navigation smart glasses, depth data and image data of the surrounding environment are collected; Simultaneous calibration and feature fusion of depth data and image data are performed to obtain fused data; Based on the analysis and fusion of data using the recognition model, obstacle distribution information is obtained; Vibration control commands are generated based on the set vibration coding rules. The location of obstacles is analyzed based on the vibration control commands to obtain location indication information. The directional information is transmitted to the terminal in a preset manner.

[0040] According to an embodiment of the present invention, the acquisition of depth data and image data of the surrounding environment based on navigation smart glasses specifically includes: The depth data is based on the 3D ToF sensor and includes the straight-line distance between the object and the wearer, the spatial contour of the object's surface, the slope variation parameters of the ground, and the diameter and depth geometric dimensions of the pit. The spatial contour shape includes the height of the protrusion and the depth of the depression; Image data is obtained by acquiring the texture features, contour features, color features of objects, as well as the motion trajectory and speed change trend of dynamic objects using image sensors.

[0041] Texture features include the texture of steps and the texture of pedestrian clothing; outline features include the shape of vehicles and the shape of tree branches; color features include the color of traffic lights and the color of warning signs.

[0042] According to an embodiment of the present invention, depth data and image data are synchronously calibrated and feature-fused to obtain fused data, specifically including: Depth data and image data are acquired, and time difference caused by response speed and data transmission link differences is eliminated based on the time alignment algorithm to obtain depth data and image data at the same time point; Features are extracted from depth data and image data at the same time point to obtain depth features and image features; The cosine similarity algorithm is used to calculate the matching degree between the contour boundary in the depth features and the contour feature point set in the image features, and to filter out the feature subsets pointing to the same obstacle in the two types of data. We obtain fused data by weighting and fusing the feature subsets pointing to the same obstacle from the two types of data.

[0043] A third aspect of the present invention provides a computer-readable storage medium including a method program for orientation indication based on local vibration of a smart guide glasses. When the method program for orientation indication based on local vibration of a smart guide glasses is executed by a processor, it implements the steps of the orientation indication method for local vibration of a smart guide glasses as described in any of the above claims.

[0044] This invention discloses a method, system, and medium for directional indication based on local vibration of smart glasses for the blind. The method involves collecting depth and image data of the surrounding environment using smart glasses; synchronously calibrating and fusing the depth and image data to obtain fused data; analyzing the fused data based on a recognition model to obtain obstacle distribution information; generating vibration control commands based on predefined vibration coding rules; analyzing obstacle orientation based on the vibration control commands to obtain directional indication information; and transmitting the directional indication information to a terminal according to a preset method. By analyzing the depth and image data of the surrounding environment in real time, the method accurately analyzes obstacles, thereby obtaining obstacle distribution and orientation information and improving detection accuracy.

[0045] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0046] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0047] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0048] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0049] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for orientation indication based on local vibration of smart guide glasses, characterized in that, include: Based on navigation smart glasses, depth data and image data of the surrounding environment are collected; Simultaneous calibration and feature fusion of depth data and image data are performed to obtain fused data; Based on the analysis and fusion of data using the recognition model, obstacle distribution information is obtained; Vibration control commands are generated based on the set vibration coding rules. The location of obstacles is analyzed based on the vibration control commands to obtain location indication information. The directional information is transmitted to the terminal in a preset manner.

2. The method for orientation indication based on local vibration of smart glasses for the blind according to claim 1, characterized in that, Based on navigation smart glasses, depth and image data of the surrounding environment are collected, specifically including: The depth data is based on the 3D ToF sensor and includes the straight-line distance between the object and the wearer, the spatial contour of the object's surface, the slope variation parameters of the ground, and the diameter and depth geometric dimensions of the pit. The spatial contour shape includes the height of the protrusion and the depth of the indentation; Image data is obtained by acquiring the texture features, contour features, color features of objects, as well as the motion trajectory and speed change trend of dynamic objects using image sensors. The texture features include the texture of steps and the texture of pedestrian clothing; the contour features include the shape of vehicles and the shape of tree branches; and the color features include the color of traffic lights and the color of warning signs.

3. The method for orientation indication based on local vibration of smart glasses for the blind according to claim 2, characterized in that, Simultaneous calibration and feature fusion of depth data and image data are performed to obtain fused data, specifically including: Depth data and image data are acquired, and time difference caused by response speed and data transmission link differences is eliminated based on the time alignment algorithm to obtain depth data and image data at the same time point; Features are extracted from depth data and image data at the same time point to obtain depth features and image features; The cosine similarity algorithm is used to calculate the matching degree between the contour boundary in the depth features and the contour feature point set in the image features, and to filter out the feature subsets pointing to the same obstacle in the two types of data. We obtain fused data by weighting and fusing the feature subsets pointing to the same obstacle from the two types of data.

4. The method for orientation indication based on local vibration of smart glasses for the blind according to claim 3, characterized in that, Based on the analysis and fusion of data using the recognition model, obstacle distribution information is obtained, specifically including: Initialize the recognition model by inputting the fused data into the initial recognition model and performing feature parsing. The results of feature parsing are matched with an obstacle feature database to calculate feature similarity. Obstacles are classified based on feature similarity, and a local spatial coordinate system is constructed based on the classification results and the three-dimensional coordinate information in the fused data. Map the three-dimensional coordinates of each obstacle to the local spatial coordinate system. Clustering algorithms are used to aggregate feature points of obstacles of the same category and adjacent locations to form the spatial bounding box of a single obstacle, thereby determining the specific location range of the obstacle in the coordinate system and obtaining the spatial positioning information of the obstacle. Obstacle distribution information is generated based on the spatial positioning information of obstacles.

5. The method for orientation indication based on local vibration of smart glasses for the blind according to claim 4, characterized in that, Vibration control commands are generated based on the established vibration coding rules. The obstacle's orientation is analyzed based on the vibration control commands to obtain orientation indication information, specifically including: After the equipment is started, the processing unit automatically loads the pre-stored multi-dimensional vibration coding rule library and generates vibration coding rules. The rule library is the unified basis for instruction generation and orientation analysis. Using obstacle distribution information obtained from fused data analysis as input, vibration control commands are generated based on vibration coding rules; Based on the vibration control command, the obstacle's orientation information is analyzed to obtain orientation indication information.

6. The method for orientation indication based on local vibration of smart glasses for the blind according to claim 5, characterized in that, The directional information is transmitted to the terminal in a preset manner, specifically including: Obtain the directional indication information, encrypt the directional indication information, and obtain ciphertext data; Generate the corresponding encryption key based on the ciphertext data; Encrypt the ciphertext data with the encryption key and transmit it to generate transmission data; A decryption key is generated based on the transmitted data and ciphertext data and matched with the encryption key; The encryption key, decryption key, and corresponding ciphertext data are transmitted together to the terminal.

7. A orientation indication system based on local vibration of smart guide glasses, characterized in that, The system includes a memory and a processor. The memory contains a program for a location indication method based on local vibrations of smart guide glasses. When the processor executes the program for the location indication method based on local vibrations of smart guide glasses, it performs the following steps: Based on navigation smart glasses, depth data and image data of the surrounding environment are collected; Simultaneous calibration and feature fusion of depth data and image data are performed to obtain fused data; Based on the analysis and fusion of data using the recognition model, obstacle distribution information is obtained; Vibration control commands are generated based on the set vibration coding rules. The location of obstacles is analyzed based on the vibration control commands to obtain location indication information. The directional information is transmitted to the terminal in a preset manner.

8. The orientation indication system based on local vibration of smart glasses for the blind according to claim 7, characterized in that, Based on navigation smart glasses, depth and image data of the surrounding environment are collected, specifically including: The depth data is based on the 3D ToF sensor and includes the straight-line distance between the object and the wearer, the spatial contour of the object's surface, the slope variation parameters of the ground, and the diameter and depth geometric dimensions of the pit. The spatial contour shape includes the height of the protrusion and the depth of the indentation; Image data is obtained by acquiring the texture features, contour features, color features of objects, as well as the motion trajectory and speed change trend of dynamic objects using image sensors. The texture features include the texture of steps and the texture of pedestrian clothing; the contour features include the shape of vehicles and the shape of tree branches; and the color features include the color of traffic lights and the color of warning signs.

9. The orientation indication system based on local vibration of smart glasses for the blind according to claim 8, characterized in that, Simultaneous calibration and feature fusion of depth data and image data are performed to obtain fused data, specifically including: Depth data and image data are acquired, and time difference caused by response speed and data transmission link differences is eliminated based on the time alignment algorithm to obtain depth data and image data at the same time point; Features are extracted from depth data and image data at the same time point to obtain depth features and image features; The cosine similarity algorithm is used to calculate the matching degree between the contour boundary in the depth features and the contour feature point set in the image features, and to filter out the feature subsets pointing to the same obstacle in the two types of data. We obtain fused data by weighting and fusing the feature subsets pointing to the same obstacle from the two types of data.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a method program for orientation indication based on local vibration of smart glasses for the blind. When the method program for orientation indication based on local vibration of smart glasses for the blind is executed by a processor, it implements the steps of the method for orientation indication based on local vibration of smart glasses for the blind as described in any one of claims 1 to 6.

Images