Switching control system and method based on blind people and medium

Abstract

The invention provides a switching control system and method based on the blind and a medium, and is applied to an intelligent glasses terminal, the method comprises the following steps: the intelligent glasses terminal constructs a computing power hierarchical structure comprising at least a first computing power layer and a second computing power layer with the computing power higher than that of the first computing power layer, and a user wears the intelligent glasses terminal, the intelligent glasses terminal collects external environment information and generates to-be-processed data. The external environment information at least comprises two different types of environment information. And identifying to-be-processed data, and determining the environmental information type to which the to-be-processed data belongs. And according to a preset environment type and a computing power mapping relationship, allocating different types of to-be-processed data to corresponding computing power layers for processing so as to obtain a processing result of each computing power layer. And the intelligent glasses terminal generates an output result for blind person navigation based on the processing result of each computing power layer, and outputs the output result to a user in a blind person perceptible manner. Through the method, the real-time performance is improved.

Description

Technical Field

[0001] This application relates to the field of navigation for the blind, and more particularly to a switching control system, method, and medium for the blind. Background Technology

[0002] Navigation systems for the blind are key intelligent devices that assist visually impaired individuals in traveling independently and safely. Their core function is to perceive external environmental information and provide users with real-time, accurate navigation prompts. External environmental information includes road images, obstacle outlines, traffic signals, and surrounding sounds.

[0003] Currently, existing navigation systems for the blind generally suffer from limitations in their computing architecture. To cope with complex computational tasks, many systems tend to adopt a centralized, single computing platform. For example, all collected environmental data is invariably transmitted to a cloud server for unified processing. While this architecture can utilize the powerful computing capabilities of the cloud, it also brings significant drawbacks: First, transmitting all data back to the cloud incurs huge network transmission overhead. When network bandwidth is unstable, the transmission of large volumes of data, such as images, can introduce severe delays, leading to lag in navigation prompts. Second, the cloud, as a single processing node, can easily become a system bottleneck. When processing tasks are heavy, the overall response time increases dramatically, failing to meet the stringent requirements of blind navigation for instantaneous responses, such as emergency obstacle avoidance.

[0004] Therefore, there is a need for a real-time switching control system, method, and medium for blind users. Summary of the Invention

[0005] In view of this, it is necessary to provide a control system, method and medium for blind people to improve real-time performance in order to solve the above problems.

[0006] Embodiments of this application provide a blind-based switching control method applied to a smart glasses terminal, the method comprising the following steps: The smart glasses terminal is constructed with a computing power hierarchy structure including at least a first computing power layer and a second computing power layer, wherein the computing power of the first computing power layer is lower than that of the second computing power layer; The user wears smart glasses, which collect external environmental information and generate data to be processed. The external environmental information includes at least two different types of environmental information. The smart glasses terminal identifies the data to be processed, determines the environmental information type to which the data belongs, and allocates different types of data to the corresponding computing power layer for processing according to the preset environmental type and computing power mapping relationship, so as to obtain the processing results of each computing power layer. The smart glasses terminal generates output results for navigation for the blind based on the processing results of each computing layer, and outputs the output results to the user in a way that the blind can perceive.

[0007] In at least one embodiment of this application, the external environment information includes at least two types of environmental information: voice information and image information; When the data to be processed is identified as voice information, the data to be processed is allocated to the first computing layer for processing. When the data to be processed is identified as image information, the data to be processed is assigned to the second computing layer for processing.

[0008] In at least one embodiment of this application, the method steps of "smart glasses terminal identifying data to be processed and determining the environmental information type to which the data to be processed belongs" include: The source sensor type of the data to be processed is determined based on either or a combination of the acquisition channel and data format of the data to be processed. The data to be processed originating from the microphone channel is identified as a voice information type; The data to be processed from the camera channel is identified as image information type.

[0009] In at least one embodiment of this application, the method for "constructing a computing power hierarchy structure including at least a first computing power layer and a second computing power layer for a smart glasses terminal" includes the following steps: The first computing power layer is a cloud computing power layer that is connected to the smart glasses terminal via a wireless network; The second computing layer is the computing layer inside the wired control box that is connected to the smart glasses terminal via a wired connection.

[0010] In at least one embodiment of this application, the "preset environment type and computing power mapping relationship" method further includes the step of: Obtain at least one system operation indicator, including network bandwidth status and load status of each computing layer; When the system operation indicators meet the preset adjustment conditions, the environment type and computing power mapping relationship are updated to dynamically adjust the computing power layer corresponding to different environment information types.

[0011] In at least one embodiment of this application, the method of "allocating different types of data to be processed to corresponding computing power layers for processing according to preset environment types and computing power mapping relationships, so as to obtain the processing results of each computing power layer" includes the following steps: When switching the data to be processed corresponding to an environmental information type from one computing layer to another; The smart glasses terminal caches the data to be processed that is continuously collected before and after the switch; After the computing power layer switch is completed, the cached data to be processed is sent to the switched computing power layer for processing according to the collection order.

[0012] In at least one embodiment of this application, the method steps of "generating output results for blind navigation based on the processing results of each computing layer" include: Obtain the speech processing results output by the first computing layer and the image processing results output by the second computing layer; Under a preset time delay constraint, feature fusion is performed on the speech processing result and the image processing result; Generate navigation output that includes both voice prompts and image perception information.

[0013] In at least one embodiment of this application, the feature fusion includes: Set the weight parameters for voice processing results and image processing results according to the real-time requirements of the navigation task; When the current processing delay is detected to be close to the preset delay limit, the weight of the voice processing result in the navigation output result is increased, so as to prioritize the real-time output of voice navigation information when processing resources are limited or delay pressure increases.

[0014] A blind-based handover control system is applied to any of the blind-based handover control methods described above.

[0015] A blind-person switching medium is provided, applicable to any of the aforementioned blind-person switching control methods. The blind-person switching control system, method, and medium provided above achieve targeted allocation of computing resources by matching information types to different computing power layers. This reduces overall latency while ensuring the processing capacity for complex tasks, thereby improving the real-time performance and stability of blind navigation. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of a blind-based switching control method described in this application. Detailed Implementation

[0017] The embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0018] It should be noted that when a component is considered to be "connected" to another component, it can be directly connected to the other component or may also have an intervening component. When a component is considered to be "placed" on another component, it can be directly placed on the other component or may also have an intervening component. The terms "top," "bottom," "upper," "lower," "left," "right," "front," "back," and similar expressions used in this article are for illustrative purposes only.

[0019] Example 1:

[0020] Embodiments of this application provide a blind-based switching control method applied to a smart glasses terminal, the method comprising the following steps: S10: The smart glasses terminal is constructed with a computing power hierarchy structure including at least a first computing power layer and a second computing power layer. The computing power of the first computing power layer is lower than that of the second computing power layer.

[0021] S20: The user wears smart glasses, which collect external environmental information and generate data to be processed. The external environmental information includes at least two different types of environmental information.

[0022] S30: The smart glasses terminal identifies the data to be processed and determines the environmental information type to which the data belongs. Based on a preset environmental type and computing power mapping relationship, it assigns different types of data to the corresponding computing power layers for processing to obtain the processing results of each computing power layer.

[0023] S40: The smart glasses terminal generates output results for navigation for the blind based on the processing results of each computing layer, and outputs the output results to the user in a way that the blind can perceive.

[0024] The aforementioned blind-person switching control system, method, and medium improve the processing efficiency and real-time performance of blind navigation by switching and hierarchically processing different types of external environmental information between different computing power layers.

[0025] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0026] Please see Figure 1 This application provides a blind-based switching control method applied to a smart glasses terminal, the method comprising the following steps: S10: The smart glasses terminal is constructed with a computing power hierarchy structure including at least a first computing power layer and a second computing power layer. The computing power of the first computing power layer is lower than that of the second computing power layer.

[0027] S20: The user wears smart glasses, which collect external environmental information and generate data to be processed. The external environmental information includes at least two different types of environmental information.

[0028] S30: The smart glasses terminal identifies the data to be processed and determines the type of environmental information to which the data to be processed belongs.

[0029] S40: Based on the preset environment type and computing power mapping relationship, different types of data to be processed are assigned to the corresponding computing power layers for processing, so as to obtain the processing results of each computing power layer.

[0030] S50: The smart glasses terminal generates output results for navigation for the blind based on the processing results of each computing layer, and outputs the output results to the user in a way that the blind can perceive.

[0031] It should be noted that this method is applied to smart glasses terminals. As wearable sensing and interaction devices, smart glasses terminals integrate cameras, microphones, speakers, vibration feedback units, processors, memory, and communication interfaces for data interaction with external computing nodes. The system is layered according to computing power, with at least a first computing layer and a second computing layer. The first computing layer is used to quickly complete latency-sensitive tasks, such as speech understanding and simple scene recognition. The second computing layer is used to complete computationally intensive tasks, such as high-precision obstacle detection, environmental semantic understanding, and multimodal reasoning. The first computing layer can be located in a mobile terminal or cloud computing node connected wirelessly to the smart glasses terminal. The second computing layer is located in a control box connected to the smart glasses terminal via a wired connection.

[0032] In step S10, it should be noted that the smart glasses terminal establishes a computing power hierarchy structure during the system initialization phase, including at least a first computing power layer and a second computing power layer, with the computing power of the first computing power layer being lower than that of the second computing power layer. The computing power hierarchy structure includes: a hierarchy identifier for each computing power layer, the types of models that can run at that layer, the communication path connected to that layer, parameters for task scheduling, and a mapping table for the correspondence between types and computing power.

[0033] The first computing layer is used for fast processing, pre-configured with lightweight speech recognition and image recognition models. The second computing layer is used for high-precision processing, pre-configured with high-precision image target recognition models, environmental semantic understanding models, and multimodal reasoning models.

[0034] The first computing layer can be deployed on a mobile terminal that is wirelessly connected to the smart glasses terminal, or on a cloud computing node. In another embodiment, the first computing layer is deployed on both the mobile terminal and the cloud computing node to achieve collaborative processing. The second computing layer is deployed in a control box that is wiredly connected to the smart glasses terminal.

[0035] To support subsequent allocation and switching, the smart glasses terminal completes the handshake with each computing layer in this step, loads the corresponding model, and establishes two buffer queues, which correspond to the wireless connection path and the wired connection path, respectively. At the same time, it establishes a synchronization rule based on timestamps.

[0036] In step S20, it should be noted that after the user wears the smart glasses terminal, the terminal continuously collects external environmental information and generates data to be processed. The external environmental information includes at least two different types of information, specifically voice information and image information.

[0037] Image information is captured by the camera at a set frame rate. After noise suppression, exposure correction, and distortion correction by the image signal processing module, image data with timestamps is generated. Bitrate adaptation and frame rate adaptation are then performed according to the current bandwidth and power consumption strategy. Voice information is captured by the microphone and generated into voice data frames after endpoint detection and noise suppression.

[0038] All data to be processed is written to a circular buffer, along with the source channel identifier, data format identifier, acquisition time, and sequence number, to facilitate tracking and alignment in subsequent steps.

[0039] In step S30, it should be noted that the smart glasses terminal performs type identification on the data to be processed to determine its corresponding environmental information type. Type identification is primarily based on the acquisition channel and data format. Data originating from the microphone channel and conforming to the audio encoding format is identified as voice information. Data originating from the camera channel and conforming to the image or video encoding format is identified as image information.

[0040] In cases where there is ambiguity in the data acquisition channel or data format, the smart glasses terminal invokes a lightweight type discriminator for secondary determination, and outputs the final type label by combining a threshold and confidence level. Simultaneously, it records the proportion and arrival rate of each type of data within a short time window for subsequent adaptive adjustments to the mapping relationship.

[0041] In step S40, it should be noted that the smart glasses terminal allocates different types of data to be processed to the corresponding computing power layer for processing based on the preset "environmental information type - computing power layer" mapping relationship, and obtains the processing results of each computing power layer.

[0042] In one implementation, voice information is assigned to the first computing layer for rapid parsing and instruction understanding, while image information is assigned to the second computing layer for obstacle recognition, path analysis, and environmental semantic understanding. The voice processing and image processing processes are executed in parallel to reduce the cumulative latency caused by sequential execution.

[0043] To adapt to changes in operating conditions, the smart glasses terminal dynamically adjusts the mapping relationship based on the current load of each computing layer, the average processing latency of each computing layer, the signal quality of the wireless connection, and the stability of the wired connection. When wireless connection jitter occurs or the latency of the first computing layer approaches the threshold, the smart glasses terminal reduces the image acquisition frame rate, increases the priority of voice tasks, or migrates some image tasks to the second computing layer. When the load on the second computing layer is high, the smart glasses terminal executes low-complexity image tasks in a lightweight model on the first computing layer.

[0044] When migrating tasks between computing power layers, the smart glasses terminal uses two buffer queues and a sequential output mechanism. Data collected continuously before and after the switch is cached. Once the target computing power layer is ready, the cached data is sent to the target computing power layer for processing according to the collection order, to avoid processing gaps and out-of-order results.

[0045] This step allows for the division of labor between latency-sensitive voice links and computationally intensive image links, processing them on different computing layers and independently transmitting the results in chronological order. On one hand, the first computing layer rapidly parses the voice task, reducing the response time from voice to command. On the other hand, the second computing layer performs high-precision recognition of the image task, ensuring reliable identification of obstacles and paths. Through type classification, layered allocation, and sequential transmission, the cumulative latency caused by serial waiting is reduced overall, improving the continuity and stability of navigation prompts, thereby enhancing the real-time performance and safety of navigation for the blind while maintaining recognition accuracy.

[0046] In step S50, it should be noted that the smart glasses terminal generates output results for navigation for the blind based on the processing results returned by each computing layer, and outputs them in a way that the blind can perceive.

[0047] First, the smart glasses terminal performs time alignment and result filtering. The voice parsing results from the first computing layer and the image recognition results from the second computing layer are aligned using a unified timeline, and information related to obstacles, road boundaries, pedestrians, zebra crossings, and traffic signals is prioritized and retained according to preset safety rules. Subsequently, the smart glasses terminal performs feature fusion and policy weighting, and generates navigation commands while meeting time latency constraints.

[0048] The output methods include voice prompts and tactile vibrations. Under normal circumstances, both can be output simultaneously. When processing resources are limited or latency pressure increases, the system increases the weight of voice prompts in navigation commands to ensure timely delivery of key prompts.

[0049] To achieve closed-loop optimization, the smart glasses terminal records the output navigation results, including the output timestamp and the computing power layer information used. The record is used for subsequent adaptive scheduling and performance evaluation.

[0050] The above methods and steps enable the reasonable division of labor and parallel processing of latency-sensitive voice tasks and computationally intensive image tasks across different computing layers without altering the user's usage patterns. Through mechanisms such as type recognition, mapping allocation, dynamic scheduling, double buffer switching, and latency constraint fusion, end-to-end response time is reduced, interruptions during computing layer switching are minimized, and the continuity and stability of navigation output are improved. Furthermore, in complex scenarios, both recognition accuracy and energy consumption control are considered, thereby enhancing the overall real-time performance and safety of navigation for the blind.

[0051] In one specific embodiment, the S30 method step includes: S31: Determine the source sensor type of the data to be processed based on either or a combination of the acquisition channel and data format of the data to be processed.

[0052] In this step, it's important to note that the smart glasses terminal attaches metadata to each piece of data to be processed, including at least the acquisition channel identifier, data format identifier, acquisition time, and serial number. The acquisition channel identifier distinguishes the physical or logical channel from which the data originates, such as a microphone channel or a camera channel. The data format identifier indicates encoding and encapsulation attributes, such as audio frame format or image frame format. The terminal first determines the source sensor type based on the acquisition channel. When the acquisition channel identifier is missing or inconsistent with the data format, a determination based on the data format is used. When both the acquisition channel and data format are valid and consistent, the consistent result is used as the source sensor type. When a conflict exists, the terminal triggers a conflict handling process. The data format determination result is prioritized, while a lightweight discrimination program is invoked to quickly verify the frame content. Only after successful verification is the source sensor type confirmed, and the conflict event is recorded in the log for subsequent statistics and optimization. Through the above sequence and conflict handling rules, the determination of the source sensor type has a clear and verifiable path.

[0053] When the acquisition channel and data format are inconsistent, the "data format" is prioritized for determination because the data format is an intrinsic characteristic of the data itself, possessing verifiable header information and structural parameters, and is less affected by driver switching, routing errors, or multi-source multiplexing. In contrast, channel identifiers are external labels, easily distorted by hot-plugging, sleep / wake-up, or thread multiplexing. Lightweight format validation can quickly confirm the data type; lightweight validation includes checking frame headers, boundaries, and key parameters. This prevents voice data from being mistakenly sent to high-computing-power image pipelines or image data from being mistakenly sent to low-computing-power voice pipelines, thereby reducing cascading latency and security risks caused by misallocation. Simultaneously, this strategy facilitates the recording and tracing of conflict events, supporting subsequent optimization of thresholds and scheduling strategies, and better meeting the real-time and stability requirements of navigation for the blind.

[0054] S32: Identify the data to be processed from the microphone channel as a voice information type.

[0055] Once the source sensor type is identified as a microphone, the terminal labels the data as speech information and enters the speech preprocessing pipeline. Preprocessing includes endpoint detection, noise suppression, and feature extraction, while maintaining the original acquisition time and sequence number to ensure time alignment with subsequent image results. To reduce latency, speech data enters the speech task queue of the first computing layer in fixed-length frames, employing a short-window incremental transmission strategy. This allows upper-layer inference to receive and parse data simultaneously, shortening the latency from acquisition to output. If non-speech pulses or abnormal silence segments are detected within a speech frame, the terminal marks it without changing the type label, allowing upper-layer modules to ignore or downgrade it as needed.

[0056] S33: Identify the data to be processed from the camera channel as an image information type.

[0057] Once the source sensor type is identified as a camera, the terminal labels the data as image information and enters the image preprocessing pipeline. Preprocessing includes noise suppression, exposure correction, and geometric correction. Then, based on the current bandwidth and power consumption strategy, bitrate and frame rate are adaptively adjusted to ensure stable data transmission under given resources. Each frame retains its acquisition time and sequence number and is written to the image task queue of the second computing layer. To ensure subsequent fusion accuracy, scene prompts, such as brightness level or motion blur degree, can be added during the preprocessing stage for upper-layer model adjustments to threshold and confidence strategies. If a corrupted or severely distorted frame is detected, the terminal marks or discards the frame and records the event for statistical analysis and alerting.

[0058] On the one hand, voice data is promptly labeled and incrementally fed into a low-latency voice processing link, thereby shortening the time from voice acquisition to navigation command generation. On the other hand, image data, under the premise of controlled quality and time traceability, enters a high-computing-power image processing link, ensuring the reliability of obstacle recognition and environmental semantic analysis. By determining the type of source sensor and consistent time labeling, the system can reduce misallocation and misaligned fusion during subsequent hierarchical allocation and multi-source fusion, reducing overall end-to-end latency and improving the continuity and stability of navigation output, thereby enhancing the real-time performance and safety of navigation for the blind.

[0059] In one specific embodiment, the method for "preset environment type and computing power mapping relationship" further includes the following steps: S41: Obtain at least one system operation indicator. The system operation indicator includes network bandwidth status and load status of each computing layer.

[0060] In this step, it's important to note that the smart glasses terminal periodically collects and maintains a set of quantifiable operational metrics to reflect the real-time status and trends of the communication link and computing layer processing capabilities. Network bandwidth status includes at least the following quantities: wireless signal strength, link throughput, round-trip latency, packet loss rate, and jitter. For wired paths, physical link speed and error frame counts are further recorded. The load status of each computing layer includes at least the following quantities: processor or accelerator utilization, task queue length, average inference latency, memory usage, and temperature status. To reduce misjudgments caused by instantaneous fluctuations, the terminal uses a fixed sampling period for data collection and calculates the mean and variance using a sliding time window. A threshold hysteresis strategy is used to set upper and lower limits to avoid frequent jitter when metrics are critical. All raw measurements and calculated smoothed values ​​are archived with timestamps to provide a basis for subsequent decision-making and backtracking.

[0061] S42: When the system operation indicators meet the preset adjustment conditions, update the environment type and computing power mapping relationship to dynamically adjust the computing power layer corresponding to different environment information types.

[0062] The preset adjustment conditions refer to the trigger rules and protection intervals set for the above-mentioned operating indicators. One scenario includes: wireless link throughput falling below the set lower limit and persisting for a set duration; wireless round-trip latency exceeding the set upper limit; packet loss rate or jitter exceeding the set threshold; average inference latency or queue length of the first computing layer exceeding the set threshold; and the second computing layer experiencing overheating and frequency throttling or persistently high occupancy. Upon meeting any trigger rule, the terminal updates the mapping relationship between "environmental information type—computing layer" according to a priority strategy. Specific measures include: When network bandwidth is insufficient, image-related tasks are preferentially migrated to the second computing layer for processing, and the image resolution or frame rate is reduced simultaneously.

[0063] When the first computing layer is overloaded, reduce the proportion of image tasks it carries and retain only low-latency tasks such as speech parsing.

[0064] When the second computing layer is overloaded or the temperature is too high, low-complexity image tasks will be temporarily switched to the first computing layer to be processed by a lightweight model, while non-safe tasks will be downgraded or postponed.

[0065] When multiple conditions are triggered simultaneously, priority should be given to ensuring processing resources for safety-related tasks such as obstacle detection and route warning.

[0066] To ensure stability, a cooling-off period is implemented after the mapping relationship is updated, during which no new adjustments are triggered. When the operating metrics recover and consistently exceed the recovery threshold, the system reverts to the normal mapping according to the recovery strategy. The entire process is coordinated with buffering and sequential transmission mechanisms to ensure data continuity and time alignment when tasks migrate between computing layers.

[0067] On the one hand, it avoids continuously feeding large amounts of data into links with poor performance or overloaded computing layers, reducing end-to-end latency and frame loss risks. On the other hand, through task degradation, priority management, and cooldown period strategies, it suppresses oscillations caused by frequent switching, ensuring continuous output of safety tasks such as obstacle detection and route warnings. This achieves synergistic optimization of real-time performance and stability, enabling blind navigation to maintain stable and timely navigation prompts even in complex environments and resource fluctuation scenarios.

[0068] In one specific embodiment, the method of "allocating different types of data to be processed to corresponding computing power layers for processing according to preset environment types and computing power mapping relationships, so as to obtain the processing results of each computing power layer" includes the following steps: S43: When switching the data to be processed corresponding to an environmental information type from one computing layer to another.

[0069] In this step, it's important to note that the handover is triggered by a mapping update. Triggering conditions include decreased wireless bandwidth, increased packet loss rate, average latency exceeding a threshold in the first computing layer, and overload or temperature exceeding limits in the second computing layer. Upon entering the handover process, the smart glasses terminal first sets a "pause allocation" flag for the environmental information type, stopping the allocation of new data frames to the original computing layer and marking the current frame sequence number with a handover marker. For data already queued or being computed in the original computing layer, the terminal records its sequence number and status, allowing for short-term termination or cancellation if rollback is possible to avoid out-of-order results. Subsequently, a ready handshake is completed with the target computing layer, including model hot-start, memory and queue capacity checks, and time base alignment. After the handshake, the terminal enters the "cache aggregation" phase to prepare for subsequent sequential connection.

[0070] S44: The smart glasses terminal caches the data to be processed that is continuously collected before and after the switch.

[0071] In this step, it's important to note that to ensure uninterrupted processing during the handover, the smart glasses terminal establishes a continuous cache window for this type of environmental information. The cache stores data collected before and after the handover marker, organized in a circular queue, and records the sequence number, acquisition time, source channel, format verification result, and integrity verification value for each data entry. The cache capacity is configured based on the target latency and peak handover duration to cover the handover handshake and target layer warm-up time. To avoid cache overflow, the terminal enables backpressure and load reduction strategies, such as reducing the image frame rate or resolution, while discarding non-critical frames according to a strategy, prioritizing the retention of security-related frames and frames at critical time points. During the caching phase, no new data of this type is sent to any computing layer; only the order and integrity are accumulated and maintained.

[0072] S45: After completing the switching of the computing power layer, the cached data to be processed is sent to the switched computing power layer for processing according to the collection order.

[0073] In this step, it's important to note that once the target computing layer enters the ready state, the terminal replays the cached data in the acquisition order, starting from the next sequence number after the switching marker, ensuring the timeline is continuous and the sequence number is monotonic. If some data has already produced valid results in the original computing layer, the terminal deduplicates the data according to the result registration table and does not resend it. If a corrupted or incomplete frame is found, it is skipped and recorded in the log to avoid blocking the replay. To reduce startup jitter, the terminal can first send a small batch of data for queue warm-up before entering a stable rhythm. Throughout the replay process, the latency and queue depth of the target computing layer are continuously monitored. If necessary, the replay rate is dynamically reduced or the load reduction strategy is continued until the cache is cleared and the real-time stream takeover is activated.

[0074] By stopping new allocations and recording the switching boundary before switching, out-of-order results and duplicate calculations are avoided. On the other hand, by covering the switching duration buffer window and replaying in the order of acquisition, processing gaps are eliminated and the timeline remains continuous. Simultaneously, backpressure and load reduction strategies are used to control peak congestion and startup jitter, making end-to-end latency controllable during switching. Navigation voice and obstacle recognition prompts remain continuous and stable, thus significantly reducing the adverse impact of switching on the real-time performance and safety of navigation for the blind.

[0075] In one specific embodiment, the S50 method steps include: S51: Obtain the speech processing results output by the first computing layer and the image processing results output by the second computing layer.

[0076] In this step, it should be noted that the smart glasses terminal sets the result aggregator to subscribe to the return channels of the first and second computing layers respectively. The voice processing results returned by the first computing layer include: command intent, command confidence, timestamp, and input frame sequence number. The image processing results returned by the second computing layer include: obstacle category, location and relative distance, passable area, scene label, result confidence, timestamp, and input frame sequence number.

[0077] The results aggregator registers both types of results using a unified time base and establishes a results registration form for deduplication and tracking. Establishing the results registration form includes: For duplicate results with the same input frame number, only the latest and complete copy is retained.

[0078] When frame loss or latency exceeding limits is detected, the event is recorded and a degradation process is initiated.

[0079] To limit the waiting time, the aggregator sets a receive timeout threshold for each fusion cycle.

[0080] If one side does not return a valid result within the timeout threshold, the subsequent processing continues with the result that has arrived on the other side, ensuring that the output link is not blocked.

[0081] S52: Under the preset time delay constraint, perform feature fusion on the speech processing result and the image processing result.

[0082] In this step, it should be noted that the preset time delay constraint is the maximum allowable time from the start of the current fusion cycle to the generation of the output, denoted as T. The result aggregator establishes a sliding fusion window for adjacent results on the same time axis, with the window width not exceeding the available budget of T. If both speech and image results are obtained simultaneously within the window, decision-level fusion is performed. Weighted scores are calculated based on their respective confidence levels, result freshness, and task priorities to form a consistent navigation intent and environmental judgment. When there is a conflict between the results on both sides, a conflict resolution rule is adopted. Safety-related conclusions, such as "obstacles ahead" or "stop immediately," take precedence over unsafe conclusions. When approaching the upper limit of T, the weight of the speech result is increased to ensure timely output. If only one side's result is obtained by the window's expiration, a downgraded navigation conclusion is formed based on that side's result, and a downgrade prompt is included in the output, while simultaneously triggering monitoring and recording of the unreached side.

[0083] S53: Generate navigation output results that simultaneously include voice prompts and image perception information.

[0084] In this step, it's important to note that the fused navigation output includes two types of information: action suggestions and environmental details. Action suggestions include at least the direction of movement and action commands, such as "turn right 15 degrees" or "stop." Environmental details include at least obstacle types, relative distances or directions, road boundaries, and traffic sign cues. The smart glasses terminal converts the action suggestions into voice-over text and simultaneously generates corresponding tactile vibration patterns to indicate urgent or high-priority events. A minimum interval is set for continuous output to avoid information congestion, and repetitive information is suppressed. The final output includes a timestamp, information about the fusion window used, and the source of the results involved in the fusion process, and is logged for subsequent evaluation and scheduling optimization.

[0085] By unifying the time base, result registration, and reception timeout control, waiting and congestion are reduced, ensuring continuous availability of the output link. On the other hand, through decision-level fusion based on confidence and priority, and weight adjustment when approaching the latency limit, a "timeliness first, risk first" output strategy is implemented. In the event of missing results on one side, a degraded output mechanism is used to maintain continuous prompts. This reduces end-to-end response time, minimizes safety risks caused by inconsistent or conflicting decisions, and improves the continuity, stability, and usability of navigation prompts for the blind.

[0086] In one specific embodiment, the feature fusion includes: The weighting parameters for voice processing and image processing results are set according to the real-time requirements of the navigation task. In this step, it should be noted that the smart glasses terminal maintains a set of adjustable weighting parameters in the fusion module to determine the relative contribution of the voice processing and image processing results in the same fusion cycle. These weighting parameters include voice weight and image weight, both of which are real numbers between zero and one, and their sum equals one.

[0087] The system configures initial weights based on the real-time level of the navigation task. When the task involves emergency obstacle avoidance or turn confirmation, the voice weight is set to a higher value to output clear action commands as quickly as possible. When the task involves environmental description or path refinement, the image weight is set to a higher value to enhance the accurate understanding of the scene. To avoid frequent fluctuations in weights within adjacent periods, the fusion module applies a smoothing strategy and hysteresis interval to weight updates. Specifically, it uses a sliding window averaging method or exponential smoothing method to limit weight changes, while specifying minimum and maximum weight boundaries to ensure that the minimum contribution of the other side's result is retained at any given time for cross-validation and as a safety net.

[0088] When the current processing delay is detected to be close to the preset delay limit, the weight of the voice processing result in the navigation output result is increased, so as to prioritize the real-time output of voice navigation information when processing resources are limited or delay pressure increases.

[0089] In this step, it's important to note that the fusion module calculates the current processing latency for each fusion cycle and compares it to a preset latency upper limit. The criteria for approaching the upper limit include: the current processing latency exceeding the upper limit threshold proportionally, or a sustained approach to the upper limit over several consecutive cycles. Once these criteria are met, the fusion module increases the speech weight according to a preset gain curve, while simultaneously decreasing the image weight by the same amount, ensuring the sum of the two equals one. The weight increase uses an incremental step or linear ramp method to prevent sudden jumps in auditory output. Once the latency pressure is relieved and the system stabilizes within a safe range, the fusion module gradually reverts to the normal weight configuration after a cooling-off period. To ensure safety, the system sets a constraint that the image weight cannot fall below a lower limit, so that even with latency constraints, a minimum amount of environmental perception information is retained, avoiding complete loss of judgment of nearby obstacles and road boundaries.

[0090] Example 2:

[0091] A blind-person-based handover control system, applied to any of the blind-person-based handover control methods described above. The system includes: The smart glasses terminal connects to and controls the control box and the cloud respectively. The smart glasses terminal is used for wearing and interaction, undertaking functions such as data acquisition, preprocessing, type recognition, task allocation, switching control, result fusion, and output. It includes at least: an acquisition unit, a preprocessing unit, a type recognition unit, a scheduling control unit, a switching control unit, a cache management unit, a performance indicator acquisition unit, a mapping relationship storage and update unit, a result aggregation and feature fusion unit, an output unit, a time synchronization unit, a log unit, and a connection unit.

[0092] The acquisition unit includes a camera and a microphone that generate image data and voice data, respectively.

[0093] The preprocessing unit performs denoising, exposure correction, geometric correction and compression on the image, endpoint detection and denoising on the speech, and adds timestamps and sequence numbers to the data.

[0094] The type identification unit determines the data type based on the acquisition channel and data format. In case of conflict, the data format shall prevail and the type shall be confirmed after performing lightweight content verification.

[0095] The scheduling and control unit allocates different types of data to the corresponding computing power layer according to the preset "environmental information type - computing power layer" mapping relationship.

[0096] The switching control unit triggers the switching when the mapping relationship is updated, completes the target layer readiness check, and works with the cache management unit to achieve seamless connection.

[0097] The cache management unit maintains continuous cache queues for the two data paths, records sequence numbers and integrity checks, and supports playback in the order of acquisition after switching.

[0098] The operational metrics acquisition unit periodically acquires wireless link quality, round-trip latency, packet loss rate, load of each computing layer, average inference latency, and temperature status.

[0099] The mapping relationship storage and update unit saves the mapping table, adjusts the threshold and cooling strategy, and updates the mapping relationship when the trigger condition is met.

[0100] The result aggregation and feature fusion unit subscribes to the two layers of feedback results, performs time alignment and deduplication, completes fusion according to the time delay constraint window, and adjusts the weights of speech and image according to the real-time requirements of the task.

[0101] The output unit provides navigation prompts to the user through voice broadcasts and tactile vibrations, and records the output time and source information involved in the fusion.

[0102] The time synchronization unit and the log unit ensure a unified time base across multiple devices, and fully record events such as conflicts, switching, and degradation, facilitating traceability and optimization.

[0103] The connection unit is used for the exchange of data and control information, including at least a wireless connection for communication between the first computing layer and the terminal, and a wired connection for communication between the second computing layer and the terminal. The connection unit supports handshake, heartbeat, bandwidth measurement, and anomaly detection to ensure that the link status is monitorable and controllable.

[0104] The cloud includes a first computing layer, which is used for low-latency processing, deploys lightweight models, and provides rapid parsing of speech recognition, semantic understanding, and simple scenarios. It includes a task queue, inference engine, result feedback interface, and status reporting interface. The feedback results include timestamps, input sequence numbers, and confidence levels.

[0105] The control box includes a second computing layer. This second computing layer is used for high-precision processing, deploying highly complex vision and multimodal models to provide obstacle detection, passable area determination, and environmental semantic understanding. It includes a task queue, a parallel inference engine, a result feedback interface, and a status reporting interface. The feedback results include category, location, distance, timestamp, and confidence level.

[0106] Example 3:

[0107] A method based on a blind person's switching medium, applicable to any of the aforementioned blind person-based switching control methods. Examples 1 and 2 are described in detail and will not be repeated here.

[0108] Therefore, the aforementioned blind-person switching control system, method, and medium achieve targeted allocation of computing resources by matching information types to different computing power layers, thereby reducing overall latency while ensuring the ability to process complex tasks, and thus improving the real-time performance and stability of blind navigation.

[0109] The above description is merely an embodiment of this application. It should be noted that those skilled in the art can make improvements without departing from the inventive concept of this application, but these improvements all fall within the protection scope of this application.

Claims

1. A blind-based switching control method applied to a smart glasses terminal, characterized in that, The method includes the following steps: The smart glasses terminal is constructed with a computing power hierarchy structure including at least a first computing power layer and a second computing power layer, wherein the computing power of the first computing power layer is lower than that of the second computing power layer; The user wears smart glasses, which collect external environmental information and generate data to be processed. The external environmental information includes at least two different types of environmental information. The smart glasses terminal identifies the data to be processed and determines the type of environmental information to which the data belongs. Based on the preset environment type and computing power mapping relationship, different types of data to be processed are assigned to the corresponding computing power layers for processing, so as to obtain the processing results of each computing power layer; The smart glasses terminal generates output results for navigation for the blind based on the processing results of each computing layer, and outputs the output results to the user in a way that the blind can perceive.

2. The blind-based switching control method according to claim 1, characterized in that, The external environment information includes at least two types of environmental information: voice information and image information; When the data to be processed is identified as voice information, the data to be processed is allocated to the first computing layer for processing. When the data to be processed is identified as image information, the data to be processed is allocated to the second computing layer for processing.

3. The blind-based switching control method according to claim 1, characterized in that, The method for "smart glasses terminal identifying data to be processed and determining the type of environmental information to which the data to be processed belongs" includes the following steps: The source sensor type of the data to be processed is determined based on at least one or any combination of the acquisition channel and data format of the data to be processed. The data to be processed originating from the microphone channel is identified as a voice information type; The data to be processed from the camera channel is identified as image information type.

4. The blind-based switching control method according to claim 1, characterized in that, The method for "constructing a hierarchical computing power structure for smart glasses terminals, including at least a first computing power layer and a second computing power layer" includes the following steps: The first computing power layer is a cloud computing power layer that is connected to the smart glasses terminal via a wireless network; The second computing layer is the computing layer within a wired control box that is connected to the smart glasses terminal via a wired connection.

5. The blind-based switching control method according to claim 1, characterized in that, The method further includes the following steps: Obtain at least one system operation indicator, including network bandwidth status and load status of each computing layer; When the system operation indicators meet the preset adjustment conditions, the environment type and computing power mapping relationship are updated to dynamically adjust the computing power layer corresponding to different environment information types.

6. The blind-based switching control method according to claim 1, characterized in that, The method of "allocating different types of data to be processed to corresponding computing power layers for processing according to preset environment types and computing power mapping relationships, so as to obtain the processing results of each computing power layer" includes the following steps: When switching the data to be processed corresponding to an environmental information type from one computing layer to another; The smart glasses terminal caches the data to be processed that is continuously collected before and after the switch; After the computing power layer switch is completed, the cached data to be processed is sent to the switched computing power layer for processing according to the collection order.

7. The blind-based switching control method according to claim 1, characterized in that, The method for "generating output results for navigation for the blind based on the processing results of each computing layer" includes the following steps: Obtain the speech processing results output by the first computing layer and the image processing results output by the second computing layer; Under a preset time delay constraint, feature fusion is performed on the speech processing result and the image processing result; Generate navigation output that includes both voice prompts and image perception information.

8. The blind-based switching control method according to claim 7, characterized in that, The feature fusion includes: Set the weight parameters for voice processing results and image processing results according to the real-time requirements of the navigation task; When the current processing delay is detected to be close to the preset delay limit, the weight of the voice processing result in the navigation output result is increased, so as to prioritize the real-time output of voice navigation information when processing resources are limited or delay pressure increases.

9. A switching control system for the blind, characterized in that, Applied to the blind-based switching control method as described in any one of claims 1-8.

10. A switching medium for the blind, characterized in that, Applied to the blind-based switching control method as described in any one of claims 1-8.

Images