Wearable visual impairment auxiliary obstacle detection glasses and implementation method for obstacle detection

Abstract

The embodiment of the invention provides wearable visual impairment auxiliary obstacle detection glasses and an obstacle detection implementation method, and the glasses comprise a pre-fuzzification visual sensor which is used for carrying out the optical fuzzification of a collected environment image, and outputting a low-frequency image signal; the embedded computing platform is used for simulating biological optic nerves, extracting environment depth motion intensity, performing hierarchical analysis based on inter-frame brightness difference, outputting potential collision risk signals in combination with a lateral suppression and point diffusion mechanism, fusing the potential collision risk signals and equipment motion states, evaluating global collision risks, filtering background motion interference, and determining the target collision risk. A collision early warning signal is provided; the power supply module is used for providing power supply support for other modules in the wearable visual impairment auxiliary obstacle detection glasses; the communication module is used for realizing interconnection with external equipment through wireless communication; and the sensing and interaction module is used for detecting the wearing state of the equipment and feeding back a collision early warning signal in real time by receiving a user voice instruction.

Description

Technical Field

[0001] This document relates to the fields of visual collision detection, privacy protection, and information security technologies, and particularly to a wearable visually impaired assistance obstacle detection glasses and an implementation method for obstacle detection. Background Art

[0002] According to authoritative surveys, the number of visually disabled people in China has exceeded 26 million, and the number of newly blind people each year is as high as 450,000, which means that a new blind person appears every 60 seconds. This alarming figure undoubtedly highlights the severity and prevalence of the visual impairment problem in China. Facing such a serious social problem, a huge demand has been generated.

[0003] Prior Art 1 provides a wearable assistance system based on visual interaction, which is a human-machine interactive device, a system based on finger detection and ROI extraction algorithms. It analyzes camera images through an ARM processor, identifies the object pointed to by the user's finger, and feeds it back to visually impaired people through a voice output module, highlighting the technical solutions of finger interaction and object recognition.

[0004] Disadvantages of Prior Art 1: This system is mainly applied to human-machine interaction rather than direct obstacle detection, resulting in limitations in practical applications. Specifically, the finger detection module of the system has limited ability to identify the finger position under different lighting conditions, especially in an environment with significant differences in indoor and outdoor light, and the recognition accuracy drops significantly. In addition, the ROI extraction module depends on the prior information of the finger position and has insufficient ability to recognize non-standard finger movements and gestures, which affects the accuracy of subsequent target detection.

[0005] Reasons for the disadvantages of Prior Art 1: The image processing algorithms used in the finger detection module, including detection based on skin color models and geometric constraint calculations, are difficult to accurately extract key features when facing complex lighting conditions or non-cooperative targets (such as finger occlusion, rapid movement). At the same time, the ROI extraction module depends on a fixed finger position model and has insufficient adaptability to the user's personalized gestures and non-standard actions. These factors, combined with the complex algorithm calculation amount, result in high requirements for the hardware performance of the system. Especially in the case of needing to process a large amount of image data in real time, the performance and response speed of the system are limited.

[0006] Prior Art 2 provides a path planning and obstacle avoidance method for visually impaired people to guide them. The core lies in using machine vision and artificial intelligence technologies, combining target detection algorithms and binocular ranging algorithms to perform 3D environment modeling, realizing path planning, and assisting visually impaired people to avoid obstacles through voice broadcast, protecting the path planning and obstacle avoidance technologies in a dynamic environment.

[0007] Disadvantages of the prior art 2: This method understands the environment through 3D modeling but fails to directly focus on obstacle detection. In an environment with dense crowds or frequently changing obstacles, the update speed of the nine-square grid map lags behind the environmental changes, unable to respond to newly emerging obstacles in a timely manner, thus affecting the safety of visually impaired people.

[0008] Reasons for the disadvantages of the prior art 2: The path planning algorithm relies on the environmental information captured by the binocular camera. However, in a dynamic environment, the processing speed of the image sequence captured by the camera is limited by the hardware performance and algorithm efficiency, resulting in a delay in updating the environmental model. At the same time, the recognition accuracy of the object detection algorithm decreases in complex scenarios, affecting the accuracy of obstacle detection. In addition, the high computational requirement of the algorithm demands that the hardware has a powerful parallel processing ability, especially in a dynamic environment where the environmental model needs to be updated quickly, further increasing the demand for hardware performance.

[0009] The prior art 3 provides a machine vision three-dimensional space reconstruction method for visually impaired people, which uses a 3D sensor to obtain depth images, combines a deep neural network model for object detection and recognition, and provides a new "graphic-audio" conversion mode of environmental information for visually impaired people through the mapping relationship between "text and image" and speech, highlighting the application of three-dimensional space reconstruction and deep learning technologies.

[0010] Disadvantages of the prior art 3: This method mainly focuses on text-image mapping rather than direct obstacle detection. The depth information obtained is inaccurate when dealing with objects on textureless or reflective surfaces, thus affecting the quality of three-dimensional space reconstruction. In addition, for fast-moving objects, the deep neural network model cannot update the object position information in real time, affecting the real-time performance of space reconstruction.

[0011] Reasons for the disadvantages of the prior art 3: The deep neural network model relies on a large amount of labeled data during training and has insufficient generalization ability for the surface characteristics of objects not included in the training data (such as textureless or reflective), resulting in inaccurate depth information extraction. At the same time, when the network model processes dynamic scenes, due to the high computational complexity, it is difficult to achieve real-time update, resulting in insufficient ability to track and reconstruct fast-moving objects. These factors all increase the requirements for hardware performance, especially when dealing with a large amount of depth image data and performing complex calculations.

[0012] In view of the deficiencies of the existing visually impaired assistance technologies that cannot directly detect obstacles and the fact that obstacle detection relies on large-scale data training and high computational resource consumption, due to the strong social demand for assistive guiding tools, an intelligent obstacle avoidance glasses specifically designed for visually impaired people is needed. The core function of this pair of glasses is to use a highly reliable and low-power bio-inspired collision detection algorithm to continuously monitor the collision risk through a visual sensor, and through voice prompts and vibration feedback, help users accurately respond to potential collisions, thus effectively avoiding danger. Summary of the Invention

[0013] The purpose of the present invention is to provide a wearable visually impaired assisted obstacle detection glasses and an implementation method for obstacle detection, aiming to solve the above problems in the prior art.

[0014] The present invention provides a wearable visually impaired assisted obstacle detection glasses, including:

[0015] A pre-fuzzy vision sensor, configured to optically blur the collected environmental image through a Fresnel lens and output a low-frequency image signal adapted to a biologically inspired vision algorithm;

[0016] An embedded computing platform, including: an enhanced lobular giant motion detector M-LGMD and a collision risk assessment network module M-DCMD, configured to, based on the low-frequency image signal, simulate biological vision nerves through M-LGMD, extract the environmental depth motion intensity, perform hierarchical analysis based on the inter-frame brightness difference, combine the mechanisms of lateral inhibition and point spread, output a potential collision risk signal, fuse the potential collision risk signal output by M-LGMD and the device motion state through M-DCMD, evaluate the global collision risk, filter background motion interference, and provide a collision warning signal;

[0017] A power module, configured to provide power support for other modules in the wearable visually impaired assisted obstacle detection glasses;

[0018] A communication module, configured to achieve interconnection with an external device through wireless communication and perform remote data transmission and transmission setting adjustment;

[0019] A sensing and interaction module, configured to detect the device wearing state through an IMU sensor, receive user voice commands through a voice recognition module, and provide real-time feedback of the collision warning signal through a buzzer and a vibration module.

[0020] The present invention also provides an implementation method for obstacle detection, which is used for the above-mentioned wearable visually impaired assisted obstacle detection glasses. The method includes:

[0021] Using the pre-fuzzy vision sensor, optically blur the collected environmental image through a Fresnel lens and output a blurred image signal of consecutive frames adapted to a biologically inspired vision algorithm;

[0022] Through the embedded computing platform, based on the blurred image signal, simulate the biological visual nerve through M-LGMD, extract the environmental depth motion intensity, conduct hierarchical analysis based on the inter-frame brightness difference, combine the mechanisms of lateral inhibition and point spread, output the potential collision risk signal, fuse the potential collision risk signal output by M-LGMD and the device motion state through M-DCMD, evaluate the global collision risk, filter the background motion interference, and provide a collision warning signal;

[0023] The power module provides power support for other modules in the wearable visually impaired assistant obstacle detection glasses;

[0024] Through the communication module, wireless communication is carried out to achieve interconnection with external devices, and remote data transmission and transmission setting adjustment are performed;

[0025] The IMU sensor of the sensing and interaction module is used to detect the device wearing state, the voice recognition module is used to receive user voice commands, and the buzzer and vibration module are used to provide real-time feedback of the collision warning signal.

[0026] Adopting the embodiment of the present invention, aiming at the disadvantage that the prior art cannot directly detect obstacles, the present invention uses a collision perception algorithm inspired by the visual nerve structure of an insect for collision warning. This algorithm does not depend on image details and does not require a cumbersome learning and training process, and can achieve real-time and effective obstacle detection in a complex environment. The embodiment of the present invention can significantly reduce the demand for large-scale data input, can use a lightweight single-chip microcomputer platform to complete real-time calculation, thereby reducing the load and cost of the computing hardware and extending the battery life of the device. At the same time, since the working principle of this algorithm does not involve the direct processing of sensitive image information, hardware blurring can be performed at the optical end to eliminate the risk of privacy leakage from the source. Description of the Drawings

[0027] In order to more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0028] Figure 1 is a schematic diagram of the wearable visually impaired assistant obstacle detection glasses according to the embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of the pre-blurring visual sensor according to the embodiment of the present invention;

[0030] Figure 3 is a schematic diagram of the data storage architecture according to the embodiment of the present invention;

[0031] Figure 4 It is a schematic diagram of the bio - vision - inspired algorithm according to an embodiment of the present invention;

[0032] Figure 5 It is a flowchart of the implementation method for obstacle detection according to an embodiment of the present invention. Detailed implementation manners

[0033] In order to enable those skilled in the art of this technology to better understand the technical solutions in one or more embodiments of this specification, the following will clearly and completely describe the technical solutions in one or more embodiments of this specification in conjunction with the accompanying drawings in one or more embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0034] According to an embodiment of the present invention, a wearable visually - impaired - assisted obstacle - detection glasses is provided. Figure 1 It is a schematic diagram of the wearable visually - impaired - assisted obstacle - detection glasses according to an embodiment of the present invention. As Figure 1 shown, the wearable visually - impaired - assisted obstacle - detection glasses according to an embodiment of the present invention specifically include:

[0035] A pre - fuzzification vision sensor 10, which is used to perform optical fuzzification processing on the collected environmental image through a Fresnel lens and output a low - frequency image signal adapted to the bio - vision - inspired algorithm; specifically includes: a micro - image sensor and a Fresnel lens placed in front of the micro - image sensor. Among them, the focal length and optical characteristics of the Fresnel lens are determined through calculation to ensure that in the image received by the micro - image sensor, the details of the object are effectively fuzzified while retaining necessary motion and contour information; the relative position between the micro - image sensor and the Fresnel lens is precisely adjusted to ensure the best fuzzification effect.

[0036] The pre - fuzzification vision sensor 10 is specifically used for:

[0037] Performing optical fuzzification through the Fresnel lens, so that in the image received by the micro - image sensor, the detailed information of the object is weakened or eliminated, and only the contour and motion information of the object are retained, and a low - frequency image signal adapted to the bio - vision - inspired algorithm is output through the micro - image sensor.

[0038] Embedded computing platform 11, comprising: an enhanced small-leaf giant motion detector M-LGMD and a collision risk assessment network module M-DCMD, which are used to, based on the low-frequency image signal, simulate the biological visual nerve through the M-LGMD, extract the environmental depth motion intensity, perform hierarchical analysis based on the inter-frame brightness difference, combine the mechanisms of lateral inhibition and point spread, output a potential collision risk signal, and fuse the potential collision risk signal output by the M-LGMD and the device motion state through the M-DCMD to evaluate the global collision risk, filter out background motion interference, and provide a collision warning signal; specifically including: a memory, which is used to store the data of the original image input cache in the YCbCr 4:2:2 format based on a pre-set data storage architecture, with each pixel occupying 16 bits, the buffer accommodating 3 frames of images, and adopting circular buffer management to ensure that there are always two frames of available image data during the calculation process; save the M-LGMD layer data, the M-LGMD layer data is divided into 6 layers, including two difference layers D0, D1, an inhibition layer D2, an expression layer D3, and an enhanced aggregation layer D4, where the D4 layer is 16-bit data and the rest of the layers are 8-bit data; save the M-LGMD intermediate variables and parameters: including the parameters, intermediate states, coefficients, and temporary storage variables for adjusting the calculation process, as well as the M-LGMD output signal representing the potential collision risk.

[0039] The specific use of the embedded computing platform is as follows:

[0040] Through the M-LGMD, perform differential calculation on the brightness information of consecutive frames to extract the inter-frame motion change signal; implement lateral inhibition according to the differential result, strengthen the potential collision information through operator convolution and comparison operations, and construct a preliminary collision expression layer; use the point spread method to perform correlation weighting on the preliminary collision expression layer, and aggregate the pixel information with local connection effectiveness to form an enhanced expression; specifically, through the M-LGMD, calculate the differential signals D0 and D1 layers based on the original frame data, which are derived from the brightness difference of the corresponding pixel points in adjacent frames. When the 0th frame is transmitting a new image, use the 2nd and 1st frames for differential calculation. When the 1st frame is transmitting, use the 0th and 2nd frames for differential calculation. When the 2nd frame is transmitting, use the 1st and 0th frames for differential calculation. The odd-numbered operation results are stored in D0, and the even-numbered operation results are stored in D1; use numerical convolution to calculate the inhibition layer D2 point by point, the data of which is from the earlier obtained layer of D0 and D1, and the boundary elements are discarded. Use the inhibition layer D2 to perform inhibition calculation on the later obtained layer of D0 and D1 to obtain the expression layer D3, and calculate the feedforward inhibition coefficient, the value of which is the average brightness of the earlier obtained layer of D0 and D1; for the expression layer D3, use the point spread method to obtain the correlation matrix of each pixel with its neighboring pixels, regard it as the effectiveness coefficient, and only perform weighted summation on the pixels with 8-connected effective association points with the surrounding area to construct the enhanced aggregation layer D4;

[0041] Through M-DCMD, based on multi-frame statistics, threshold determination and average calculation of potential collision intensity are carried out. If high risk is prompted for multiple consecutive frames and there is no large-scale motion interference globally, the M-DCMD effective alarm is triggered. Specifically, through M-DCMD, all pixel points in D4 with intensity exceeding a specific threshold are selected, and their average intensity is calculated, regarded as the potential collision risk coefficient of the current frame. The potential collision risk coefficients of several consecutive frames are statistically analyzed. If collision risk is prompted for multiple consecutive frames and at the same time the feedforward inhibition FFI does not indicate a global large motion trend, the M-DCMD effective alarm is triggered.

[0042] The embedded computing platform is a single-chip microcomputer main control with a Cortex-M4 architecture.

[0043] The power supply module 12 is used to provide power support for other modules in the wearable visually impaired assistance obstacle detection glasses; the power supply module specifically includes:

[0044] A rechargeable lithium battery is used to provide the power required for the wearable visually impaired assistance obstacle detection glasses;

[0045] A power management chip is used for battery charge and discharge management, voltage regulation and protection.

[0046] The communication module 13 is used to realize interconnection with external devices through wireless communication, and perform remote data transmission and transmission setting adjustment;

[0047] The sensing and interaction module 14 is used to detect the device wearing state through the IMU sensor, receive user voice commands through the voice recognition module, and give real-time feedback of collision warning signals through the buzzer and vibration module. The IMU sensor is specifically used for: detecting the attitude and motion state of the device, assisting the algorithm for feedforward inhibition FFI, and judging the global large motion trend. The sensing and interaction module specifically includes:

[0048] An LED indicator light is used for status indication, for power, connection status and collision alarm.

[0049] Buttons and a microphone are used to receive user input, supporting manual and voice operations;

[0050] A buzzer and a vibrator are used to provide sound and vibration feedback to timely remind the user of potential collision risks.

[0051] The embodiment of the present invention realizes a low-cost and high-efficiency obstacle detection function, and has the following advantages:

[0052] Lightweight and real-time: It does not rely on large-scale data training and can be quickly processed on an embedded platform. Privacy protection: Based on fuzzification perception, it avoids collecting sensitive image information. Adaptation to complex environments: The algorithm has robustness and can handle dynamic and complex scenarios. Meeting the daily travel obstacle avoidance needs of the visually impaired group: Without complex interaction instructions, it realizes natural and convenient applications. The embodiments of the present invention provide an intelligent and portable auxiliary tool for the visually impaired group, which is expected to significantly improve their travel safety and life convenience.

[0053] The embodiments of the present invention innovatively adopt a bio-vision-inspired collision perception algorithm, combined with a fuzzified vision sensor, an embedded computing platform, and software and hardware carriers, to provide accurate and reliable obstacle detection functions for the visually impaired.

[0054] 1. Prefuzzified vision sensor

[0055] To fundamentally avoid the potential privacy leakage problems caused by traditional high-definition image acquisition and reduce the computational complexity at the same time, the present invention adopts a prefuzzified vision sensor to optically fuzzify the collected environmental images through a Fresnel lens. The design of this lens optimizes the focal length and aperture parameters, enabling the image signal to retain low-frequency motion information while removing the detailed features of objects.

[0056] Technical functions: Output low-frequency image signals adapted to bio-vision-inspired algorithms; avoid privacy leakage risks from the optical end; provide a unified input signal basis to ensure the robustness and consistency of subsequent algorithms in various environments.

[0057] Scientific principle: Utilize the spatial frequency modulation characteristics of the Fresnel lens to spread and process high-frequency details, and only retain the low-frequency motion features related to the dynamics of obstacles, thereby simplifying the computational requirements of subsequent algorithms.

[0058] 2. Visual collision detection algorithm

[0059] To achieve real-time obstacle detection and collision risk assessment for fuzzified image signals, the present invention designs a bio-vision-inspired algorithm based on an embedded computing platform, including an enhanced lobular giant motion detector (M-LGMD) and a collision risk assessment network (M-DCMD):

[0060] Enhanced lobular giant motion detector (M-LGMD): Simulate the bio-vision nerve, extract the environmental depth motion intensity, conduct hierarchical analysis based on the inter-frame brightness difference, and combine mechanisms such as lateral inhibition and point spread to output potential collision risk signals.

[0061] Collision risk assessment network (M-DCMD): Integrate the output signals of M-LGMD and the device motion state, evaluate the global collision risk, filter out background motion interference, and achieve high-reliability early warning.

[0062] The algorithm process takes video frames as the basic unit and is divided into four stages:

[0063] Stage P1: Use the luminance information of consecutive frames to perform differential calculation and extract the inter-frame motion change signal.

[0064] Stage P2: Apply lateral inhibition to the differential signal, suppress non-collision-related signals, and strengthen the potential collision expression layer.

[0065] Stage P3: Adopt the point spread method to aggregate risk signals and generate an enhanced layer through local correlation weighting.

[0066] Stage P4: Based on multi-frame statistics, calculate the potential collision risk coefficient and determine whether to trigger a warning signal.

[0067] Technical effect: Without relying on high-definition images or complex deep learning models, the algorithm can achieve real-time collision detection by blurring the input signal; provide a lightweight algorithm framework optimized for embedded systems, which is suitable for low-power embedded computing platforms; combine multi-frame risk assessment and global large motion determination mechanisms to improve detection accuracy and reduce false alarm rates.

[0068] 3. Software and hardware carriers

[0069] In the embodiment of the present invention, wearable glasses are used as the hardware carrier, integrating a visual sensor and an embedded computing platform to form a set of compact and portable systems, and realizing a friendly human-machine interface through a user interaction module:

[0070] Hardware structure: Power supply module: Adopt a rechargeable lithium battery and a power management chip to support long-term stable operation; Communication module: Realize interconnection with external devices through 2.4G wireless communication, supporting remote data transmission and setting adjustment; Sensing and interaction module: The IMU sensor is used to detect the device wearing state, the voice recognition module is used to receive user voice commands, and the buzzer and vibration module are used to give real-time feedback on collision warning signals.

[0071] Technical effect: Provide power supply and physical support for the embedded computing platform; improve the user experience through various user interaction methods; realize data synchronization and personalized settings with external devices.

[0072] Through the above technical solutions, the present invention can solve the following problems in the prior art:

[0073] 1. Overcome the privacy and security risks of traditional high-definition image acquisition solutions and achieve full-link privacy protection from the optical end to the algorithm end;

[0074] 2. Eliminate the dependence on complex deep learning models and large-scale training data, enabling the obstacle detection algorithm to operate efficiently on resource-constrained embedded platforms;

[0075] 3. Implement real-time collision warning with a low false alarm rate in a dynamic and complex environment, significantly improving the travel safety and convenience of visually impaired people.

[0076] The following specifically describes the above technical solutions of the embodiments of the present invention in conjunction with the accompanying drawings.

[0077] The embodiments of the present invention include: (1) a pre-fuzzified vision sensor for collecting pre-fuzzified videos that remove the detailed features of objects from the environment, which is used to detect potential collision risks in subsequent algorithms; (2) a vision collision detection algorithm mounted on an embedded computing platform, including an enhanced small-leaf giant motion detector M-LGMD for characterizing the depth motion intensity in the environmental information, and a collision risk assessment network M-DCMD for assessing the risk of collision occurrence; and (3) a software and hardware carrier equipped with the embedded algorithm: a wearable device in the form of glasses for providing power supply, computing resources, user input and output, and communication devices.

[0078] (1) Pre-fuzzified vision sensor:

[0079] The pre-fuzzified vision sensor uses a compact optical component. By placing a Fresnel lens in front of the micro-image sensor, pre-fuzzification processing of the environmental image is achieved. Its structure is as Figure 2 shown.

[0080] Optical component design: Fresnel lens: Select a Fresnel lens with a short focal length and a moderate aperture, which has the advantages of being thin, light, and low-cost. The focal length and optical characteristics of the lens are precisely calculated to ensure that the detailed features of objects are effectively blurred in the image received by the sensor, while retaining the necessary motion and contour information. Micro-image sensor: Use a low-resolution and low-power image sensor to reduce the amount of data processing. The relative position between the sensor and the Fresnel lens is precisely adjusted to achieve the best fuzzification effect.

[0081] Pre-fuzzification principle: Removing detailed features: Through optical blurring, the detailed information such as the texture and color of objects in the image received by the sensor is weakened or eliminated, and only the contour and motion information of the objects are retained. This processing avoids the need for high-resolution images and reduces the computational complexity of subsequent algorithms. Protecting user privacy: Since the image is blurred at the acquisition stage, the device does not store or process clear environmental images, eliminating the risk of leakage of sensitive image information from the source and ensuring the privacy and security of users and others.

[0082] An embodiment of the present invention also provides a visual collision detection algorithm mounted on an embedded computing platform. The algorithm is mounted on the embedded computing platform, and the calculation process of the algorithm takes video frames as units and is divided into four stages (P1 - P4). Among them, P1 - P3 are the implementation processes of the enhanced lobular giant motion detector (M - LGMD), and P4 is the implementation process of the collision risk assessment network (M - DCMD):

[0083] The data storage architecture, such as Figure 3 shown below:

[0084] a. Image raw input cache: The data is stored in the YCbCr 4:2:2 format, and each pixel occupies 16 bits. The buffer can hold 3 frames of images and uses circular buffer management to ensure that there are always two frames of available image data during the calculation process.

[0085] b. M - LGMD layer data: It is divided into 6 layers, including two difference layers D0, D1, an inhibition layer D2, an expression layer D3, and an enhanced aggregation layer D4. Among them, the D4 layer is 16 - bit data, and the rest of the layers are 8 - bit data.

[0086] c. M - LGMD intermediate variables and parameters: Include parameters for adjusting the calculation process, intermediate states, coefficients, and temporary storage variables, as well as the M - LGMD output signal indicating potential collision risks.

[0087] The calculation process is as Figure 4 shown below, specifically as follows:

[0088] P1: Differential calculation

[0089] Based on the original frame data, calculate the differential signals of the D0 and D1 layers, which come from the luminance differences of corresponding pixel points in adjacent frames. The buffer management method is as follows:

[0090] When the 0th frame is transmitting a new image, use the 2nd and 1st frames for calculation.

[0091] When the 1st frame is transmitting, use the 0th and 2nd frames for calculation.

[0092] When the 2nd frame is transmitting, use the 1st and 0th frames for calculation.

[0093] The results are stored in the D0 or D1 layer. For odd - numbered runs, they are stored in D0, and for even - numbered runs, they are stored in D1.

[0094] P2: Lateral inhibition

[0095] The suppression layer D2 is calculated point - by - point using numerical convolution. Its data source is the earlier - obtained layer among D0 and D1, and boundary elements are discarded. The later - obtained layer among D0 and D1 is suppressed using the suppression layer D2 to obtain the expression layer D3. The feed - forward suppression coefficient is calculated, and its value is the average brightness of the earlier - obtained layer among D0 and D1.

[0096] The lateral suppression in the P2 stage uses the data of the earlier - obtained difference layer to suppress the later - obtained difference layer, enhancing the signals related to the approaching motion while suppressing the signals related to the background lateral motion or other irrelevant motions, and generating a preliminary expression layer describing the potential collision risk.

[0097] P3: Local Association Weighting

[0098] For the expression layer D3, the point - spread method is used to obtain the correlation matrix of each pixel with its neighboring pixels, which is regarded as the validity coefficient. Only the pixels with 8 - connected valid correlation points around are weighted and summed to construct the enhanced aggregation layer D4.

[0099] The point - spread method in the P3 stage only weights and sums the target points with effective connection relationships with surrounding pixels, making the aggregated data layer form a more spatially consistent collision - enhanced layer and improving the stability of subsequent threshold determination.

[0100] P4: Collision Risk Assessment

[0101] Select all pixel points in D4 whose intensity exceeds a specific threshold, calculate their average intensity, and regard it as the potential collision risk coefficient of the current frame. Statistically analyze the potential collision risk coefficients of several consecutive frames. If multiple consecutive frames indicate a collision risk and the feed - forward inhibition (FFI) does not indicate a global large - motion trend, then trigger the M - DCMD effective alarm.

[0102] The whole set of software and hardware carrier system of the embodiment of the present invention is integrated in the form of wearable glasses. The main functional modules include:

[0103] Core control module: The single - chip microcomputer main control with the Cortex - M4 architecture is used to run the visual collision detection algorithm, which has the characteristics of low power consumption and high performance, meeting the real - time computing requirements. The embedded computing platform is based on the microcontroller with the Cortex - M4 architecture to achieve low power consumption and real - time processing, enabling the algorithm to still achieve stable and real - time inter - frame difference, lateral suppression, and risk assessment operations in an embedded environment with limited resources.

[0104] Memory: It is used to store the data and programs required by the algorithm, including layer data, buffer areas, and intermediate variables.

[0105] Camera data interface: Connects to the pre - blurred vision sensor and is responsible for the acquisition and transmission of image data.

[0106] IMU Sensor: It is used to detect the attitude and motion state of the device, assist the algorithm in performing feedforward inhibition (FFI), and judge the global large-motion trend.

[0107] Voice Recognition Module: Receives the voice commands of the user and realizes the human-computer interaction function.

[0108] 2.4G Wireless Communication Module: Supports remote connection with devices such as smartphones, facilitating data synchronization and function expansion.

[0109] Rechargeable Lithium Battery: Provides the power required by the device, and the capacity design meets the needs of long-term continuous operation.

[0110] Power Management Chip: Responsible for the charge and discharge management, voltage regulation and protection functions of the battery, ensuring the stable operation of the system.

[0111] LED Indicator: Used for status indication, such as battery level, connection status and collision alarm, etc.

[0112] Button and Microphone: Used to receive user input, supporting manual and voice operations.

[0113] Buzzer and Vibrator: Provide sound and vibration feedback to timely remind the user of potential collision risks.

[0114] As can be seen from the above technical solutions, in the technical solution of the embodiment of the present invention, a pre-fuzzification visual sensor component is provided with a Fresnel lens in front of the micro-image sensor, which is used to perform optical fuzzification processing on the optical signal input in the environment, so that the image signal obtained by the image sensor only retains the low-frequency motion information and removes the object detail features; the micro-image sensor is used to capture the fuzzified image signal and transmit it to the backend processing unit; the embedded computing platform is equipped with a bio-vision neural-inspired collision detection algorithm module, including an enhanced lobular giant motion detector (M-LGMD) and a collision risk assessment network (M-DCMD); this algorithm can utilize the fuzzified image signal to extract the depth motion features in real time and evaluate the potential collision risk without the need for high-definition image details; the data cache and management module is used to store multiple frames of image signals to support the differential and suppression calculations based on the inter-frame information; the control unit is used to coordinate the algorithm execution and input / output interaction; the hardware carrier and user interaction module: integrates the embedded computing platform, pre-fuzzification visual sensor and power supply, communication, IMU sensor, voice recognition unit, buzzer, LED and button into a wearable device in the form of glasses; outputs a collision warning signal to the user through real-time voice or vibration prompts.

[0115] When using a wearable device, a Fresnel lens is used to optically blur the external environment image, and a micro-image sensor is used to obtain the blurred image signal; the blurred image signals of consecutive frames are input into a biologically inspired collision detection algorithm, and through multi-layer difference, suppression, aggregation, and statistical evaluation, the depth motion information is extracted and the potential collision risk is determined; if it is determined that there is a collision risk in multiple consecutive frames and there is no global large motion interference, a warning message is output to the user through a buzzer, vibration, or voice prompt. During the process of obtaining the blurred image, the lens parameters and sensor acquisition parameters are dynamically adjusted according to the environmental light conditions, so that the blurred effect maintains a consistent motion information extraction ability in various complex scenarios. When performing risk assessment, the average potential collision intensity of multiple frames is calculated and combined with the global large motion determination mechanism (such as the feedforward inhibition factor FFI) to effectively reduce the false alarm rate and ensure that the output collision warning has high reliability.

[0116] In summary, the technical solution of the embodiment of the present invention aims at the problems that the vision-based wearable assistive system relies on human-computer interaction rather than direct obstacle detection, the path planning and obstacle avoidance method for visually impaired people to guide blind relies on 3D modeling to understand the environment and does not directly focus on obstacle detection, and the machine vision three-dimensional space reconstruction method focuses on text image mapping rather than direct obstacle detection. The embodiment of the present invention performs collision warning by adopting a biologically inspired LGMD visual collision perception algorithm, which does not rely on image details and does not require a cumbersome learning and training process, and can achieve real-time and effective obstacle detection in a complex environment. The LGMD algorithm is specifically designed to detect fast approaching objects, reducing the dependence on high-computation-performance hardware, and reducing the system cost and energy consumption. In addition, the present invention reduces the dependence on a large amount of image data by optimizing the algorithm, improves the real-time response ability of the system, and enables more reliable and accurate obstacle avoidance feedback to be provided in a complex and changeable environment. Through these technical means, the present invention not only improves the accuracy and real-time performance of detection, but also reduces the hardware performance requirements of the device, thus providing a more practical assistive tool for visually impaired people. An obstacle detection glasses for visually impaired assistance of the embodiment of the present invention realizes real-time and efficient obstacle detection and collision warning based on an embedded lightweight computing platform. The embodiment of the present invention adopts a biologically inspired collision perception algorithm, combines a blurred vision sensor, an embedded computing platform and software and hardware carriers, and provides accurate and reliable obstacle detection functions for visually impaired people.

[0117] Method Embodiment

[0118] According to an embodiment of the present invention, an implementation method for obstacle detection is provided for the above-mentioned obstacle detection glasses for visually impaired assistance of a wearable device. Figure 5 It is a flowchart of the implementation method for obstacle detection of the embodiment of the present invention, as Figure 5As shown in the figure, the implementation method for obstacle detection according to an embodiment of the present invention specifically includes:

[0119] Step S501: Use a pre-fuzzy vision sensor to perform optical blurring processing on the collected environmental image through a Fresnel lens, and output a blurred image signal of continuous frames adapted to the bio-vision inspired algorithm;

[0120] Step S502: Through an embedded computing platform, based on the blurred image signal, simulate bio-vision nerves through M-LGMD to extract the environmental depth motion intensity, perform hierarchical analysis based on the inter-frame brightness difference, combine the mechanisms of lateral inhibition and point spread, output a potential collision risk signal, fuse the potential collision risk signal output by M-LGMD and the device motion state through M-DCMD to evaluate the global collision risk, filter background motion interference, and provide a collision warning signal;

[0121] Step S503: Use the power module to provide power support for other modules in the wearable visually impaired assisted obstacle detection glasses;

[0122] Step S504: Perform wireless communication through the communication module to achieve interconnection with external devices, and perform remote data transmission and transmission setting adjustment;

[0123] Step S505: Detect the device wearing state through the IMU sensor of the sensing and interaction module, receive user voice commands through the voice recognition module, and provide real-time feedback of the collision warning signal through the buzzer and vibration module.

[0124] The pre-fuzzy vision sensor specifically includes: a micro-image sensor and a Fresnel lens placed in front of the micro-image sensor. Among them, the focal length and optical characteristics of the Fresnel lens are determined through calculation to ensure that in the image received by the micro-image sensor, the details of the object are effectively blurred while retaining necessary motion and contour information; the relative position between the micro-image sensor and the Fresnel lens is precisely adjusted to ensure the best blurring effect.

[0125] Through optical blurring by the Fresnel lens, the detail information of the object in the image received by the micro-image sensor is weakened or eliminated, only the contour and motion information of the object are retained, the micro-image sensor receives and outputs a blurred image signal of continuous frames adapted to the bio-vision inspired algorithm, and the parameters of the Fresnel lens and the acquisition parameters of the micro-image sensor are dynamically adjusted according to the environmental light conditions during the acquisition of the blurred image.

[0126] The embedded computing platform specifically includes: a memory, which stores the data cached from the original image input in the YCbCr 4:2:2 format based on a pre-set data storage architecture. Each pixel occupies 16 bits, and the buffer can accommodate 3 frames of images. Ring buffer management is adopted to ensure that there are always two available frames of image data during the calculation process; it saves the M-LGMD layer data. The M-LGMD layer data is divided into 6 layers, including two difference layers D0 and D1, an inhibition layer D2, an expression layer D3, and an enhanced aggregation layer D4. Among them, the D4 layer is 16-bit data, and the rest of the layers are 8-bit data; it saves the M-LGMD intermediate variables and parameters: including the parameters for adjusting the calculation process, intermediate states, coefficients, and temporary storage variables, as well as the M-LGMD output signal indicating the potential collision risk.

[0127] Through M-LGMD, perform differential calculation on the luminance information of the blurred image signals of consecutive frames to extract the inter-frame motion change signals; implement lateral inhibition based on the differential results, strengthen the potential collision information through operator convolution and comparison operations, and construct a preliminary collision expression layer; use the point spread method to perform correlation weighting on the preliminary collision expression layer, and aggregate the pixel information with local connection effectiveness to form an enhanced expression; specifically, through M-LGMD, calculate the differential signals D0 and D1 layers based on the original frame data, which comes from the luminance difference of corresponding pixel points in adjacent frames. When the 0th frame is transmitting a new image, use the 2nd and 1st frames for differential calculation. When the 1st frame is transmitting, use the 0th and 2nd frames for differential calculation. When the 2nd frame is transmitting, use the 1st and 0th frames for differential calculation. The results of odd-numbered runs are stored in D0, and the results of even-numbered runs are stored in D1; calculate the inhibition layer D2 point by point using numerical convolution. Its data comes from the earlier obtained layer among D0 and D1, and the boundary elements are discarded. Use the inhibition layer D2 to perform inhibition calculation on the later obtained layer among D0 and D1 to obtain the expression layer D3, and calculate the feedforward inhibition coefficient, whose value is the average luminance of the earlier obtained layer among D0 and D1; for the expression layer D3, use the point spread method to obtain the correlation matrix of each pixel with its neighboring pixels, regard it as the effectiveness coefficient, and only perform weighted summation on the pixels with 8-connected effective association points with the surrounding area to construct the enhanced aggregation layer D4;

[0128] Through M-DCMD, perform threshold determination and average calculation on the potential collision intensity based on multi-frame statistics. If it continuously indicates a high risk for multiple frames and there is no large-scale motion interference globally, an effective M-DCMD alarm is triggered. Specifically, through M-DCMD, select all pixel points in D4 whose intensity exceeds a specific threshold, calculate their average intensity, regard it as the potential collision risk coefficient of the current frame, and statistically analyze the potential collision risk coefficients of consecutive several frames. If it continuously indicates a collision risk for multiple frames and at the same time the feedforward inhibition FFI does not indicate a global large motion trend, an effective M-DCMD alarm is triggered.

[0129] The embedded computing platform is the main control of a single-chip microcomputer with a Cortex-M4 architecture.

[0130] The IMU sensor detects the attitude and motion state of the device, assists the algorithm in performing feedforward inhibition FFI, and judges the global large-motion trend; it also assists in detecting whether it is in a worn state or a stationary state. If it is in a stationary state, the glasses device is set to the standby mode to save power.

[0131] The power required for the wearable visual impairment assistance obstacle detection glasses is provided by a rechargeable lithium battery;

[0132] The charge and discharge management, voltage regulation, and protection of the battery are carried out through a power management chip;

[0133] Status indication is carried out through LED indicators for battery power, connection status, and collision alarms.

[0134] User input is received through buttons and microphones, supporting manual and voice operations;

[0135] Sound and vibration feedback are provided through a buzzer and a vibrator to timely remind the user of potential collision risks.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A wearable visually impaired assistance obstacle detection glasses, characterized in that, Including: A pre-blurring vision sensor for optically blurring the acquired environmental images through a Fresnel lens and outputting a blurred image signal of continuous frames adapted to the bio-vision inspired algorithm; An embedded computing platform, including: an enhanced lobular giant motion detector M-LGMD and a collision risk assessment network module M-DCMD, for based on the blurred image signal, simulating bio-vision nerves through M-LGMD, extracting the environmental depth motion intensity, performing hierarchical analysis based on the inter-frame luminance difference, combining the mechanisms of lateral inhibition and point spread, outputting a potential collision risk signal, and fusing the potential collision risk signal output by M-LGMD and the device motion state through M-DCMD to evaluate the global collision risk, filtering background motion interference, and providing a collision warning signal; A power module for providing power support to other modules in the wearable visually impaired assisted obstacle detection glasses; A communication module for realizing interconnection with external devices through wireless communication and performing remote data transmission and transmission setting adjustment; A sensing and interaction module for detecting the device wearing state through an IMU sensor, receiving user voice commands through a voice recognition module, and providing real-time feedback of the collision warning signal through a buzzer and a vibration module.

2. The wearable visually impaired assistance obstacle detection glasses according to claim 1, characterized in that, The pre-blurring vision sensor specifically includes: a micro-image sensor and a Fresnel lens placed in front of the micro-image sensor. Among them, the focal length and optical characteristics of the Fresnel lens are calculated and determined to ensure that the details of the object are effectively blurred in the image received by the micro-image sensor, while retaining necessary motion and contour information; the relative position of the micro-image sensor and the Fresnel lens is precisely adjusted to ensure the best blurring effect.

3. The wearable visually impaired assisted obstacle detection glasses according to claim 2, characterized in that, The pre-blurring vision sensor is specifically used for: Performing optical blurring through the Fresnel lens, so that the detail information of the object in the image received by the micro-image sensor is weakened or eliminated, only the contour and motion information of the object are retained, receiving and outputting a blurred image signal of continuous frames adapted to the bio-vision inspired algorithm through the micro-image sensor, and dynamically adjusting the parameters of the Fresnel lens and the acquisition parameters of the micro-image sensor according to the environmental illumination conditions during the blurred image acquisition process.

4. The wearable visually impaired assistance obstacle detection glasses according to claim 1, wherein, The embedded computing platform specifically includes: a memory for storing the data of the original image input buffer in the YCbCr 4:2:2 format based on a pre-set data storage architecture, with each pixel occupying 16 bits, the buffer accommodating 3 frames of images, and adopting circular buffer management to ensure that there are always two frames of available image data during the calculation process; saving the M-LGMD layer data, the M-LGMD layer data is divided into 6 layers, including two difference layers D0, D1, an inhibition layer D2, an expression layer D3, and an enhanced aggregation layer D4, where the D4 layer is 16-bit data and the remaining layers are 8-bit data; saving the M-LGMD intermediate variables and parameters: including the parameters for adjusting the calculation process, intermediate states, coefficients, and temporary storage variables, as well as the M-LGMD output signal representing the potential collision risk.

5. The wearable visually impaired assisted obstacle detection glasses according to claim 4, characterized in that, The embedded computing platform is specifically used for: Through M-LGMD, differential calculation is performed on the luminance information of the blurred image signals of consecutive frames to extract the inter-frame motion change signals; lateral inhibition is implemented based on the differential results, and potential collision information is enhanced through operator convolution and comparison operations, and a preliminary collision expression layer is constructed; the point spread method is used to perform correlation weighting on the preliminary collision expression layer, and the pixel information with local connection effectiveness is aggregated to form an enhanced expression. Through M-DCMD, threshold determination and average calculation are performed on the potential collision intensity based on multi-frame statistics. When high risk is prompted for multiple consecutive frames and there is no large-scale motion interference globally, an effective alarm of M-DCMD is triggered.

6. The wearable visually impaired assistance obstacle detection glasses according to claim 5, characterized in that, The embedded computing platform is specifically used for: Through M-LGMD, differential signals D0 and D1 layers are calculated based on the original frame data, which are derived from the luminance differences of corresponding pixel points in adjacent frames. When the 0th frame is transmitting a new image, the 2nd and 1st frames are used for differential calculation. When the 1st frame is transmitting, the 0th and 2nd frames are used for differential calculation. When the 2nd frame is transmitting, the 1st and 0th frames are used for differential calculation. The results of odd-numbered runs are stored in D0, and the results of even-numbered runs are stored in D1; the suppression layer D2 is calculated point by point using numerical convolution, and its data is from the earlier obtained layer among D0 and D1. The boundary elements are discarded, and the suppression layer D2 is used to perform suppression calculation on the later obtained layer among D0 and D1 to obtain the expression layer D3, and the feedforward suppression coefficient is calculated, and its value is the average luminance of the earlier obtained layer among D0 and D1. For the expression layer D3, the point spread method is used to obtain the correlation matrix of each pixel with its neighboring pixels, which is regarded as the effectiveness coefficient, and only the pixels with 8-connected effective association points with the surrounding are weighted and summed to construct the enhanced aggregation layer D4. Through M-DCMD, all pixel points in D4 with intensities exceeding a specific threshold are selected, and their average intensity is calculated, which is regarded as the potential collision risk coefficient of the current frame. The potential collision risk coefficients of several consecutive frames are statistically analyzed. If collision risks are prompted for multiple consecutive frames and the feedforward inhibition FFI does not indicate a global large motion trend, an effective alarm of M-DCMD is triggered.

7. The wearable visually impaired assistance obstacle detection glasses according to claim 1, characterized in that, The embedded computing platform is a single-chip microcomputer main control with a Cortex-M4 architecture.

8. The wearable visually impaired assisted obstacle detection glasses according to claim 1, wherein, The IMU sensor is specifically used for: detecting the posture and motion state of the device, assisting the algorithm in performing feedforward inhibition FFI to judge the global large motion trend; assisting in detecting whether it is in a wearing state or a stationary state. If it is in a stationary state, the glasses device is set to the standby mode to save power.

9. The wearable visually impaired assistance obstacle detection glasses according to claim 1, characterized in that The power supply module specifically includes: A rechargeable lithium battery for providing the power required for the wearable visually impaired assistance obstacle detection glasses. A power management chip for performing charge and discharge management, voltage regulation and protection of the battery. The sensing and interaction module specifically includes: An LED indicator for status indication, for power, connection status and collision alarm. A button and a microphone for receiving user input, supporting manual and voice operations. A buzzer and a vibrator are used to provide sound and vibration feedback to timely remind the user of potential collision risks.

10. An implementation method for obstacle detection, characterized in that, For the wearable visually impaired assisted obstacle detection glasses according to any one of claims 1 to 9, the method specifically includes: Using a pre-fuzzification vision sensor to perform optical fuzzification processing on the collected environmental images through a Fresnel lens, and outputting a fuzzified image signal of continuous frames adapted to the bio-vision inspired algorithm; Through an embedded computing platform, based on the fuzzified image signal, simulating bio-vision nerves through M-LGMD to extract the environmental depth motion intensity, performing hierarchical analysis based on the inter-frame brightness difference, combining the mechanisms of lateral inhibition and point spread, outputting a potential collision risk signal, fusing the potential collision risk signal output by M-LGMD and the device motion state through M-DCMD to evaluate the global collision risk, filtering background motion interference, and providing a collision warning signal; Providing power support for other modules in the wearable visually impaired assisted obstacle detection glasses through a power module; Realizing interconnection with external devices through wireless communication via a communication module for remote data transmission and transmission setting adjustment; Detecting the device wearing state through the IMU sensor of the sensing and interaction module, receiving user voice commands through the voice recognition module, and providing real-time feedback of the collision warning signal through the buzzer and vibration module.

Images