A method and system for obstacle avoidance for the blind
Through the combination of depth cameras and RGB cameras, depth images are divided and obstacle detection and voice broadcast are performed, which solves the accuracy and stability of existing blind obstacle avoidance equipment and realizes safe guidance for blind people under strong light.
Patent Information
- Application Number
- CN202210050077.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-17
- Publication Date
- 2025-07-25
- Estimated Expiration
- 2042-01-17
AI Technical Summary
The existing blind obstacle avoidance equipment based on ultrasonic or infrared monitoring methods have problems such as slow measurement speed, susceptibility to external interference, and low accuracy. It is impossible to accurately locate the position and distance of the obstacle, resulting in unstable obstacle avoidance effect.
The depth camera is used to obtain the depth image, divide it into multiple column areas, calculate the minimum depth value and remind the blind people through vibration of the alarm device. Combined with the RGB camera to identify the location and type of obstacles, use a deep learning model for obstacle detection and voice broadcast.
It realizes the rapid and accurate identification of obstacle locations and types under strong light, improves the stability and safety of obstacle avoidance, and meets the real-time needs of blind people to avoid obstacles.
Smart Images
Figure CN114399695B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing. Specifically, it relates to the field of blind obstacle avoidance technology based on images. More specifically, it relates to a blind obstacle avoidance method and system based on image processing. Background Art
[0002] The number of blind people globally has exceeded 40 million. Among them, China is one of the countries with the largest number of blind people in the world and also the country with the largest number of visually disabled patients. For this special group of blind people, how to use scientific and technological means to facilitate their lives and ensure their safety has become an irresistible trend and a problem that must be properly solved.
[0003] Existing traditional blind obstacle avoidance devices mainly use ultrasonic or infrared-based monitoring methods to detect obstacles and give obstacle alarms or reminders to blind people to help them avoid obstacles. Among them:
[0004] The ultrasonic-based monitoring method mainly uses ultrasonic obstacle avoidance sensors to detect obstacles. Such a method has problems such as slow measurement speed and being easily affected by temperature / wind direction. In addition, if the ultrasonic wave faces an object that absorbs sound, the detection will fail, and the obstacle avoidance effect is unstable.
[0005] The infrared-based monitoring method mainly uses infrared detectors, but this method is easily affected by strong light, and the accuracy rate will decrease.
[0006] In summary, traditional blind obstacle avoidance devices cannot effectively guarantee the safety of blind people and may cause harm to blind people due to their dependence. In addition, existing obstacle avoidance devices cannot accurately locate the position and accurate distance of obstacles, and the reminder to blind people is not timely and accurate enough. Summary of the Invention
[0007] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a blind obstacle avoidance method and system.
[0008] According to the first aspect of the present invention, there is provided a blind obstacle avoidance method, the method comprising: S1, obtaining a depth image of the current environment; S2, dividing the depth image into multiple column regions and processing each column region to obtain the minimum depth value in each column region; S3, performing obstacle alarms for each column region based on the minimum depth value of each column region.
[0009] Preferably, the step S2 includes: S21, dividing the depth image into a preset number of column regions and calculating the depth value of each column region; S22, filtering the depth values of each column region according to a preset maximum depth threshold and minimum depth threshold, removing the depth values outside the range of the maximum depth threshold and the minimum depth threshold, and obtaining the minimum depth value of each column region. In some embodiments of the present invention, the preset number of the column regions is 9. Wherein, the maximum depth threshold refers to the farthest distance of the obstacle that needs to be recognized and will affect the blind; the minimum depth threshold refers to the closest distance of the obstacle acceptable to the blind from the blind.
[0010] Preferably, the step S3 includes: S31, obtaining the alarm intensity of the corresponding area of the alarm device for each column region based on the minimum depth value of each column region in a preset alarm intensity standard; wherein, the alarm intensity standard refers to the alarm intensity corresponding to preset different depth values; S32, according to the alarm intensity of each column region of the alarm device, obtaining the PWM duty cycle of the corresponding signal intensity frequency, and controlling the corresponding area of the alarm device to alarm the blind in a vibration manner, wherein the vibration frequency of each column region is proportional to its corresponding PWM duty cycle. In some embodiments of the present invention, the duty cycle of each column region is calculated by the following method:
[0011] P = (S - D) / S
[0012] Wherein, D is the minimum depth value of the current column region, and S is the maximum value of the depth measurement range.
[0013] In some embodiments of the present invention, the step S1 further includes obtaining an RGB image; the method further includes: S4, using a preset target detection model to detect the position and type of the obstacle in the RGB image and reporting them to the blind, wherein the target detection model is obtained by training a neural network with the RGB image as the input and the position and type of the obstacle in the image as the output. Preferably, in step S4, the position and type of the obstacle are reported to the blind by voice.
[0014] Preferably, the target detection model is obtained by the following method: P1, constructing an initial target detection model with a res-block network as the backbone network and a squeeze-and-excitation network as the pre-network; P2, training the initial target detection model to convergence based on a preset RGB image training set and using the following loss function:
[0015] L = Ls + λLc
[0016] Wherein, Ls is the softmax loss function, Lc is the Center Loss function, and λ is the weight of the loss function.
[0017] According to the second aspect of the present invention, there is provided a blind obstacle avoidance system for implementing the method described in the first aspect of the present invention. The system includes: an image acquisition module for acquiring a depth image of the current environment; an image processing module for dividing the depth image acquired by the image acquisition module into multiple column regions and processing each column region to obtain the minimum depth value in each column region; an alarm control module for outputting a control signal for controlling an alarm device to alarm a blind person about an obstacle according to the minimum depth value of each column region obtained by the image processing module; and an alarm device for respectively alarming about obstacles in each column region according to the control signal of the alarm control module. Preferably, the alarm device includes a plurality of alarm units, different alarm units corresponding to different column regions and each alarm unit corresponding to one column region.
[0018] In some embodiments of the present invention, the alarm control module is configured to perform the following operations: obtaining the alarm intensity of the corresponding area of the alarm device for each column region in a preset alarm intensity standard based on the minimum depth value of each column region; wherein, the alarm intensity standard refers to the alarm intensity corresponding to preset different depth values; obtaining the PWM duty cycle of the corresponding signal intensity frequency according to the alarm intensity of each area of the alarm device, and outputting a control signal for controlling the alarm unit of the corresponding area of the alarm device to vibrate at a vibration frequency for alarming a blind person about an obstacle based on the PWM duty cycle of each area, wherein the vibration frequency of the alarm unit of each area is proportional to its corresponding PWM duty cycle.
[0019] In some embodiments of the present invention, the image acquisition module is further configured to acquire an RGB image of the current environment; and the system further includes: an obstacle information acquisition module for acquiring the position and type of obstacles in the environment where the blind person is located based on the RGB image and voice-broadcasting them to the blind person.
[0020] Preferably, the obstacle information acquisition module includes: a target detection model for detecting the position and type of obstacles in the environment where the blind person is located, wherein the target detection model is a model obtained by training a neural network with an RGB image as the input and the position and type of obstacles in the image as the output; and a voice module for broadcasting the position and type of obstacles detected by the target detection module to the blind person.
[0021] Compared with the prior art, the advantages of the present invention are as follows: By using a depth camera for obstacle avoidance detection, the present invention can quickly detect obstacles in front; by using an RGB camera to obtain visible light images (RGB images), it will not be interfered even under strong light, and has extremely strong anti-interference ability. By combining depth images with color images and using deep learning-based image recognition, it can not only effectively avoid obstacles, but also distinguish the position and type of obstacles in front, providing more accurate guidance for blind users and improving safety. The present invention fully considers the characteristics of the blind. By splitting the depth image to calculate the minimum depth value, it changes the PWM duty cycle of the corresponding area of the alarm device, and uses different vibration intensities and frequencies to remind the blind, so that the blind can effectively obtain the distance and orientation of the obstacles through vibration. In addition, for the problem that the traditional SSD object detection method has poor real-time performance on embedded devices with limited computing power when used for image semantic segmentation, the present invention fully considers the very high real-time requirement for blind obstacle avoidance, optimizes the traditional object detection method, and improves the inference time of the algorithm while ensuring the detection accuracy remains unchanged. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The following further describes the embodiments of the present invention with reference to the accompanying drawings, where:
[0023] Figure 1 is a schematic flowchart of a blind obstacle avoidance method according to an embodiment of the present invention;
[0024] Figure 2 is a schematic diagram of depth image splitting according to an embodiment of the present invention;
[0025] Figure 3 is a schematic structural diagram of a blind obstacle avoidance system according to an embodiment of the present invention;
[0026] Figure 4 is a schematic diagram of the obstacle recognition effect according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0027] In order to make the objectives, technical solutions and advantages of the present invention clearer, the following further details the present invention through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0028] As described in the background art, existing traditional blind obstacle avoidance solutions generally use ultrasonic or infrared technology, which have problems such as low accuracy, unstable effects, and being easily interfered by the outside world. The present invention proposes an image-based obstacle avoidance method that accurately identifies the distance, type, and orientation of obstacles through depth images and gives an alarm.
[0029] According to an embodiment of the present invention, as Figure 1As shown, a method for blind people to avoid obstacles is provided. The method includes steps S1, S2, S3, and S4, which are described in detail below for each step.
[0030] In step S1, a depth image of the current environment is obtained. According to an embodiment of the present invention, a depth image of obstacles and an RGB image of obstacles in the environment where the blind person is located are collected through a depth camera and an RGB camera.
[0031] In step S2, the depth image is divided into multiple column regions and each column region is processed to obtain the minimum depth value in each column region;
[0032] . According to an embodiment of the present invention, the depth image is processed as follows: S21: The depth image is divided into a preset number of column regions and the depth value of each column region is calculated; S22: The depth values of each column region are filtered according to a preset maximum depth threshold and minimum depth threshold, and the depth values outside the range of the maximum depth threshold and the minimum depth threshold are removed, and the minimum depth value of each column region is obtained. Among them, the purpose of setting the maximum depth threshold and the minimum depth threshold is to remove the maximum value and the minimum value in the depth values of each column region, so as to ensure that the minimum depth value is obtained within the effective region. According to an embodiment of the present invention, the maximum depth threshold and the minimum depth threshold are set according to the environment where the blind person is located, where: the maximum depth threshold refers to the farthest distance of the obstacle that needs to be recognized and affects the blind person; the minimum depth threshold refers to the closest distance of the obstacle that the blind person can accept from the blind person.
[0033] According to an embodiment of the present invention, as Figure 2 shown, the pixel size (resolution size) of the depth image is set to 480X640, the accuracy is set to 1 / 3 mm (the accuracy corresponds to the accuracy of the depth. For example, if the accuracy is 1 / 3 mm and the returned depth is 3000 mm, the actual depth is: 3000X1 / 3 = 1000 mm), and the depth image is divided into 9 columns, the depth value in each column region is calculated, and the maximum value and the minimum value in the depth values of each column region are removed to obtain the minimum depth value in each column region.
[0034] In step S3, obstacle alarms are respectively given for each column region based on the minimum depth value of each column region. According to an embodiment of the present invention, the step S3 includes: S31. Obtaining the alarm intensity of the corresponding area of the alarm device for each column region based on the minimum depth value of each column region in a preset alarm intensity standard; wherein, the alarm intensity standard refers to the alarm intensity corresponding to preset different depth values; S32. According to the alarm intensity of each column region of the alarm device, obtaining the PWM duty cycle of the corresponding signal intensity frequency, and controlling the corresponding area of the alarm device to give an obstacle alarm to the blind in a vibration manner, wherein the vibration frequency of each column region is proportional to its corresponding PWM duty cycle. According to an embodiment of the present invention, the PWM duty cycle P obtained according to the minimum depth value is calculated by the following method:
[0035] P = (S - D) / S
[0036] wherein, D is the minimum depth value of each column region, and S is the maximum value of the depth measurement range.
[0037] As described in the previous embodiment, the depth image is sliced into 9 columns, and the minimum depth value of each column is obtained, which are respectively denoted as [D1, D2, D3, D4, D5, D6, D7, D8, D9]. Thus, the PWM duty cycle of each column can be calculated, which are respectively denoted as [PWM1, PWM2, PWM3, PWM4, PWM5, PWM6, PWM7, PWM8, PWM9]. Then, for different regions, vibration alarms are given based on the corresponding duty cycles. The higher the PWM duty cycle, the higher the vibration frequency. During the continuous cyclic acquisition of depth images, the blind person will appear to be getting closer or farther away from the obstacle, and the corresponding minimum depth value will become smaller or larger, the duty cycle will increase or decrease accordingly, and the vibration frequency will strengthen or weaken, thereby reminding the blind person of the distance of the obstacle. For each sliced column region, vibration alarms are respectively given, which can accurately remind the blind person of the position and distance of the obstacle.
[0038] In step S4, a preset target detection model is used to detect the position and type of obstacles in the RGB image and report them to the blind person. Among them, the target detection model is obtained by training a neural network with the RGB image as the input and the position and type of obstacles in the image as the output. According to an embodiment of the present invention, while obtaining the depth image of the current environment of the blind person, the RGB image of the current environment of the blind person is obtained through an RGB camera, and the position and type of obstacles in the RGB image are detected through the target detection model and reported to the blind person by voice.
[0039] According to an embodiment of the present invention, the present invention provides a blind person obstacle avoidance system, as Figure 3As shown in the figure, the system includes: an image acquisition module, configured to acquire images of the environment where the blind person is currently located, including depth images and RGB images; an image processing module, configured to divide the depth images acquired by the image acquisition module into multiple column regions and process each column region to obtain the minimum depth value in each column region; an alarm control module, configured to output a control signal for controlling an alarm device to give an obstacle alarm to the blind person according to the minimum depth value of each column region obtained by the image processing module; an alarm device, configured to give an obstacle alarm to each column region respectively according to the control signal of the alarm control module; and an obstacle information acquisition module, configured to acquire the position and type of obstacles in the environment where the blind person is located based on the RGB image and announce them to the blind person by voice.
[0040] According to an embodiment of the present invention, depth images of environmental obstacles and RGB images of obstacles can be collected through a depth camera (such as a TOF sensor) and an RGB camera. Through deep learning analysis, the type and distance of the obstacles can be obtained and fed back to the corresponding area of the alarm device (such as a vibration motor), and an alarm signal (such as causing vibration) is output to remind the blind person that there is an obstacle in front and what type of obstacle it is.
[0041] According to an embodiment of the present invention, the image processing module is configured to perform the following processing on the depth image: dividing the depth image into a preset number of column regions and calculating the depth value of each column region; filtering the depth value of each column region according to a preset maximum depth threshold and minimum depth threshold, removing the depth values outside the range of the maximum depth threshold and the minimum depth threshold, and obtaining the minimum depth value of each column region. The maximum depth threshold and the minimum depth threshold are set according to the environment where the blind person is located, where: the maximum depth threshold refers to the farthest distance of the obstacle that needs to be recognized and will affect the blind person; the minimum depth threshold refers to the closest distance of the obstacle that the blind person can accept from the blind person.
[0042] According to an embodiment of the present invention, the alarm control module is configured to perform the following operations: obtaining the alarm intensity of the corresponding area of the alarm device for each column region in a preset alarm intensity standard based on the minimum depth value of each column region; where the alarm intensity standard refers to the alarm intensity corresponding to preset different depth values; obtaining the PWM duty cycle of the signal intensity frequency corresponding to the alarm intensity of each area of the alarm device, and outputting a control signal for controlling the vibration frequency of the alarm unit in the corresponding area of the alarm device based on the PWM duty cycle of each area, where the vibration frequency of the alarm unit in each area is proportional to its corresponding PWM duty cycle.
[0043] According to an embodiment of the present invention, the alarm device includes a plurality of alarm units. Different alarm units correspond to different regions, and each alarm unit corresponds to one region. According to an embodiment of the present invention, each alarm unit can be set as a vibration motor. The vibration intensity and vibration frequency of the vibration motor are proportional to the PWM duty cycle of its corresponding region. The greater the alarm intensity, the greater the vibration intensity, the higher the PWM duty cycle, and the higher the vibration frequency.
[0044] According to an embodiment of the present invention, the obstacle information acquisition module includes: a target detection model for detecting the position and type of obstacles in the environment where the blind person is located. Among them, the target detection model is a model obtained by training a neural network with RGB images as input and the position and type of obstacles in the image as output; a voice module for reporting the position and type of obstacles detected by the target detection module to the blind person.
[0045] Preferably, the target detection model includes a front network and a backbone network. According to an embodiment of the present invention, the front network is constructed with a squeeze-and-excitation network, and the backbone network is constructed with a res-block network. It should be noted that the target detection model constructed in the embodiment of the present invention has been improved compared with the traditional target detection model. Without changing the detection accuracy, the inference time of the algorithm has been improved. Specifically, the traditional target detection model SSD uses VGG or resnet50 as the backbone, while in the embodiment of the present invention, based on the res-block, combined with the squeeze-and-excitation image recognition structure as the front network, and using a loss function combining Center Loss and softmax loss, a neural network structure is constructed to maximize the extraction of the specific features of the appearance of each obstacle. Among them: The Res-block structure can effectively accelerate the gradient backpropagation in the model training process by introducing a skip connection mechanism, and to a certain extent alleviates the phenomenon of gradient disappearance. The squeeze-and-excitation image recognition structure is a brand-new structure introduced in SENet for image recognition. It models the correlation between the feature channels of the network layer and enhances the important features to improve the accuracy. Since the construction method and training of the neural network are common techniques in the art, they will not be elaborated here. The setting of the loss function of the target detection model will be introduced below
[0046] According to an embodiment of the present invention, the present invention uses a loss function that combines Center Loss and softmax loss, constructs a neural network structure, and trains the constructed neural network structure to obtain an object detection model; then preprocesses the RGB color image, including performing low-light enhancement on an overly dark image with a brightness lower than a set threshold, performing exposure suppression on a backlit image, etc. to generate a relatively clear background RGB image, and then inputs the processed RGB image into the object detection model. Through the forward inference of the object detection model, the position and type of obstacles in the RGB color image are obtained.
[0047] According to an embodiment of the present invention, the loss function of the object detection model can be expressed as:
[0048] L = Ls + λLc
[0049] Where Ls is the softmax loss function, Lc is the Center Loss function, and λ is the weight of the loss function. L is the object recognition loss function, which can make up for the shortcoming that the softmax loss function only focuses on maximizing the inter-class distance in the classification task. During the model training process, by combining center loss and softmax loss, that is, the loss function is L = Ls + λLc, it can simultaneously maximize the inter-class distance and minimize the intra-class distance. Center loss hopes that the sum of the squares of the distances from the features of each sample in a training set to the average center of the features is as small as possible, that is, the intra-class distance is as small as possible. And the definition of Center Loss is as follows:
[0050]
[0051] Where N is the size of the training set, x i is the feature before the fully connected layer, and C represents the feature center of the y i th category.
[0052] The present invention processes the RGB image in the environment where the blind person is located through the object detection model to obtain the type and position of the obstacle, and broadcasts it to the blind person through voice to prompt the blind person that there is an obstacle in front and the type and position of the obstacle. For example, Figure 4 as shown in the example, it can be obtained that the obstacle in front of the blind person is a vehicle and the position of the vehicle. At the same time, the distance of the vehicle can be obtained by combining with the depth image, and different vibration intensities and frequencies are used to prompt the distance of the vehicle. Thus, it can accurately alarm the blind person about the obstacle and achieve obstacle avoidance without being affected by the lighting environment.
[0053] The present invention uses a depth camera for obstacle avoidance detection, which can quickly detect obstacles in front; it obtains visible light images (RGB images) through an RGB camera and is not disturbed even in strong light, with extremely strong anti-interference ability. By combining depth images with color images and using deep learning-based image recognition, it can not only effectively avoid obstacles but also distinguish the positions and types of obstacles in front, providing more accurate guidance for blind users and enhancing safety. The present invention fully considers the characteristics of the blind. By segmenting the depth image to calculate the minimum depth value, it changes the PWM duty cycle of the corresponding area of the alarm device and uses different vibration intensities and frequencies to alert the blind, enabling the blind to effectively obtain the distance and orientation of obstacles through vibration. In addition, for the problem that the traditional SSD object detection method has poor real-time performance in image semantic segmentation on embedded devices with limited computing power, the present invention fully considers the very high real-time requirements for blind obstacle avoidance and optimizes the traditional object detection method, improving the inference time of the algorithm while ensuring the detection accuracy remains unchanged.
[0054] It should be noted that although the above steps are described in a specific order, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.
[0055] The present invention can be a system, method, and / or computer program product. The computer program product can include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement various aspects of the present invention.
[0056] A computer-readable storage medium can be a tangible device that retains and stores instructions for use by an instruction execution device. A computer-readable storage medium can, for example, include, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing.
[0057] The embodiments of the present invention have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.
Claims
1. A method for obstacle avoidance for the blind, characterized in that, The method includes: S1. Obtain the depth image of the current environment; S2. Split the depth image into multiple column regions and process each column region to obtain the minimum depth value in each column region; The step S2 includes: S21. Split the depth image into a preset number of column regions and calculate the depth value of each column region; S22. Filter the depth values of each column region according to the preset maximum depth threshold and minimum depth threshold, remove the depth values outside the range of the maximum depth threshold and the minimum depth threshold, and obtain the minimum depth value of each column region; The maximum depth threshold refers to the farthest distance of the obstacle that needs to be recognized and will affect the blind; The minimum depth threshold refers to the closest distance between the obstacle and the blind that the blind can accept; S3. Based on the minimum depth value of each column region, perform obstacle alarm for each column region respectively; The step S3 includes: S31. Based on the minimum depth value of each column region, obtain the alarm intensity of the corresponding area of the alarm device for each column region in the preset alarm intensity standard; Among them, the alarm intensity standard refers to the alarm intensity corresponding to the preset different depth values; S32. Obtain the PWM duty cycle corresponding to the signal strength frequency according to the alarm intensity of each column area of the alarm device, and control the corresponding areas of the alarm device to alarm the blind of obstacles in a vibrating manner respectively, where the vibration frequency of each column area is proportional to its corresponding PWM duty cycle; among them, the duty cycle of each column area is calculated by the following method: Among them, is the minimum depth value of the current column area, is the maximum value of the depth measurement range.
2. The method according to claim 1, wherein The preset number of the column regions is 9.
3. The method according to any one of claims 1-2, wherein In step S1, obtaining an RGB image is further included; The method further includes: S4. Use a preset target detection model to detect the position and type of obstacles in the RGB image and report them to the blind, wherein the target detection model is obtained by training a neural network with the RGB image as the input and the position and type of obstacles in the image as the output.
4. The method according to claim 3, wherein In step S4, the position and type of the obstacles are broadcast to the blind by voice.
5. The method according to claim 4, wherein The target detection model is obtained in the following manner: P1. Construct an initial target detection model with the res-block network as the backbone network and the squeeze-and-excitation network as the pre-network; P2. Based on a preset RGB image training set and using the following loss function, train the initial target detection model until convergence: Among them, is the softmax loss function, is the Center Loss function, is the weight of the loss function.
6. A blind person obstacle avoidance system for implementing the method according to any one of claims 1-5, characterized in that, The system includes: An image acquisition module, configured to acquire the depth image of the current environment; An image processing module, configured to split the depth image acquired by the image acquisition module into multiple column regions and process each column region to obtain the minimum depth value in each column region; An alarm control module, configured to output a control signal for controlling the alarm device to perform obstacle alarm for the blind according to the minimum depth value of each column region obtained by the image processing module; An alarm device, configured to perform obstacle alarm for each column region respectively according to the control signal of the alarm control module.
7. The system according to claim 6, wherein The alarm device includes a plurality of alarm units, different alarm units correspond to different column regions and each alarm unit corresponds to one column region.
8. The system according to claim 7, wherein The alarm control module is configured to perform the following operations: Obtain the alarm intensity of the corresponding area of the alarm device for each column area based on the minimum depth value of each column area in a preset alarm intensity standard; wherein, the alarm intensity standard refers to the alarm intensity corresponding to different preset depth values. Obtain the PWM duty cycle corresponding to the signal intensity frequency according to the alarm intensity of each area of the alarm device, and output a control signal for controlling the alarm unit of the corresponding area of the alarm device to vibrate at a vibration frequency for obstacle alarm to the blind based on the PWM duty cycle of each area. Among them, the vibration frequency of the alarm unit in each area is proportional to its corresponding PWM duty cycle. Among them, the duty cycle of each column area is calculated by the following method: Among them, is the minimum depth value of the current column area, is the maximum value of the depth measurement range.
9. The system according to any one of claims 7-8, wherein The image acquisition module is further configured to acquire an RGB image of the current environment. The system further includes: An obstacle information acquisition module for acquiring the position and type of obstacles in the environment where the blind person is located based on the RGB image and voice-broadcasting them to the blind person.
10. The system according to claim 9, characterized in that The obstacle information acquisition module includes: A target detection model for detecting the position and type of obstacles in the environment where the blind person is located. Among them, the target detection model is a model obtained by training a neural network with an RGB image as the input and the position and type of obstacles in the image as the output. A voice module for broadcasting the position and type of obstacles detected by the target detection module to the blind person.
11. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 5.
12. An electronic device, characterized in that, Including: One or more processors; A storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device realizes the steps of the method according to any one of claims 1 to 5.