Bluetooth glasses integrated with computer vision software algorithm

Abstract

The invention discloses a pair of Bluetooth glasses integrated with a computer vision software algorithm, and relates to the technical field of computer vision. Identifying the position of the pedestrian zebra crossing, and guiding the visual and auditory disorder user to reach the position of the pedestrian zebra crossing; when the visual and auditory disorder user is identified to arrive at the position of the pedestrian zebra crossing, identifying the color of the traffic signal lamp; detecting coming vehicles, and judging whether the visual and auditory disorder user can safely cross the road or not according to the detected distance and speed of the vehicles; and comparing and analyzing the distance with a preset safe distance, and guiding the visual and auditory disorder user to keep the safe distance from the pedestrian according to an analysis result. According to the invention, the problem that existing Bluetooth glasses cannot accurately provide instant feedback for people with visual and auditory impairment through image recognition is solved, and it can be ensured that a user obtains timely and clear auxiliary information when needed through image recognition and audio prompt, so that the people with visual and auditory impairment can pass through a traffic intersection more safely.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology. More specifically, the present invention relates to a Bluetooth glasses integrated with computer vision software algorithms. Background Art

[0002] Currently, Bluetooth glasses are mainly used to implement basic functions such as audio playback and calls, providing users with a convenient mobile audio experience. However, for visually and hearing-impaired people, they face many difficulties and risks during travel. Especially in traffic scenarios, it is difficult for them to accurately obtain information such as the color of traffic lights, the location of pedestrian crosswalks, and the surrounding vehicles and pedestrians. Traditional assistive devices such as blind canes, although they can provide certain help, their functions are very limited for complex traffic environments. Therefore, it is of great practical significance to develop a device that can provide comprehensive traffic information for visually and hearing-impaired users.

[0003] Existing Bluetooth glasses cannot accurately provide instant feedback or assistance for visually and hearing-impaired people through image recognition. However, Bluetooth glasses integrated with computer vision software algorithms can ensure that users obtain timely and clear auxiliary information when needed through image recognition and audio prompts, enabling visually and hearing-impaired people to pass through traffic intersections more safely. The present invention specifically relates to a Bluetooth glasses integrated with computer vision software algorithms, especially suitable for visually and hearing-impaired users, which can identify the color of traffic lights, pedestrian crosswalks, detect oncoming vehicles through image recognition and help users maintain a safe distance from pedestrians.

[0004] In view of the above problems, the present invention proposes a solution. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a Bluetooth glasses integrated with computer vision software algorithms, which can identify the color of traffic lights, pedestrian crosswalks, detect oncoming vehicles through image recognition and help users maintain a safe distance from pedestrians to solve the problems raised in the above background art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A Bluetooth glasses integrated with computer vision software algorithms includes the following steps:

[0008] S1, Real-time collect the surrounding environment images through the wide-angle camera on the Bluetooth glasses;

[0009] S2, Process the collected surrounding environment images, identify the location of the pedestrian crosswalk, and guide the visually and hearing-impaired user to the location of the pedestrian crosswalk;

[0010] S3. When it is recognized that the visually and auditorily impaired user has reached the location of the pedestrian crossing, identify the color of the traffic light.

[0011] S4. If the traffic light is recognized as red, send a waiting prompt to the visually and auditorily impaired user; if the traffic light is recognized as green, conduct oncoming vehicle detection, and judge whether the visually and auditorily impaired user can safely cross the road based on the detected vehicle distance and speed.

[0012] If the judgment result is that it is not safe to cross, return to S3.

[0013] If the judgment result is that it is safe to cross, remind the visually and auditorily impaired user that they can cross the road, and then execute S5.

[0014] S5. During the process of the visually and auditorily impaired user crossing the road, identify nearby pedestrians through the surrounding environment image, calculate the distance between the visually and auditorily impaired user and the pedestrians, and compare and analyze this distance with the preset safety distance, and guide the visually and auditorily impaired user to keep a safe distance from the pedestrians according to the analysis result.

[0015] In a preferred embodiment, the image processing of the collected surrounding environment image includes grayscale conversion of the collected image, using Gaussian filtering to remove image noise, enhancing image contrast, and extracting the edge information of the image.

[0016] In a preferred embodiment, the process of grayscale conversion of the collected image is as follows: Convert the color image to a grayscale image to reduce the data volume, and the grayscale conversion expression is:

[0017] H = 0.299×R + 0.587×G + 0.114×B;

[0018] In the formula, H is the converted grayscale value, and R, G, and B are the red, green, and blue channel values of the image pixels respectively;

[0019] Calculate each pixel in the image through this formula to obtain the grayscale image;

[0020] The process of using Gaussian filtering to remove image noise is as follows:

[0021] For each pixel in the image, calculate the weighted average of the pixel values within the pixel neighborhood to obtain the filtered pixel value;

[0022] Construct a Gaussian kernel and perform a convolution operation on each pixel in the image and its neighboring pixels;

[0023] Multiply the Gaussian kernel by the pixel values in the corresponding neighborhood in the image and sum them to obtain the filtered pixel value, thereby achieving the removal of image noise;

[0024] The process of enhancing image contrast is as follows:

[0025] By adjusting the histogram of the image, the gray distribution of the image is made more uniform, thereby enhancing the contrast of the image.

[0026] In a preferred embodiment, the process of extracting the edge information of the image is as follows:

[0027] The Canny edge detection algorithm is used to perform Gaussian smoothing on the image to reduce the influence of noise on edge detection;

[0028] Then the Sobel operator is used to calculate the gradients in the x-direction and y-direction respectively;

[0029] For each pixel (x, y) in the image, it is convolved with the Sobel templates in the x-direction and y-direction respectively to obtain G x (x, y) and G y (x, y). Then the calculation formulas for the gradient magnitude G(x, y) and the gradient direction θ(x, y) are:

[0030]

[0031] For each pixel, its gradient magnitude is compared with the gradient magnitudes of adjacent pixels along the gradient direction. If the gradient magnitude of this pixel is not a local maximum, it is set to 0, that is, this pixel is suppressed and the real edge pixels are retained;

[0032] Two thresholds T low and T high are set, where T low < T high ;

[0033] If the gradient magnitude of a pixel is greater than T high , it is marked as a strong edge pixel;

[0034] If the gradient magnitude of a pixel is less than T low , it is discarded;

[0035] If the gradient magnitude of a pixel is between T low and T high , it is retained as an edge pixel only when this pixel is connected to a strong edge pixel;

[0036] After double-threshold processing, the edge information in the image is obtained.

[0037] In a preferred embodiment, the process of identifying the location of the pedestrian zebra crossing is as follows:

[0038] The Hough transform is used to convert the description of the line from the Cartesian coordinate system (x, y) to the polar coordinate system (ρ, θ) to find the lines in the image;

[0039] In the parameter space (ρ, θ), record the value of each edge point corresponding to the straight line and accumulate these values;

[0040] For each edge point in the image, calculate all possible straight line parameters through the polar coordinate transformation formula;

[0041] Generate a two-dimensional accumulator array A to represent the occurrence times of the straight line parameters;

[0042] Use the straight line parameter value of each edge point as the index of the accumulator and increment the value at the corresponding position by 1;

[0043] Determine the parameters of the straight line by finding the peak value in the accumulator in the parameter space, and filter out the pedestrian zebra crossing;

[0044] According to the characteristics of the zebra crossing being parallel and equidistant, group and verify the detected straight lines, calculate the angle between two straight lines. If the angle is less than the preset threshold, the two straight lines are considered approximately parallel;

[0045] And calculate the distance between two parallel straight lines. According to the set distance threshold range, filter the parallel straight lines, group the straight lines that meet the equidistant characteristics into one group, and determine the zebra crossing area;

[0046] And calculate the center position and boundary position of the zebra crossing area, obtain its coordinate range in the image, and further obtain the zebra crossing position information.

[0047] In a preferred embodiment, the process of identifying the color of the traffic signal is as follows:

[0048] Use a deep learning-based object detection algorithm to locate the position of the traffic signal in the image;

[0049] Perform inference on the image through the trained object detection model, output the bounding box coordinates of the signal lamp, and extract the area of the signal lamp in the image;

[0050] Convert the image of the signal lamp area from the RGB color space to the HSV color space;

[0051] According to the value ranges of different colors in the HSV space, set the thresholds for the three colors of red, green, and yellow;

[0052] And count the number of pixels in the signal lamp area that fall within the threshold ranges of each color. The color with the largest number of pixels is the color of the signal lamp.

[0053] In a preferred embodiment, the process of detecting oncoming vehicles is as follows:

[0054] Use the YOLOv5 algorithm based on deep learning to detect vehicles in the image and output the bounding box coordinates and class information of the vehicles;

[0055] Use the scale estimation method of binocular vision to calculate the distance between the vehicle and the user. Given the actual height H of the vehicle, the pixel height h of the vehicle in the image, and the focal length f of the camera, the formula for calculating the distance d between the vehicle and the camera is:

[0056]

[0057] In two consecutive frames of images, by tracking the position change of the same vehicle, calculate the displacement Δx of the vehicle between the two frames, and combine the time interval Δt between the two frames to obtain the vehicle speed v.

[0058] In a preferred embodiment, the process of judging whether a visually and auditorily impaired user can safely cross the road according to the detected vehicle distance and speed is as follows:

[0059] Estimate the time t required for the user to cross the road according to the total length of the zebra crossing and the user's normal walking speed;

[0060] Calculate the time T required for the vehicle to reach the zebra crossing according to the vehicle speed v and the distance d between the vehicle and the camera;

[0061] Compare the time t required for the user to cross the road with the time T required for the vehicle to reach the zebra crossing;

[0062] If the time required for the user to cross the road is less than the time required for the vehicle to reach the zebra crossing, the judgment result is that it is not safe to pass;

[0063] If the time required for the user to cross the road is greater than the time required for the vehicle to reach the zebra crossing, the judgment result is that it is safe to pass, and remind the visually and auditorily impaired user that they can cross the road.

[0064] In a preferred embodiment, the specific process of comparing and analyzing the distance with a preset safety distance and guiding the visually and auditorily impaired user to keep a safe distance from pedestrians according to the analysis result is as follows:

[0065] Use a pedestrian detection model based on a convolutional neural network to detect pedestrians in the image and output the bounding box coordinates of the pedestrians;

[0066] Use the binocular vision method to calculate the distance L between the user and the pedestrian;

[0067] And compare the distance L between the user and the pedestrian with the preset safety distance S;

[0068] If the distance L between the user and the pedestrian is less than the preset safety distance S, it is judged that the distance between the user and the pedestrian is too close and the position needs to be adjusted, and a voice prompt is sent to the user through the Bluetooth headset to guide the user to keep a safe distance.

[0069] In a preferred embodiment, it includes an acquisition module, a processing module, an identification module, and an adjustment module, and there are connections between the modules:

[0070] The acquisition module is used to collect real-time images of the surrounding environment through the wide-angle camera on the Bluetooth glasses;

[0071] The processing module is used to process the collected images of the surrounding environment, identify the location of the pedestrian zebra crossing, and guide visually and auditorily impaired users to the location of the pedestrian zebra crossing;

[0072] The identification module is used to identify the color of the traffic signal when it is recognized that the visually and auditorily impaired user has reached the location of the pedestrian zebra crossing;

[0073] If the traffic signal is recognized as red, a waiting prompt is sent to the visually and auditorily impaired user;

[0074] If the traffic signal is recognized as green, vehicle detection is performed, and it is judged whether the visually and auditorily impaired user can safely cross the road according to the detected vehicle distance and speed;

[0075] The adjustment module is used to identify nearby pedestrians through the surrounding environment images during the process of the visually and auditorily impaired user crossing the road, calculate the distance between the visually and auditorily impaired user and the pedestrians, and compare and analyze this distance with a preset safety distance, and guide the visually and auditorily impaired user to keep a safe distance from the pedestrians according to the analysis result.

[0076] Technical effects and advantages of a Bluetooth glasses integrating computer vision software algorithms of the present invention:

[0077] 1. By collecting real-time images of the surrounding environment and through the integrated computer vision software algorithm, the present invention can instantly process and analyze the images without the user having to operate other devices additionally. During walking, the glasses can quickly identify whether there is a zebra crossing ahead. Compared with using a mobile phone to view a map or using other auxiliary devices, it greatly improves the convenience of information acquisition, enabling the user to know key environmental information without having to distract to operate other tools during the movement. The Bluetooth glasses are small in size, light in weight, comfortable to wear and do not cause too much burden on the user's movement. Compared with traditional large-scale image acquisition and analysis devices, users can carry and use them anytime and anywhere. Whether it is for daily travel, sports or in complex outdoor environments, they can easily utilize the functions of the glasses to meet the need for identifying surrounding environment information. For those with poor eyesight, the zebra crossing recognition function of the Bluetooth glasses is particularly important.

[0078] 2. By accurately detecting the position of the zebra crossing and guiding the user to the zebra crossing in the form of voice prompts, the present invention can effectively prevent pedestrians from randomly crossing the road due to failure to detect the zebra crossing in time, reducing the risk of traffic accidents. For example, in the night or under adverse weather conditions, traditional vision is greatly affected, while the algorithm of the glasses can work stably based on image analysis to ensure the user's safe passage across the road. The computer vision algorithm of the glasses can not only identify the zebra crossing, but also be further extended to the identification of other potential dangers. Once an abnormal situation is detected, it can promptly prompt the user through voice and enable the user to take corresponding measures in advance, providing all-round protection for the user's travel safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a schematic structural diagram of a method of a Bluetooth glasses integrating a computer vision software algorithm according to the present invention.

[0080] Figure 2 It is a schematic structural diagram of a system of a Bluetooth glasses integrating a computer vision software algorithm according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0081] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0082] Embodiment 1 Figure 1 A Bluetooth glasses integrating a computer vision software algorithm according to the present invention is provided.

[0083] S1. The surrounding environment image is collected in real time through the wide-angle camera on the Bluetooth glasses;

[0084] The hardware part of the Bluetooth glasses includes a frame, lenses, a wide-angle camera, a microphone, a main control chip, a Bluetooth module, a speaker, a bone conduction component, and a power module. The frame and lenses are of a conventional glasses structure to meet the user's daily vision correction or decoration needs; the wide-angle camera should have a wide field of view and good resolution to ensure clear capture of the surrounding environment image; the microphones are distributed on both sides of the frame for collecting the surrounding environment sound signals; the main control chip, as the core operation unit, is built into the frame and is responsible for processing data; the Bluetooth module realizes wireless data transmission with external intelligent devices such as mobile phones and tablet computers; the speaker is designed in a miniaturized manner and is placed near the ear for playing the processed sound; a bone conduction component is equipped to directly transmit the sound to the inner ear through skull vibration to adapt to the hearing needs and wearing habits of different users; the power module uses a rechargeable small lithium battery to supply power to the entire device.

[0085] The main control chip of the Bluetooth glasses sends an initialization instruction to the camera to set the basic parameters of the camera;

[0086] When the user turns on the Bluetooth glasses, the power module supplies power to the wide-angle camera and other components;

[0087] The surrounding ambient light enters the camera through the wide-angle lens. The lens converges and corrects the light, reduces aberration and distortion, and focuses the light on the image sensor;

[0088] The photosensitive pixels of the image sensor convert the light into an electrical signal. The stronger the light, the more the charge quantity. The camera uses the built-in image sensor to collect the surrounding environment image in real time.

[0089] S2. Perform data processing on the collected surrounding environment image, identify the location of the pedestrian zebra crossing, and guide the visually and hearing impaired user to the location of the pedestrian zebra crossing;

[0090] Grayscale the collected image to convert the color image into a grayscale image to reduce the data volume. The grayscale conversion formula is:

[0091] H = 0.299×R + 0.587×G + 0.114×B;

[0092] In the formula, H is the converted grayscale value, and R, G, and B are the red, green, and blue channel values of the image pixel respectively;

[0093] By calculating each pixel in the image with this formula, the grayscale image can be obtained;

[0094] Gaussian filtering is a linear smoothing filter used to reduce noise in the image. The process of using Gaussian filtering to remove image noise is as follows:

[0095] For each pixel in the image, calculate the weighted average of the pixel values in the pixel neighborhood to obtain the filtered pixel value;

[0096] Construct a Gaussian kernel and perform a convolution operation on each pixel in the image and its neighboring pixels;

[0097] Then multiply the Gaussian kernel by the pixel values in the corresponding neighborhood in the image and sum them to obtain the filtered pixel value, thereby achieving the removal of image noise.

[0098] And enhance the image contrast through histogram equalization; Histogram equalization is to adjust the histogram of the image to make the grayscale distribution of the image more uniform, thereby enhancing the image contrast.

[0099] Adopt the Canny edge detection algorithm. First, perform Gaussian smoothing processing on the image to further reduce the influence of noise on edge detection;

[0100] Then use the Sobel operator to calculate the gradients in the x - direction and y - direction respectively;

[0101] For each pixel (x, y) in the image, convolve it with the Sobel templates in the x - direction and y - direction respectively to obtain G x (x, y) and G y (x, y). Then the calculation formulas for the gradient magnitude G(x, y) and the gradient direction θ(x, y) are:

[0102]

[0103]

[0104] For each pixel, compare its gradient magnitude with the gradient magnitudes of adjacent pixels along the gradient direction. If the gradient magnitude of this pixel is not a local maximum, set it to 0, that is, suppress this pixel and retain the real edge pixels.

[0105] Set two thresholds T low and T high , where T low <T high ;

[0106] If the gradient magnitude of a pixel is greater than T high , mark it as a strong edge pixel;

[0107] If the gradient magnitude of a pixel is less than T low , discard it;

[0108] If the gradient magnitude of a pixel is between T low and T high , only retain it as an edge pixel when this pixel is connected to a strong edge pixel;

[0109] After double - threshold processing, the edge information in the image is obtained.

[0110] The Hough transform is a method for detecting straight lines in an image. It detects straight lines by transforming the straight lines in the image space into the parameter space for detection;

[0111] Apply the Hough transform to convert the description of the straight line from the Cartesian coordinate system (x, y) to the polar coordinate system (ρ, θ) to find the straight lines in the image;

[0112] In the parameter space (ρ, θ), record the values corresponding to each edge point for the straight line and accumulate these values;

[0113] For each edge point in the image, calculate all possible straight - line parameters through the polar - coordinate conversion formula;

[0114] Generate a two-dimensional accumulator array A to represent the occurrence times of the line parameters;

[0115] Use the line parameter values of each edge point as the index of the accumulator, and increment the value at the corresponding position by 1;

[0116] Determine the line parameters by finding the peak value in the accumulator in the parameter space, and filter out the pedestrian crosswalks;

[0117] According to the characteristics of parallelism and equal spacing of the crosswalks, group and verify the detected lines, calculate the angle between two lines. If the angle is less than the preset threshold, the two lines are considered approximately parallel;

[0118] And calculate the distance between two parallel lines. According to the set distance threshold range, filter the parallel lines, group the lines that meet the equal-spacing characteristics into a group, and determine the crosswalk area;

[0119] And calculate the central position and boundary position of the crosswalk area, obtain its coordinate range in the image, and then obtain the crosswalk position information;

[0120] Transmit the crosswalk position information to the user's receiving device via Bluetooth, and guide the user to the crosswalk in the form of voice prompts, such as "There is a crosswalk on the left front, please walk towards the left front".

[0121] S3. When it is recognized that the visually and auditorily impaired user arrives at the location of the pedestrian crosswalk, identify the color of the traffic signal;

[0122] Use a deep learning-based object detection algorithm to locate the position of the traffic signal in the image. The process includes:

[0123] Perform inference on the image through the trained object detection model, output the bounding box coordinates of the signal lamp, and extract the area of the signal lamp in the image;

[0124] Since color recognition is more stable in the HSV space, especially for changes under different lighting conditions;

[0125] Convert the image of the signal lamp area from the RGB color space to the HSV color space;

[0126] Set the thresholds for the three colors of red, green, and yellow according to the value ranges of different colors in the HSV space;

[0127] And count the number of pixels in the signal lamp area that fall within the range of each color threshold. The color with the largest number of pixels is the color of the signal lamp.

[0128] S4. If the traffic signal is recognized as red, send a waiting prompt to the visually and auditorily impaired user;

[0129] When it is determined that the signal light color is red, the data processing module transmits this information to the user's receiving device via Bluetooth and issues a voice prompt, such as "It is a red light now. Please wait."

[0130] If the traffic signal light is recognized as green, vehicle detection is performed, and it is judged whether the visually and hearing-impaired user can safely cross the road based on the detected vehicle distance and speed;

[0131] Use the YOLOv5 algorithm based on deep learning to detect vehicles in the image and output the bounding box coordinates and class information of the vehicles;

[0132] Use the scale estimation method of binocular vision to calculate the distance between the vehicle and the user. Given the actual height H of the vehicle, the pixel height h of the vehicle in the image, and the focal length f of the camera, the formula for calculating the distance d between the vehicle and the camera is:

[0133]

[0134] In two consecutive frames of images, by tracking the position change of the same vehicle, calculate the displacement Δx of the vehicle between the two frames, and combine the time interval Δt between the two frames to obtain the calculation formula for the vehicle speed v as follows:

[0135]

[0136] Estimate the time t required for the user to cross the road based on the total length of the zebra crossing and the user's normal walking speed;

[0137] Calculate the time T required for the vehicle to reach the zebra crossing based on the vehicle speed v and the distance d between the vehicle and the camera;

[0138] Compare the time required for the user to cross the road with the time required for the vehicle to reach the zebra crossing;

[0139] If the time required for the user to cross the road is less than the time required for the vehicle to reach the zebra crossing, the judgment result is unsafe to pass, and then return to S3;

[0140] If the time required for the user to cross the road is greater than the time required for the vehicle to reach the zebra crossing, the judgment result is safe to pass, and remind the visually and hearing-impaired user that they can cross the road, and then execute S5.

[0141] With the help of advanced image recognition technology, the Bluetooth glasses can monitor the dynamics of vehicles on the surrounding roads in real time. Continuously calculate the distance between the vehicle and the visually and hearing-impaired user, and obtain the driving speed information of the vehicle. Once it detects that a vehicle is approaching and its driving situation may pose a threat to the user crossing the road, it will immediately issue a clear voice alarm, such as "There is a vehicle approaching rapidly on the left ahead. Do not move forward." This enables the visually and hearing-impaired user to perceive potential dangers in a timely manner, avoid rashly stepping into the road when the vehicle is approaching, thus effectively avoiding serious traffic accidents and ensuring their own life safety.

[0142] At busy intersections, vehicles move in various directions and at different speeds. The oncoming vehicle detection function can track vehicles in multiple directions simultaneously and comprehensively analyze their distances and speeds. For example, on a multi-lane main road, there are both straight-going vehicles and turning vehicles. Through precise calculations, the glasses can enable the user to understand the approaching situation of vehicles in all directions, helping them choose a safe time to cross the road and allowing them to cross the road safely even in complex traffic conditions.

[0143] Visually and hearing-impaired users can independently judge whether it is safe to cross the road based on the distance and speed information of the vehicle. When it detects that the vehicle is at a relatively far distance and slow speed, within a safe range, the user can confidently start to cross the road; conversely, if the vehicle is at a relatively close distance and fast speed, the user can choose to wait. This autonomous decision-making based on accurate information greatly enhances the user's sense of control over the travel process, enabling them to complete the daily travel task of crossing the road relatively independently without the assistance of others and improving travel autonomy. By accurately judging the vehicle situation, visually and hearing-impaired users can better integrate into the normal traffic travel rhythm. After the green light comes on, according to the oncoming vehicle detection results, they can start to cross the road in a timely manner like ordinary pedestrians, coordinating with the passing rhythm of surrounding pedestrians and vehicles, no longer waiting for a long time or acting rashly due to the inability to accurately judge the road conditions, thus improving travel efficiency and the ability to integrate into social travel.

[0144] For visually and hearing-impaired users, crossing the road is often accompanied by great psychological pressure and uncertainty. The existence of the oncoming vehicle detection function enables them to understand the situation of vehicles on the road in real time and no longer feel excessive anxiety due to ignorance of the oncoming vehicle situation. For example, when waiting to cross the road, clear vehicle distance and speed prompts allow the user to wait patiently for the right time, relieving the tension when waiting at the road intersection and enhancing the psychological comfort of travel. Continuous and accurate oncoming vehicle detection feedback gives visually and hearing-impaired users a more reliable perception of the safety of crossing the road. Each experience of safely crossing the road further enhances their confidence in independent travel, encouraging them to go out more often and participate in social activities, expanding their living space.

[0145] S5. During the process of visually and auditorily impaired users crossing the road, nearby pedestrians are recognized through the surrounding environment images, the distance between the visually and auditorily impaired users and the pedestrians is calculated, and this distance is compared and analyzed with a preset safe distance. According to the analysis results, the visually and auditorily impaired users are guided to keep a safe distance from the pedestrians.

[0146] Use a pedestrian detection model based on a convolutional neural network to detect pedestrians in the image and output the bounding box coordinates of the pedestrians;

[0147] Use the binocular vision method to calculate the distance L between the user and the pedestrian;

[0148] And compare the distance L between the user and the pedestrian with the preset safe distance S;

[0149] If the distance L between the user and the pedestrian is less than the preset safe distance S, it is judged that the distance between the user and the pedestrian is too close and the position needs to be adjusted. A voice prompt is sent to the user through the speaker, such as "The pedestrian in front is too close, please avoid to the right", to guide the user to keep a safe distance.

[0150] For visually and auditorily impaired users, depending on a variety of preset hearing loss models, parameters such as the frequency and amplitude of the voice prompt are precisely adjusted. For users with high-frequency hearing loss, the algorithm automatically increases the gain of the high-frequency sound; for users with low-frequency hearing loss, the low-frequency sound signal is emphasized. At the same time, the sound signal collected by the microphone is analyzed in real time through the hearing aid algorithm, and the surrounding environmental noise level is monitored in real time. When in a noisy traffic environment or a bustling public place, the algorithm automatically enhances the noise reduction effect and optimizes the sound processing strategy to highlight key sound signals such as human voices; in a quiet indoor environment, the noise reduction intensity is reduced to ensure the natural restoration of the sound.

[0151] For visually and auditorily impaired users, knowing the distance information from the surrounding pedestrians can give them a clearer understanding of their surrounding environment, thereby enhancing their sense of control over the action of crossing the road. For example, when waiting to cross the road, they can understand the distribution of the surrounding pedestrians through the distance prompt and no longer feel anxious due to uncertainty about the surrounding situation. This sense of control helps to enhance the user's self-confidence and makes them more willing to go out independently and integrate into social life.

[0152] Through image recognition technology, the surrounding pedestrians are monitored in real time, and the distance between the visually and auditorily impaired users and the pedestrians is continuously calculated. When the distance approaches the preset safe distance, the Bluetooth glasses can quickly send a voice prompt, such as "There is a pedestrian approaching on the right in front, please keep a safe distance". This allows the visually and auditorily impaired users to promptly detect potential collision risks and make avoidance actions in advance, effectively avoiding collisions with other pedestrians and ensuring their own safety.

[0153] At a traffic intersection, the pedestrian flow is large and the walking directions are complex. At this time, this function can simultaneously identify multiple pedestrians and calculate the distances to the visually and hearing-impaired users respectively. For example, when the green light is on and pedestrians are passing, there may be pedestrians walking in multiple directions around the visually and hearing-impaired user. By analyzing the distances, the glasses can help the user find a safe walking path in the complex crowd, avoiding being interfered with or even collided by pedestrians from different directions, greatly improving the safety of crossing the road in complex traffic scenarios.

[0154] Based on the comparison result between the detected distance and the safe distance, the visually and hearing-impaired user can independently decide when to move forward and when to pause. For example, if it is detected that a pedestrian is approaching quickly within the safe distance, the user can choose to pause and wait for the pedestrian to pass; if the surrounding pedestrians are at a relatively far distance and within the safe range, the user can cross the road at ease according to the original rhythm. This improvement in the ability of independent decision-making makes the visually and hearing-impaired user more calm when crossing the road and reduces their dependence on others.

[0155] Avoiding close contact and possible collisions with pedestrians can provide a more comfortable experience for the visually and hearing-impaired user during the process of crossing the road. They will not feel nervous due to the worry of collision, and the walking process will also be smoother and more natural. For example, when crossing the road at a busy commercial street intersection, continuous distance reminders can help the user maintain a comfortable walking rhythm without disturbing the passage of others, improving the overall comfort of travel.

[0156] By maintaining a safe distance from the surrounding pedestrians, the visually and hearing-impaired user can better integrate into the normal pedestrian traffic rhythm. During the green light time, they can keep an appropriate interval from other pedestrians according to the distance reminder, neither obstructing the traffic nor having trouble passing the road smoothly. This helps them travel in an orderly manner according to traffic rules like ordinary pedestrians, further improving the convenience of life and travel efficiency.

[0157] Embodiment 2, a Bluetooth glasses integrated with a computer vision software algorithm, includes an acquisition module, a processing module, an identification module, and an adjustment module, and there are connections between the modules:

[0158] The acquisition module is used to collect real-time images of the surrounding environment through the wide-angle camera on the Bluetooth glasses;

[0159] The processing module is used to process the collected images of the surrounding environment, identify the location of the pedestrian crosswalk, and guide the visually and hearing-impaired user to the location of the pedestrian crosswalk;

[0160] The identification module is used to identify the color of the traffic signal when it is recognized that the visually and hearing-impaired user has reached the location of the pedestrian crosswalk;

[0161] The detection module, if it identifies that the traffic signal is red, issues a waiting prompt to the visually and hearing-impaired user;

[0162] If the traffic signal is recognized as green, vehicle detection is performed, and it is determined whether the visually and auditorily impaired user can safely cross the road based on the detected vehicle distance and speed.

[0163] An adjustment module is configured to, during the process of a visually and auditorily impaired user crossing the road, recognize nearby pedestrians through the surrounding environment image, calculate the distance between the visually and auditorily impaired user and the pedestrians, and compare and analyze this distance with a preset safe distance, and guide the visually and auditorily impaired user to maintain a safe distance from the pedestrians according to the analysis result.

[0164] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to obtain a formula that is closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0165] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product.

[0166] Those of ordinary skill in the art can realize that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0167] In addition, in each embodiment of this application, the functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0168] As described above, only the specific implementation manners of this application are provided, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0169] Finally: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A Bluetooth glasses integrated with computer vision software algorithms, characterized in that, It includes the following steps: S1, real-time collect the surrounding environment images through the wide-angle camera on the Bluetooth glasses; S2, perform image processing on the collected surrounding environment images, identify the location of the pedestrian zebra crossing, and guide the visually and hearing impaired user to the location of the pedestrian zebra crossing; S3, when it is recognized that the visually and hearing impaired user arrives at the location of the pedestrian zebra crossing, identify the color of the traffic light; S4, if the traffic light is recognized as red, send a waiting prompt to the visually and hearing impaired user; if the traffic light is recognized as green, perform oncoming vehicle detection, and judge whether the visually and hearing impaired user can safely cross the road according to the detected vehicle distance and speed; If the judgment result is that it is not safe to cross, return to S3; If the judgment result is that it is safe to cross, remind the visually and hearing impaired user that they can cross the road, then execute S5; S5, during the process of the visually and hearing impaired user crossing the road, identify nearby pedestrians through the surrounding environment images, calculate the distance between the visually and hearing impaired user and the pedestrians, and compare and analyze this distance with a preset safety distance, and guide the visually and hearing impaired user to keep a safe distance from the pedestrians according to the analysis result.

2. The Bluetooth glasses integrated with a computer vision software algorithm according to claim 1, characterized in that, The performing image processing on the collected surrounding environment images includes grayscale processing of the collected images, using Gaussian filtering to remove image noise, enhancing image contrast, and extracting the edge information of the images.

3. The Bluetooth glasses integrated with a computer vision software algorithm according to claim 2, characterized in that, The process of grayscale processing the collected images is as follows: Convert the color image to a grayscale image to reduce the data volume, and the grayscale expression is: H = 0.299×R + 0.587×G + 0.114×B; In the formula, H is the converted grayscale value, and R, G, and B are the red, green, and blue channel values of the image pixels respectively; Calculate each pixel in the image through this formula to obtain the grayscale image; The process of using Gaussian filtering to remove image noise is as follows: For each pixel in the image, calculate the weighted average of the pixel values within the pixel neighborhood to obtain the filtered pixel value; Construct a Gaussian kernel and perform a convolution operation on each pixel in the image and its neighborhood pixels; Multiply the Gaussian kernel by the pixel values of the corresponding neighborhood in the image and sum them to obtain the filtered pixel value, thereby realizing the removal of image noise; The process of enhancing image contrast is as follows: Adjust the histogram of the image to make the grayscale distribution of the image more uniform, thereby enhancing the contrast of the image.

4. The Bluetooth glasses integrating the computer vision software algorithm according to claim 3, characterized in that, The process of extracting the edge information of the images is as follows: Adopt the Canny edge detection algorithm to perform Gaussian smoothing processing on the image to reduce the influence of noise on edge detection; Then use the Sobel operator to calculate the gradients in the x direction and y direction respectively; For each pixel (x, y) in the image, it is convolved with the Sobel templates in the x - direction and y - direction respectively to obtain G x (x, y) and G y (x, y). Then the calculation formulas for the gradient magnitude G(x, y) and the gradient direction θ(x, y) are as follows: For each pixel, compare its gradient magnitude with the gradient magnitudes of adjacent pixels along the gradient direction. If the gradient magnitude of this pixel is not a local maximum, set it to 0, that is, suppress this pixel and retain the real edge pixels; Set two thresholds T low and T high , where T low <T high ; If the gradient magnitude of a pixel is greater than T high , then it is marked as a strong edge pixel; If the gradient magnitude of a pixel is less than T low , it is discarded; If the gradient magnitude of a pixel is between T low and T high then the pixel is retained as an edge pixel only if it is connected to a strong edge pixel; After double-threshold processing, obtain the edge information in the image.

5. The Bluetooth glasses integrated with a computer vision software algorithm according to claim 4, characterized in that, The process of identifying the location of the pedestrian zebra crossing is as follows: Use the Hough transform to convert the description of the straight line from the Cartesian coordinate system (x, y) to the polar coordinate system (ρ, θ) to find the straight lines in the image; In the parameter space (ρ, θ), record the values of each edge point corresponding to the line and accumulate these values; For each edge point in the image, calculate all possible line parameters through the polar coordinate transformation formula; Generate a two-dimensional accumulator array A representing the number of occurrences of line parameters; Use the line parameter values of each edge point as the index of the accumulator and increment the value at the corresponding position by 1; Determine the line parameters by finding the peak value in the accumulator in the parameter space, and filter out the pedestrian crosswalks; According to the characteristics of parallelism and equal spacing of the crosswalks, group and verify the detected lines, calculate the angle between two lines, and if the angle is less than the preset threshold, the two lines are considered approximately parallel; Calculate the distance between two parallel lines, and according to the set distance threshold range, filter the parallel lines, group the lines that meet the equal-spacing characteristics into a group, and determine the crosswalk area; Calculate the central position and boundary position of the crosswalk area, obtain its coordinate range in the image, and then obtain the crosswalk position information.

6. The Bluetooth glasses integrating a computer vision software algorithm according to claim 5, characterized in that, The process of identifying the color of traffic lights is as follows: Use a deep learning-based object detection algorithm to locate the position of traffic lights in the image; Infer the image through the trained object detection model, output the bounding box coordinates of the traffic lights, and extract the area of the traffic lights in the image; Convert the image of the traffic light area from the RGB color space to the HSV color space; According to the value ranges of different colors in the HSV space, set the thresholds for red, green, and yellow; Count the number of pixels in the traffic light area that fall within the threshold ranges of each color, and the color with the largest number of pixels is the color of the traffic light.

7. The Bluetooth glasses integrating a computer vision software algorithm according to claim 6, characterized in that The process of detecting oncoming vehicles is as follows: Use the YOLOv5 algorithm based on deep learning to detect vehicles in the image and output the bounding box coordinates and class information of the vehicles; Use the scale estimation method of binocular vision to calculate the distance between the vehicle and the user. Given the actual height H of the vehicle, the pixel height h of the vehicle in the image, and the focal length f of the camera, the formula for calculating the distance d between the vehicle and the camera is: In two consecutive frames of images, by tracking the position change of the same vehicle, calculate the displacement Δx of the vehicle between the two frames, and combine the time interval Δt between the two frames to obtain the vehicle speed v.

8. A Bluetooth glasses integrated with a computer vision software algorithm according to claim 7, characterized in that The process of judging whether a visually and hearing-impaired user can safely cross the road based on the detected vehicle distance and speed is as follows: Estimate the time t required for the user to cross the road according to the total length of the crosswalk and the normal walking speed of the user; Calculate the time T required for the vehicle to reach the crosswalk according to the vehicle speed v and the distance d between the vehicle and the camera; Compare the time t required for the user to cross the road with the time T required for the vehicle to reach the crosswalk; If the time required for the user to cross the road is less than the time required for the vehicle to reach the crosswalk, the judgment result is that it is not safe to pass; If the time required for the user to cross the road is greater than the time required for the vehicle to reach the crosswalk, the judgment result is that it is safe to pass, and remind the visually and hearing-impaired user that they can cross the road.

9. The Bluetooth glasses integrating the computer vision software algorithm according to claim 8, characterized in that The specific process of comparing this distance with a preset safe distance and guiding the visually and hearing-impaired user to keep a safe distance from pedestrians according to the analysis result is as follows: Detect pedestrians in the image using a pedestrian detection model based on a convolutional neural network and output the bounding box coordinates of the pedestrians; Calculate the distance L between the user and the pedestrian using the binocular vision method; Compare the distance L between the user and the pedestrian with the preset safe distance S; If the distance L between the user and the pedestrian is less than the preset safe distance S, it is determined that the distance between the user and the pedestrian is too close and the position needs to be adjusted. A voice prompt is sent to the user through the Bluetooth headset to guide the user to maintain a safe distance.

10. A Bluetooth glasses integrated with a computer vision software algorithm according to claim 9, characterized in that, It includes an acquisition module, a processing module, an identification module, and an adjustment module. There are connections between the modules: The acquisition module is used to collect real-time images of the surrounding environment through the wide-angle camera on the Bluetooth glasses; The processing module is used to process the collected images of the surrounding environment, identify the location of the pedestrian zebra crossing, and guide the visually and auditorily impaired user to the location of the pedestrian zebra crossing; The identification module is used to identify the color of the traffic light when it is recognized that the visually and auditorily impaired user has reached the location of the pedestrian zebra crossing; If the traffic light is recognized as red, a waiting prompt is sent to the visually and auditorily impaired user; If the traffic light is recognized as green, vehicle detection is performed, and it is judged whether the visually and auditorily impaired user can safely cross the road based on the detected vehicle distance and speed; The adjustment module is used to identify nearby pedestrians through the surrounding environment images during the process of the visually and auditorily impaired user crossing the road, calculate the distance between the visually and auditorily impaired user and the pedestrian, and compare and analyze this distance with the preset safe distance, and guide the visually and auditorily impaired user to maintain a safe distance from the pedestrian according to the analysis result.

Images