Road turning prompt algorithm and system based on visual solution

Abstract

The invention relates to the field of computer vision systems, and particularly discloses a road turning prompt algorithm and system based on vision solution. The method comprises the following steps: acquiring a surrounding environment through a binocular camera on the auxiliary equipment, and acquiring a surrounding map and an advancing vector through visual calculation; calculating a safe advancing route according to the conditions of the pedestrian and the safe distance; correcting the scheme according to the obtained route and the direction of the advancing vector; and processing schemes for some simple and special conditions are provided. The problems that visual impaired people cannot determine whether to turn to the correct direction or not during traveling, and then walk on wrong roads and bypass far roads are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision systems, and specifically discloses a road steering prompt algorithm and system based on visual solution. Background Art

[0002] When modern visually impaired people travel, they still mainly rely on the combination of traditional blind canes, guide dogs and blind paths, and there are still many problems. When using a blind cane, there may be a situation where an obstacle is not detected, resulting in bumps. At the same time, when the sense of direction is lost, one may hit a wall, resulting in injuries, getting lost, etc. The problems of blind paths are also significant. The main problem is that the blind paths do not form a closed loop, which leads to three main phenomena. One is that they cannot leave home, the second is that they cannot accurately reach the destination, and the third is that it is difficult to go home after going out. Secondly, various items such as shared bicycles occupy the blind paths. When planning motor vehicle and non-motor vehicle parking spaces, the blind paths are avoided, but there are still some vehicles with longer bodies that occupy part of the blind paths. At the same time, a large number of public places do not allow pets to enter, such as shopping malls, schools, etc. Even if the situation is explained clearly, it is difficult to enter.

[0003] The present invention aims to help solve the problem of difficult guide for the blind. Summary of the Invention

[0004] To solve the above problems, the present invention provides a road steering prompt algorithm and system based on visual solution,

[0005] To achieve the above object, the present disclosure proposes a road steering prompt method based on visual solution, including the following steps:

[0006] Obtain the visual signals collected by the sensor module carried by the user;

[0007] Use the visual solution algorithm to solve the visual signals and construct a two-dimensional map of the surrounding environment;

[0008] Preprocess the data and calculate the current position vector, traveling speed and direction of the user;

[0009] Calculate the recommended traveling route based on the map information and position vector;

[0010] Compare the recommended traveling route with the actual traveling direction of the user to determine the steering prompt.

[0011] In addition, the present disclosure proposes a road steering prompt system based on visual solution, including:

[0012] A data acquisition module for collecting visual signals of the surrounding environment through a visual sensor, where the visual signals include obstacle, road marking, pedestrian and intersection information;

[0013] An information preprocessing module for processing the visual signals to extract map information and a position vector, where the position vector includes the traveling speed and direction of the user;

[0014] An information calculation module for calculating a recommended traveling route and a recommended traveling vector based on the map information and the position vector, comparing them with the actual traveling vector of the user, and generating a steering prompt;

[0015] A voice prompt module for providing prompt information to the user in a voice manner according to the steering prompt.

[0016] Advantageous effects:

[0017] Some embodiments of the present disclosure can, through a binocular event camera and a visual calculation algorithm, obtain real-time surrounding environment information and generate an accurate two-dimensional map. By calculating a recommended traveling route and comparing it with the actual traveling vector of the user, steering prompts are provided for the user, significantly reducing the collision risk caused by wrong directions or misjudgments during the travel of visually impaired people and effectively ensuring travel safety. Description of the Drawings

[0018] Figure 1 An intersection scenario fictionalized and non-generally lost for the embodiments of the present disclosure;

[0019] Figure 2 Part obtained after data collection by the visual calculation algorithm in the embodiments of the present disclosure;

[0020] Figure 3 A simplified mathematical model of the embodiments of the present disclosure;

[0021] Figure 4 Schematic diagram of the recommended traveling vector in the embodiments of the present disclosure;

[0022] Figure 5 Flowchart of the method in the embodiments of the present disclosure;

[0023] Figure 6 System flowchart of the embodiments of the present disclosure. Detailed Embodiments

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0025] The present disclosure proposes a road steering prompt algorithm and system based on visual solution, which obtains visual signals of the surrounding environment through a binocular event camera, constructs a two-dimensional map and calculates the user's travel vector in real time, and then generates a recommended travel route. By comparing the recommended travel route with the user's actual travel direction, steering prompts are provided to avoid direction errors or collisions for visually impaired people and ensure travel safety.

[0026] The following specifically describes a road steering prompt algorithm based on visual solution proposed by the present disclosure. As Figure 5 shown, it specifically includes the following steps:

[0027] Visual signals are obtained through the sensor module carried by the user. Assume that the user starts from a certain starting point and is about to turn to a junction. By using the visual solution algorithm, the system can obtain visual signals from the environment and construct a map of the surroundings based on these signals. The construction of the map is completed by the data collected by the sensor module. At the same time, the system can also calculate the user's current travel speed and travel direction, so as to obtain the user's current position vector, and generate a two-dimensional grid map based on the collected information.

[0028] Furthermore, the system can dynamically update the map information to provide accurate navigation support for the user.

[0029] As Figure 1 shown, in some embodiments, assume that the user starts from A and turns to intersection B. Through the visual solution algorithm, the map can be obtained as shown in the appendix Figure 2 shown. At the same time, the user's travel speed and direction can be obtained, and the position vector can be obtained. The part to the right of the dotted line m in the appendix Figure 2 is the part of the map that can be constructed by the event camera.

[0030] To construct a two-dimensional map using a binocular event camera, the following steps are as follows:

[0031] (1): Read the visual data obtained by the binocular event camera, use IMU assistance for motion compensation, and obtain the corresponding reconstructed image.

[0032] (2): Extract the feature points and feature edges in the image, obtain their depth information, and track them.

[0033] (3): Thus, the gmapping function package can be used to generate a two-dimensional grid map.

[0034] In some embodiments, the sensor module includes a vision module, a solution module, and a voice module. The vision module consists of two event cameras, has a ranging function, and also has an obstacle recognition function for obtaining visual signals. The solution module is responsible for solving the obtained visual signals, constructing a scene map, calculating a recommended travel route using a road steering assistance algorithm, and giving the deviation between the position vector and the recommended travel vector, so as to determine how to give a reminder. The voice module is mainly composed of Bluetooth earphones, which are connected to the solution module. After the result is calculated, it gives a voice reminder to the user.

[0035] Preprocess the data to obtain map information and the position vector of the user's movement. During the map construction process, the system first needs to obtain and process some information. The information includes locating the user's current position and marking this position on the map; setting the dividing line of the intersection at the position where the roads meet; setting the center line of the road and marking the intersection points with each other; marking the outer corners adjacent to the road, denoted as "upper wall corner" and "lower wall corner"; setting the midpoint of the dividing line of the intersection.

[0036] In a specific embodiment, the following information is obtained in the map as Figure 2 shown: Point O is the user's current position, the dotted line l is the mid-dividing line of intersection A, the dotted line n is the mid-dividing line of intersection B, and the two dotted lines intersect at point K, which is called the mid-convergence point. W1 and W2 are two wall corners, where W1 is the lower wall corner and W2 is the upper wall corner, point M is the midpoint of the line segment W1W2, and ∠OKW1 is denoted as θ, ∠MKW1 is where θ and can be measured by the sensor module, and the length of W1K can be measured by the ranging function and denoted as 4h.

[0037] Based on the obtained data, calculate and plan the route through an algorithm. The specific steps are as follows: Select a coordinate system with a certain point as the origin to construct a plane rectangular coordinate system.

[0038] Use two parabolas spliced together to represent the travel route. The two parabolas intersect at a specific point, which is defined as the "convergence point". To ensure that the user can pass safely and avoid collisions with the wall corners, the convergence point is set at 1 / 2 to 1 / 5 of the distance closest to the upper wall corner.

[0039] To ensure the smoothness and continuity of the route, the travel route can be set to be tangent to the center lines of the two roads.

[0040] Furthermore, to ensure the accuracy of the model, it is necessary to prove that the function is continuous at the confluence point without any discontinuity. Through mathematical derivation, the analytical formula of the parabola is obtained, and the derivatives of both the left and right halves at the confluence point are zero, thus ensuring the continuity of the entire function at this point. This mathematical property ensures the correctness and stability of the recommended travel vector.

[0041] In the following embodiment, the path planning process is as follows: constructing the recommended travel route function, taking K as the origin and the direction from the mid-confluence point to the lower corner as the positive y-axis direction to construct a plane rectangular coordinate system, thereby simplifying the model. The simplified model is as Figure 3 shown. Since the widths of the two intersections are different, two parabolas are used to splice the route function. The two parabolas intersect at a point between W1 and K, and this point is denoted as the confluence point. To ensure the safe passage of the user without colliding with the corner, the confluence point is set at the quarter point on W1K closest to point K. At the same time, it is required that the travel route is tangent to the center lines of the two roads. At this time, the analytical formulas of the two side parabolas are uniquely determined as follows:

[0042]

[0043] Obviously, the parabola is smooth and continuous. It only needs to be proved that the function is continuous at x = 0. The derivatives of both the left and right halves of the function at zero are 0, so the function is continuous at x = 0, ensuring the existence of the recommended travel vector, as Figure 4 shown.

[0044] Based on the acquired data and the recommended travel route function, calculate the recommended travel vector and compare it with the user's travel vector to determine the steering prompt;

[0045] In some publicly disclosed embodiments, the direction of the recommended travel vector is the same as the tangent vector direction of f(χ) (by default, the right direction is taken as the positive direction), and the magnitude is the same as the magnitude of the current position vector. Let's assume that the slope of the line where the position vector of a certain point on the function is located in the appendix Figure 3 is k1, and the slope of the line where the recommended travel vector is located at this point is k2. Then there are the following three situations:

[0046]

[0047] According to the three scenarios calculated in the above steps, the following prompts are given to the user through the voice module:

[0048] ①: At this time, the travel direction is correct, and the prompt is "The current travel route is correct."

[0049] ②: At this time, the steering amplitude is too small, and the prompt is "Please increase the steering angle."

[0050] ③: At this time, the steering amplitude is too large, and the prompt is "Please decrease the steering angle."

[0051] In the above disclosed embodiment, the present invention obtains the surrounding environment through a binocular camera and uses a vision algorithm to construct a two-dimensional map, while calculating the user's travel vector in real time. Based on this information, a recommended travel route can be generated and compared with the user's actual travel direction, thereby prompting whether the user's turn is correct. The technology embodied in this embodiment can effectively prevent the risk of direction errors or collisions when visually impaired people turn, ensuring safe travel.

[0052] Another implementation of the present disclosure is as Figure 6 shown, a road turning prompt system based on vision algorithm. The system consists of a data acquisition module, an information preprocessing module, an information algorithm module, and a voice prompt module. Through the cooperation of multiple sensors and data processing, it provides accurate turning prompts for users in real time, as follows;

[0053] Data acquisition module, the system first obtains street environment data through the data acquisition module. This module includes sensors, cameras, etc., and collects visual signals of the surrounding environment. Through the visual sensor, the system can capture surrounding obstacles, road markings, pedestrians, and intersection information in real time, providing a basis for subsequent data processing and map construction.

[0054] Information preprocessing module, the data collected by the data acquisition module is transmitted to the information preprocessing module. The information preprocessing module processes the collected visual information through a signal processing device, extracts map information and position vectors. The position vectors include travel speed and direction; road marking points are obtained based on the extracted map information.

[0055] Information algorithm module, processes the information of the information preprocessing module through a travel assistance function. By analyzing the road conditions, obstacles, and the user's real-time position, the algorithm module calculates the recommended travel vector. According to the current travel state and the recommended travel vector, the system determines the amplitude and direction of the turn through angle analysis. By comparing the recommended travel vector with the user's actual travel vector, the system calculates the next travel action to be taken.

[0056] Voice prompt module, according to the calculation results of the information algorithm module, the system converts the travel advice into voice prompts.

[0057] Furthermore, the prompts can be daily expressions such as "turn right", "please reduce the turning angle", "wait", etc.

[0058] The above-mentioned disclosed embodiments construct a road steering prompt system based on visual calculation. The system can generate a recommended travel route and compare it with the user's actual travel direction, thereby prompting whether the user's steering is correct. The application scenarios are extensive and not limited to roads, addressing the problem of limited application scenarios for guide pets. The coverage area is wide and can be used in places without blind paths. It effectively avoids problems such as bumps and injuries caused by inaccurate steering when blind people travel.

[0059] In the description of this specification, the description referring to terms such as "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0060] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not elaborate on all details and do not limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and utilize the present invention well. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A road steering prompt method based on visual solution, characterized in that, It includes the following steps: Obtain the visual signals collected by the sensor module carried by the user; Use the visual solution algorithm to solve the visual signals and construct a two-dimensional map of the surrounding environment; Preprocess the data to calculate the current position vector, traveling speed and direction of the user; Calculate the recommended travel route based on the map information and position vector; Compare the recommended travel route with the actual travel direction of the user to determine the steering prompt.

2. The method for prompting road steering based on visual solution according to claim 1, wherein, The visual signals are collected by a binocular event camera.

3. The method for prompting road steering based on visual solution according to claim 2, wherein The collection of the visual signals specifically includes: Read the visual data obtained by the binocular event camera, perform motion compensation using the IMU, and generate a reconstructed image; Extract the feature points and feature edges in the image, obtain their depth information and perform tracking; Use the gmapping package to generate a two-dimensional grid map.

4. A road steering prompt method based on visual solution according to claim 1, characterized in that The preprocessing of the data includes: Locate the current position of the user and mark this position on the map; Set the dividing line of the intersection at the position where the roads meet; Set the center line of the road and mark the intersection points with each other; Mark the outer corners adjacent to the road; set the midpoint of the dividing line marking the intersection.

5. A road steering prompt method based on vision solution according to claim 1, characterized in that The calculation of the recommended travel route includes the following steps: Based on the current position of the user and the geometric features of the intersection, construct a plane rectangular coordinate system; Represent the recommended travel route by splicing two parabolas to ensure that the travel route is tangent to the center lines of the two roads.

6. A method for prompting road steering based on visual solution according to claim 1, characterized in that Ensure the continuity of the function at the convergence point of the recommended travel route to ensure the stability of the recommended travel vector.

7. A method for road steering prompt based on visual solution according to claim 5, characterized in that, The convergence point is set at 1 / 2 to 1 / 5 of the distance closest to the upper outer corner.

8. A method for road steering prompt based on visual solution according to claim 1, wherein, The steering prompt includes the following situations: The current travel route is correct, prompt "The current travel route is correct"; The steering amplitude is too small, prompt "Please increase the steering angle"; The steering amplitude is too large, prompt "Please decrease the steering angle".

9. A method for road steering prompt based on visual solution according to claim 5, characterized in that The method includes providing corresponding voice prompts to the user through the voice module according to the steering prompt.

10. A road steering prompt system based on visual solution, characterized in that, It includes: A data acquisition module for collecting visual signals of the surrounding environment through a visual sensor, and the visual signals include obstacle, road marking, pedestrian and intersection information; An information preprocessing module for processing the visual signals, extracting map information and position vector, and the position vector includes the traveling speed and direction of the user; An information solution module for calculating the recommended travel route and recommended travel vector based on the map information and position vector, comparing with the actual travel vector of the user, and generating a steering prompt; A voice prompt module for providing prompt information to the user in a voice manner according to the steering prompt.

Images