A method for real-time determination of the driving trajectory of blind people based on drone guidance

Abstract

This invention discloses a real-time method for judging the driving trajectory of blind people based on drone-guided navigation, which is implemented by a data calculation module and a data generation module. The real-time method for judging the driving trajectory of blind people includes the following steps: (1) judging whether the driving trajectory of blind people is incorrect; (2) if the driving trajectory of blind people is incorrect, then replanning the optimal route. This invention judges whether blind people have deviated from the optimal road by detecting abnormalities in the real-time driving trajectory of blind people on non-motorized vehicle lanes when there is no tactile paving. That is, the data calculation module judges whether the specific location data of blind people on non-tactile road sections is within the optimal path area. If so, the next step is performed; the terminal information judgment module judges whether the positions of multiple roads traveled by blind people are abnormal before reaching the target destination. If so, it judges that the trajectory on a certain road during the blind people's travel is abnormal. This invention improves the accuracy and safety of blind people's travel and solves the problem that the existing technology cannot intelligently, accurately and in real-time judge the driving trajectory of blind people.

Description

Technical Field

[0001] This invention relates to a method for judging the navigation trajectory of a blind person using a drone, and more particularly to a method for real-time judgment of the driving trajectory of a blind person based on a drone navigation system. Background Technology

[0002] According to annual statistics from the World Health Organization, as many as 450,000 new cases of blindness are added each year, and the inconvenience of travel causes great distress to many blind people. Tactile paving is an indispensable part of urban roads, acting as "static eyes" for the blind, helping them to sense the path by feeling their feet and using their guide canes. However, in many cities today, tactile paving is illegally occupied or has collapsed, making travel even more difficult for the blind, especially in rural areas or other places where tactile paving is not provided.

[0003] Traditional guide canes and guide dogs are greatly affected by external factors and are prone to accidents; emerging guide glasses and electronic guide canes are also constrained by the environment, with high uncertainty, making it difficult to meet the needs of blind people. However, with the rapid development of drone technology, how to integrate drones into the field of guide services to intelligently guide blind people, judge their travel routes in real time, reduce safety hazards, and improve the accuracy, safety, and efficiency of their travel has become an urgent technical problem to be solved.

[0004] Current research on drone-based navigation for the blind mainly focuses on new methods and systems to facilitate travel for the blind, but lacks research on the accuracy of navigation for the blind. Patent application CN108791881B authorizes a drone system and method for navigation, establishing an information processing module and processor, and proposing a flattened ellipsoidal body containing multiple devices, enabling the drone to perceive the surrounding environment of the blind for navigation, providing convenient and safe travel for the blind. Patents CN111610544A and CN107049719A respectively authorize a drone-based following positioning method and an intelligent navigation warning method and system for the blind. The former uses a fixed triangle composed of three distance signal transmitters for assisted positioning, solving the problems of high computational load, limited accuracy, and susceptibility to obstruction in current navigation drone systems. The latter establishes a connection between the drone, cane, connecting device, and processing device to detect the environment and traffic lights ahead of the blind in real time, providing warnings and reminders. However, blind people still face many uncertainties when traveling, which can easily lead to problems. Current research has not yet mentioned how to judge the travel trajectory of blind people in real time, so as to effectively solve the problem of incorrect travel trajectory and how to deal with it, and provide blind people with more accurate and safe travel. Summary of the Invention

[0005] Purpose of the invention: To address the shortcomings of existing technologies, this invention proposes a real-time method for judging the driving trajectory of blind people based on drones, in order to solve the problem that current technologies cannot intelligently, accurately, and in real-time judge the driving trajectory of blind people.

[0006] Technical Solution: This invention, based on a drone-guided navigation method for the blind, implements a data calculation module and a data generation module. The data calculation module includes an optimal path location information acquisition module, a blind person location information acquisition module, a data processing module, and a location prediction module. The data generation module includes a terminal information judgment module, a terminal information storage module, and an information receiving module. The real-time trajectory judgment method includes the following steps:

[0007] Step (1), determine whether the blind person's driving trajectory is incorrect, the process is as follows:

[0008] (1.1) Logical judgment is performed by establishing a three-dimensional coordinate system;

[0009] (1.2) Obtain the optimal path for the blind person and set the three-dimensional coordinate data M(X,Y,Z); N(X,Y,Z);

[0010] (1.3) Generate data tables to classify and store the data;

[0011] (1.4) Set the road area as D, and the curve equations of the road in the horizontal and vertical directions as A1, A2, B1, B2. Generate a two-dimensional scatter plot about (X,Y) by combining the coordinates of the points captured by the road with the coordinates of the blind person's points and fit it. Set X as third order and Y as first order to generate a curve fitting image and road curve equation.

[0012] (1.5) Determine whether blind person M is within road area D;

[0013] (1.6) If the judgment is yes, it means that the blind person's driving trajectory is correct, and a blind person arrival signal is sent;

[0014] (1.7) If it is determined that the region is not within region D, proceed to the next step;

[0015] (1.8) Locate the blind person's position, replan the optimal path, and send a warning signal;

[0016] (1.9) While sending the warning signal, recalculate;

[0017] (1.10) Reacquire the blind person's location and optimal path, and set the three-dimensional coordinates as M(X,Y,Z); N(X,Y,Z).

[0018] Step (2): If the blind person's driving trajectory is incorrect, the optimal route is replanned. The process is as follows:

[0019] (2.1) Use the optimized ant colony algorithm to perform global path planning for the target road segment;

[0020] (2.2) Real-time obstacle avoidance is performed using the optimized dynamic window algorithm;

[0021] (2.3) Intelligent obstacle avoidance of UAV is achieved by supplementing static local path planning with dynamic local obstacle avoidance algorithm, thus completing the path planning task.

[0022] In step (1.2), the optimal position for the blind person is obtained by acquiring the original value through GIS, creating a new layer on the GIS, establishing a three-dimensional coordinate system, determining the origin position, and generating specific coordinates N1, N2…N on the path based on the distance values ​​from specific points on the planned path to the origin. n The coordinates of the blind person are M(X,Y,Z).

[0023] In step (1.3), the optimized ant colony algorithm is used for global path planning of the UAV and the optimized dynamic window algorithm is used for real-time obstacle avoidance. The dynamic local obstacle avoidance algorithm is used to supplement the static local path planning to realize the intelligent planning of the global path of the UAV and complete the optimal path planning task.

[0024] In step (1.4), the road area is set as D, and the curve equations of the road in the horizontal and vertical directions are A1, A2, B1, and B2. The coordinates of the points captured by the road and the coordinates of the blind person's points are used to generate a two-dimensional scatter plot about (X,Y) and fit it. X is defined as third order and Y is defined as first order. The output fitting result is a function expression about (X,Y), which is the road curve equation.

[0025] In step (1.5), it is determined whether the blind person M is in the road area D. The judgment condition is a 0-1 logic judgment.

[0026] The judgment process in step (1.5) is as follows:

[0027]

[0028] Where logic represents a conditional statement, M represents the blind person's location, and D represents the optimal path region;

[0029] If the logic value of the judgment statement is 0, then the driving trajectory is correct and has not deviated within the optimal path area, and an arrival signal is sent.

[0030] If the logic condition is 1, then the blind person is not within the optimal path area. Locate the blind person's position, replan the optimal path, and send a warning signal.

[0031] If the path is outside the optimal path area, a warning signal is sent and the calculation is recalculated.

[0032] The determination process in step (2.1) is as follows: first, set the parameters for initialization, then evaluate the ant colony, and if the conditions are met, it is the optimal path; if it is not the optimal path, n = n + 1, select the direction of movement with probability, update the pheromone, and re-evaluate the ant colony.

[0033] The algorithm used in step (2.1) is the ant colony algorithm. The path planning process of the ant colony algorithm is as follows:

[0034]

[0035] α is the information heuristic factor, reflecting the relative importance of the amount of information remaining in the ant colony during its movement; β is the expectation heuristic factor, reflecting the relative importance of the expected value; η ij (t) represents the heuristic information on the path (i,j) at time t, which is called prior knowledge or heuristic function.

[0036] In the formula,

[0037]

[0038] Where, (x goal ,y goal ) represents the target node, (x j ,y j Let represent the coordinates of the next possible node. The reciprocal of the Manhattan distance between the two nodes is used as a heuristic function to enhance the guiding effect on the target point. By improving the original heuristic function, the problem of ant colony optimization getting stuck in local optima is compensated for, thereby improving the efficiency of target search.

[0039] The determination process in step (2.2) is as follows: parsing the sensor data to create a cost map, parsing the position data, parsing the Footprint data, then sampling the velocity, generating the trajectory space, evaluating the trajectory, and selecting the optimal path.

[0040] The determination process in step (2.2) uses the dynamic window algorithm, and the formula is as follows:

[0041]

[0042]

[0043] Among them, G dist This represents the Euclidean distance between the end of the trajectory and the target point at the current moment, heading represents the angle θ between the UAV at the end of the trajectory and the target point, and path is the length of the globally planned path.

[0044] In this study, considering the complexity of the path, an evaluation function is added to the angle θ between the UAV and the target point at the end of the trajectory. The weights are dynamically adjusted according to the angle θ between the UAV and the target point, which facilitates the UAV to quickly approach the target point and reduces unnecessary searches. The evaluation function is as follows:

[0045]

[0046] In the formula, when the included angle θ is 0°, 60°, 90°, or 120°, the weights are assigned to 0, 1, 2, and 3, respectively.

[0047] Working principle: This invention uses a drone to establish a three-dimensional coordinate system to determine whether a blind person is within the correct road area. If they are, it indicates they have not deviated, and an arrival signal is sent; if they are not, the optimal path is replanned to guide the blind person forward.

[0048] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0049] (1) This invention uses the internal data calculation module and specific formula of the UAV to determine whether the blind person’s driving trajectory deviates from the optimal path, so that the blind person can adjust the road and direction in time after the driving trajectory is wrong, thereby improving the accuracy, safety and efficiency of blind people’s travel.

[0050] (2) The optimized ant colony algorithm in this invention improves the heuristic function η compared with the traditional ant colony algorithm. ij (t) (that is, using the reciprocal of the Manhattan distance between the coordinates of the target node and the next optional node as a heuristic function) solves the problem that the ant colony algorithm is prone to getting trapped in local optima, enhances the guiding role of the target point, and improves the target search efficiency.

[0051] (3) Compared with the traditional dynamic window algorithm, the optimized dynamic window algorithm in this invention adds an evaluation function (that is, adds an evaluation function to the angle θ between the UAV at the end of the trajectory and the target point, so that the weight of the angle θ is dynamically adjusted. When the angle θ is 0°, 60°, 90° and 120°, the corresponding weights are 0, 1, 2 and 3 respectively), which makes it easier for the UAV to quickly approach the target point and reduce unnecessary searches.

[0052] (4) The present invention uses Excel to generate a two-dimensional scatter plot of (X,Y) using the coordinates of the points captured on the road and the coordinates of the blind person. The Excel fitting tool is then used to fit the plot. X is set to the third order and Y is set to the first order. The final fitted image is more convergent than other orders, and the road curve equation is more accurate.

[0053] (5) The improved guide cane with embedded positioning function in this invention is equipped with a voice broadcast module and a vibration mode. When the blind person’s driving trajectory deviates from the predetermined trajectory, the voice broadcast module inside the guide cane will give an error prompt. At the same time, the vibration mode inside the guide cane is divided into left and right parts. When the driving trajectory is adjusted to the left, the left side of the guide cane will vibrate; when the driving trajectory is adjusted to the right, the right side of the guide cane will vibrate, so that the blind person can intuitively feel the specific direction and make adjustments. Attached Figure Description

[0054] Figure 1 This is a flowchart of the real-time trajectory determination method for blind people according to the present invention;

[0055] Figure 2 This is a flowchart of the part of the present invention for obtaining the optimal driving path for a blind person;

[0056] Figure 3 This is a flowchart illustrating the process of determining whether a blind person's driving trajectory is incorrect, as per the present invention.

[0057] Figure 4 This is a flowchart illustrating the method for generating the fitted road equation in this invention;

[0058] Figure 5 The image shows the result of fitting the east-west road generation function according to the present invention. Detailed Implementation

[0059] Example:

[0060] like Figure 1 As shown, the present invention provides a real-time trajectory determination method for blind people based on drone-guided navigation. It uses GIS technology to obtain the road safety area and the real-time location coordinates of the blind person. The determination method is implemented using seven modules, namely, the optimal path location information acquisition module, the blind person location information acquisition module, the data processing module, the location determination module, the signal receiving module, the terminal information storage module, and the terminal information determination module.

[0061] The optimal path location information module utilizes the UAV's internal GIS positioning technology to pinpoint the blind person's location, acquires the current road data set to draw road curves, establishes a three-dimensional coordinate system, and creates a new layer on the GIS to obtain the optimal path location information. The blind person's location information acquisition module also utilizes GIS, using a smart guide cane with embedded positioning functionality as a medium to remind the blind person to make directional adjustments. The guide cane is equipped with a voice broadcast system and vibration modes. When the blind person's travel trajectory deviates from the predetermined trajectory, the voice broadcast module inside the guide cane provides an error message. Simultaneously, the internal vibration mode of the guide cane is divided into left and right parts: when the travel trajectory is adjusted to the left, the left side of the guide cane vibrates; when the travel trajectory is adjusted to the right, the right side of the guide cane vibrates, allowing the blind person to intuitively perceive their specific location.

[0062] The data processing module uses a drone to scan the road surface, collecting all safety area information and the location information of the blind person, and draws a three-dimensional image, including the safety area D and the blind person's location coordinates M(x,y,z), which are represented by points.

[0063] The location determination module A includes an information storage device and a judgment device. The information storage device is used to store the information obtained from the three-dimensional image generated by the data module and the preset map information. The judgment device is used to determine whether the blind person is on the planned optimal path.

[0064] The location determination module B includes an information storage device and a judgment device. The information storage device is used to store the three-dimensional image generated by the data module, and the judgment device is used to determine whether the real-time location information M(x,y,z) of the blind person is within the safe area D generated by the data processing module.

[0065] The signal receiving module includes a signal receiver, which is used to receive information transmitted by the signal transmitter.

[0066] The terminal information storage module includes an information storage device and a counter. The information storage device is used to store the information transmitted by each signal transmitting module, and the counter is used to count the number of vehicles at each location.

[0067] The terminal judgment module includes an information storage device and a judgment device. The information storage device is used to store real-time three-dimensional image information, including the blind person's location information and the optimal path safety area. The judgment device is used to determine whether the blind person's location has reached the destination requirement.

[0068] The process of determining whether a blind person's driving trajectory is incorrect based on drone-guided navigation in this invention is as follows:

[0069] Step (1) Obtain the coordinates of the safe zone and the blind person's location;

[0070] Step (2) Determine whether the blind person is in a safe area.

[0071] The specific steps of this invention to determine whether a blind person's driving trajectory is incorrect are as follows:

[0072] (1.1) The safe walking area for blind people is obtained by fitting the road curve, and the range is set as D;

[0073] The specific process of step (1.1) is as follows:

[0074] The optimal path location information module detects the location information of blind people, and the optimal path three-dimensional coordinate data M(x,y,z) and N(x,y,z). The general curve equations of the road in the horizontal and vertical directions are set as A(x,y,z)=0 and B(x,y,z)=0. The road curve equation is generated by fitting the equations using Excel software.

[0075] The generated path trajectories are summarized into a set {N1, N2, N3…N}. n The optimized ant colony algorithm and dynamic window algorithm are used to process the coordinate data M(x,y,z) and N(x,y,z) to obtain the path arrangement with the most obstacles and the fewest obstacles, denoted as {A1, A2, A3…A2}. n}

[0076] Calculate the distance to the target destination for each path, and arrange them from longest to shortest, denoted as {B1, B2, B3…B... n Output the optimal results A1 and B1 for arranging the two sets of data, prioritizing the number of obstacles. If the distance difference between the two paths is greater than 1000 meters, then the optimal path B1 is generated; otherwise, the optimal path A1 is generated.

[0077] (1.2) The judgment module determines whether the blind person's location is on the planned optimal path A1. When there is a deviation in the blind person's driving trajectory, the guide cane with the embedded positioning function vibrates and announces the walking error in voice, and replans the route to the designated optimal road.

[0078] (1.3) Once the blind person is identified on the designated optimal road, the optimal path location information module is activated.

[0079] (1.4) Set the safe zone D for the blind person. The optimal path location information module detects the blind person's location information, and the judgment module determines whether the blind person's location is within the designated safe zone D. When the blind person is not within the safe zone, the real-time location of the blind person is recorded. See the detailed steps below. Figure 4 flow chart.

[0080] (1.5) If the judgment is yes, then no further operation is performed;

[0081] (1.6) If the location information of the blind person is determined to be outside the designated safe area D, then the signal receiving module is activated and the location information signal of the blind person is sent to the signal receiving module; the judgment condition is a 0-1 logic judgment, and the judgment process is as follows:

[0082]

[0083] In this context, `logic` represents a conditional statement, `M` represents the location of the blind person, and `D` represents the safe zone of the optimal path. If the blind person's coordinates are not within the safe zone `D`, the conditional statement `logic` equals 1, indicating that the blind person's walking is abnormal; if the blind person is within the safe zone `D`, the conditional statement `logic` equals 0, indicating that the blind person's walking trajectory is normal.

[0084] (1.7) The signal receiving module receives signals and stores them. The vibration and sound broadcast functions of the guide cane with embedded positioning function issue warning signals and recalculate and plan the optimal path.

[0085] The specific steps for (1.7) are as follows:

[0086] Within region D, if the blind person's trajectory is correct and does not deviate, complete the first calculation, update the blind person's position, obtain the latest position information, start the second calculation, and overwrite the content of the first calculation to reduce the computational memory space; if the blind person is not within region D, locate the blind person's position and replan the optimal path.

[0087] (1.8) Repeat the above steps until the blind person reaches the end.

[0088] For example, a blind person M walks from the Shitahu Road intersection to the Renmin North Road intersection, with the road length represented by l. GIS positioning technology locates the blind person's position, acquires the current road data set, draws a curve, establishes a three-dimensional coordinate system, and creates a new layer on the GIS to obtain the optimal path location information. The blind person's location information acquisition module also utilizes GIS, using a guide cane with embedded positioning function as a medium to remind the blind person to adjust their position. The guide cane has a built-in voice broadcast system and vibration mode. When the blind person's walking trajectory deviates from the predetermined trajectory, the voice broadcast module inside the guide cane provides an error prompt. Simultaneously, the vibration mode inside the guide cane is divided into left and right parts: when the walking trajectory needs to be adjusted to the left, the left side of the guide cane vibrates; when the walking trajectory needs to be adjusted to the right, the right side of the guide cane vibrates. A drone scans the surrounding road sections, collects all safe area information data and blind person's location information data, and draws a three-dimensional image, including the safe area D and the blind person's location coordinates M(x0, y0, z0). Considering the low weight of z, the value range of z is set to (0.5, 2.5). To determine the location of a blind person relative to the road, a two-dimensional coordinate system (x, y) is drawn, such as... Figure 5 As shown, the east-west road is represented by the Excel fitted expression as follows:

[0089]

[0090] The fitting formula is defined as the road expression. The location of the blind person is then determined. If the following conditions are met, it means the person is on the optimal path D; otherwise, it means the person is not on the optimal path D:

[0091]

[0092] The optimal path location information module detects the blind person's location information. The judgment module determines whether the blind person's location is within the designated safe area D. If the blind person is outside the safe area, the real-time location is recorded. If the blind person is within the safe area, no further action is taken. If the blind person's location information exceeds the designated safe area D, the signal receiving module is activated, and the blind person's location information signal is sent to the signal receiving module. If the blind person's coordinates are not within the safe area D, the judgment statement `logic = 1` is executed, indicating that the blind person's walking is abnormal. If the blind person is within the safe area D, the judgment statement `logic = 0` is executed, indicating that the blind person's walking trajectory is normal, and the blind person continues walking. The above steps are repeated until the blind person reaches the intersection of Renmin North Road.

Claims

1. A method for real-time determination of the driving trajectory of blind people based on drone-guided navigation, characterized in that: Includes the following steps: Step (1), determine whether the blind person's driving trajectory is incorrect, the process is as follows: (1.1) Logical judgment is performed by establishing a three-dimensional coordinate system; (1.2) Obtain the optimal path for the blind person and set the three-dimensional coordinate data M(X,Y,Z);N(X,Y,Z); (1.3) Generate data tables to classify and store the data; (1.4) Set the road area as D, and the curve equations of the road in the horizontal and vertical directions as A1, A2, B1, B2. Generate a two-dimensional coordinate scatter plot by combining the coordinates of the road acquisition point with the coordinates of the blind person's point and fit it. Set X as third order and Y as first order to generate a curve fitting image and road curve equation. (1.5) Determine whether blind person M is within road area D; (1.6) If the determination is yes, then send a signal indicating that the blind person has arrived; (1.7) If it is determined that the region is not within region D, proceed to the next step; (1.8) Locate the blind person's position, replan the optimal path, and send a warning signal; (1.9) While sending the warning signal, recalculate; (1.10) Reacquire the blind person's location and optimal path, and let the three-dimensional coordinates be M(X,Y,Z); N(X,Y,Z); Step (2), if the blind person's driving trajectory is incorrect, the process is as follows: (2.1) Use the optimized ant colony algorithm to perform global path planning for the target road segment; The path planning process of the ant colony algorithm in step (2.1) is as follows: ; α is the information heuristic factor, reflecting the relative importance of the amount of information left behind by the ant colony during its movement; β is the expectation heuristic factor, reflecting the relative importance of the expected value. This represents the heuristic information on the path (i,j) at time t; In the formula, ; in, Indicates the target node. Indicates the coordinates of the next possible node; (2.2) Real-time obstacle avoidance is performed using the optimized dynamic window algorithm; The determination process in step (2.2) is as follows: parsing the sensor data to create a cost map, parsing the position data, parsing the Footprint data, then sampling the velocity, generating the trajectory space, evaluating the trajectory, and selecting the optimal path; The formula for the dynamic window method used in step (2.2) is as follows: ; ; In the formula This represents the Euclidean distance between the end of the trajectory and the target point at the current moment. The angle between the drone's trajectory and the target point , The path length is the global planning path length; The angle between the drone at the end of its trajectory and the target point Add an evaluation function based on the angle between the UAV and the target point. The weights are dynamically adjusted, and the evaluation function is as follows: ； (2.3) Intelligent obstacle avoidance of UAV is achieved by supplementing static local path planning with dynamic local obstacle avoidance algorithm, thus completing the path planning task.

2. The method for real-time determination of the driving trajectory of a blind person based on a drone-guided vehicle according to claim 1, characterized in that: In step (1.2), the optimal position for the blind person is obtained by acquiring the original value through GIS, creating a new layer on the GIS, establishing a three-dimensional coordinate system, determining the origin position, and generating the specific coordinates N on the path based on the distance values ​​of specific points on the planned path from the origin. 1, N2…N n The coordinates of the blind person are M(X,Y,Z).

3. The method for real-time determination of the driving trajectory of blind people based on UAV guidance as described in claim 1, characterized in that: In step (1.4), the road area is set as D, and the curve equations of the road in the horizontal and vertical directions are A1, A2, B1, and B2. The coordinates of the points captured by the road and the coordinates of the blind person's points are used to generate a two-dimensional scatter plot about (X,Y) and fit it. X is defined as third order and Y is defined as first order. The output fitting result is a function expression about (X,Y), which is the road curve equation.

4. The method for real-time determination of the driving trajectory of a blind person based on a drone-guided vehicle as described in claim 1, characterized in that: In step (1.5), the judgment process is as follows: ; Where logic represents a conditional statement, M represents the blind person's location, and D represents the optimal path region; If the logic of the judgment statement is 0, then the blind person is within the optimal path area, the driving trajectory is correct and there is no deviation, and an arrival signal is sent. If the logic statement is 1, then the blind person is not within the optimal path area. Locate the blind person's position, replan the optimal path, and send a warning signal. When a blind person is outside the optimal path area, a warning signal is sent and the calculation is recalculated.

5. The method for real-time determination of the driving trajectory of a blind person based on a drone-guided vehicle according to claim 1, characterized in that: The determination process in step (2.1) is as follows: first, set the parameters for initialization, then evaluate the ant colony. If the conditions are met, it is the optimal path. If it is not the optimal path, n=n+1, select the direction of movement with probability, update the pheromone, and re-evaluate the ant colony.

Images