Route planning method and system based on AR glasses and earphone, medium and equipment

Abstract

The invention provides a path planning method and system based on AR glasses and earphones, a medium and equipment, and the method comprises the steps: estimating the motion state of a user and the motion state of a vehicle in real time through an inertial measurement device and a camera on the AR glasses, and calculating the relative motion state according to the motion state of the user and the motion state of the vehicle; based on the relative motion state, calculating a danger cost function for each vehicle so as to carry out danger degree scoring on the current state of the user; presetting a risk score threshold, and if the risk score is greater than the risk score threshold, evaluating an optimal moving action according to the speed state of the user; and dynamically displaying the optimal movement action in AR glasses, and broadcasting the optimal movement action to a user through a loudspeaker of an earphone in real time. According to the method, the dangerous condition of the current path of the user can be calculated in real time, and the optimal moving path is fed back to the user through the AR glasses and the earphone.

Description

Technical Field

[0001] This invention relates to the field of AR glasses technology, and in particular to a path planning method and system based on AR glasses and headphones. Background Technology

[0002] With the popularization of augmented reality technology, pedestrian navigation and safety assistance systems based on AR glasses have become a research hotspot. These systems aim to plan safe routes for wearers and overlay virtual cues into their field of vision to avoid static obstacles and dynamic vehicles, thereby improving travel safety.

[0003] However, existing solutions still face significant challenges in complex and dynamic urban environments.

[0004] First, at the risk assessment level, traditional methods often use collision time or distance based on fixed thresholds as risk measures. This approach fails to fully consider the uncertainties in sensor perception and the randomness of target motion, resulting in overly rigid risk assessments—either generating false alarms in safe situations, disturbing users, or failing to provide timely warnings in truly dangerous situations due to uncertainty.

[0005] Next, at the path planning and decision-making level, existing methods often separate global path planning from local real-time obstacle avoidance. Paths generated by global planners (such as the A* algorithm) cannot cope with dynamic obstacles that suddenly appear in the environment; while simple local reactive obstacle avoidance rules (such as "turn right when encountering an obstacle") lack long-term considerations, which can easily lead pedestrians into local traps (such as dead ends) or deviate significantly from the global goal, resulting in "short-sighted" and unnatural navigation behavior, reducing user trust and the system's usability. Finally, prolonged use of AR glasses can cause the content to attract a lot of the user's attention, creating a sense of disconnect from reality and posing a safety hazard. Summary of the Invention In view of the shortcomings of the prior art, the purpose of this invention is to provide a path planning method based on AR glasses and headphones, which aims to solve the technical problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: A path planning method based on AR glasses and headphones includes the following steps: The user's own motion state and the vehicle's motion state are estimated in real time through the inertial measurement unit and camera on the AR glasses, and the relative motion state is calculated based on the user's own motion state and the vehicle's motion state. Based on the relative motion state, a hazard cost function is calculated for each vehicle to score the hazard level of the user's current state; A preset danger rating threshold is set. If the danger rating is greater than the danger rating threshold, the optimal movement action is evaluated based on the user's speed status. The optimal movement action is dynamically displayed in the AR glasses, and the optimal movement action is broadcast to the user in real time through the speaker of the headphones.

[0007] According to one aspect of the above technical solution, in the real-time estimation of the user's own motion state and the vehicle's motion state using a camera and inertial measurement device on the AR glasses, and the calculation of the relative motion state based on the user's own motion state and the vehicle's motion state, the user's own motion state is represented as P. p = [x p , y p vx p vy p , θ p The vehicle's motion state is represented as: Car i = [ID, x i , y i vx i vy i , width i , length i ]; Where, x p and y p This represents the user's 2D position in the world coordinate system, vx p and vy p θ represents the user's velocity in the x and y directions, respectively. p Indicates the pedestrian's facing angle; ID represents the marker for each vehicle, x i and y i This indicates the 2D position of the vehicle in the world coordinate system, vx i and vy i These represent the vehicle's speed in the x and y directions, respectively, and width. i , length i These represent the width and length of the vehicle, respectively.

[0008] According to one aspect of the above technical solution, the specific steps of calculating the relative motion state based on the self-motion state and the vehicle motion state include: The user is defined as a cylindrical collider with a radius of R. The vehicle is defined as a rectangular collision body, with the center of the rectangular collision body being x. i , y i The size is width i ,length i Orientation refers to the direction of the vehicle's speed; Calculate the shortest distance between the boundaries of the cylindrical collider and the rectangular collider to obtain the nearest point d. min ; Calculate the relative velocity scalar v between the user and the vehicle in the direction of vehicle velocity. r ; Based on the nearest point and the relative speed scalar, the precise TTC between the user and the vehicle is calculated.

[0009] According to one aspect of the above technical solution, the step of calculating a hazard cost function for each vehicle based on the relative motion state to score the hazard level of the user's current state includes: Calculate the collision probability, distance cost, and speed cost between the user and the vehicle; A hazard score is calculated based on the collision probability, the distance cost, and the speed cost.

[0010] According to one aspect of the above technical solution, if the preset danger score threshold is greater than the danger score threshold, then the optimal movement action is evaluated based on the user's speed status, and the specific steps include: A preset risk score threshold is used to determine whether the risk score is greater than the risk score threshold. If so, then obtain the speed window and acceleration window based on the user's speed information, and generate a dynamic window based on the speed window and acceleration window; Extract each set of velocity and angular velocity from the dynamic window to simulate the future trajectory; Each of the aforementioned future trajectories is scored to select the optimal future trajectory, and the action in the optimal future trajectory is then taken as the optimal movement action.

[0011] According to one aspect of the above technical solution, in the step of dynamically displaying the optimal movement action in the AR glasses, and broadcasting the optimal movement action to the user in real time through the speaker of the earphone, the speaker of the earphone dynamically adjusts the volume according to the user's current movement action and the offset of the optimal movement action.

[0012] The present invention also provides a path planning system based on AR glasses and headphones, comprising: Evaluation module: used to estimate the user's own motion state and the vehicle's motion state in real time through the inertial measurement unit and camera on the AR glasses, and calculate the relative motion state based on the user's own motion state and the vehicle's motion state; Calculation module: used to calculate a hazard cost function for each vehicle based on the relative motion state, in order to score the hazard level of the user's current state; Scoring module: used to preset a danger score threshold. If the danger score is greater than the danger score threshold, the optimal movement action is evaluated based on the user's speed status. Feedback module: used to dynamically display the optimal movement action in the AR glasses and broadcast the optimal movement action to the user in real time through the speaker of the headphones.

[0013] The present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the path planning method based on AR glasses and headphones as described above.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the path planning method based on AR glasses and headphones as described above.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention introduces a probabilistic collision risk model, transforming the traditionally predetermined collision time into a probability distribution. This model quantitatively expresses the uncertainties caused by sensor noise and unclear target movement intentions, thus enabling the system to have "self-awareness." It provides accurate warnings when perception is clear and proactively enhances risk awareness and preventative measures when perception is uncertain, greatly improving the system's robustness and safety while reducing false alarms and missed alarms.

[0016] This invention achieves a perfect balance between global optimization and local real-time performance by integrating an improved dynamic window method with a global path planning architecture. The dynamic window method ensures that each obstacle avoidance command generated conforms to the dynamics of pedestrian movement and is physically feasible; while its multi-objective evaluation function, by balancing safety distance, hazard avoidance, direction of travel, path fit, and movement efficiency, can select the comprehensive optimal solution from a range of possible actions. The guided behavior is natural, smooth, and intuitive, enabling decisive obstacle avoidance in critical situations and ensuring a rapid return to the globally optimal path after obstacle avoidance, avoiding short-sighted behavior and local oscillations, and greatly improving user comfort and trust in the system.

[0017] This invention improves safety by transmitting obstacle avoidance content from AR glasses to headphones, preventing users from becoming sluggish due to being engrossed in the content of AR glasses. Attached Figure Description

[0018] Figure 1 This is a flowchart of the path planning method based on AR glasses and headphones in the first embodiment of the present invention; Figure 2 This is a structural block diagram of the path planning system based on AR glasses and headphones in the second embodiment of the present invention; Figure 3 This is a structural block diagram of the electronic device in the third embodiment of the present invention; The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0019] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0020] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] Please see Figure 1 The diagram illustrates a path planning method and system based on AR glasses and headphones according to a first embodiment of the present invention, comprising the following steps: S10, through the inertial measurement device and camera on the AR glasses, the user's own motion state and the vehicle's motion state are estimated in real time, and the relative motion state is calculated based on the user's own motion state and the vehicle's motion state. S20, based on the relative motion state, calculate the danger cost function for each vehicle to score the danger level of the user's current state; S30, a preset danger rating threshold is set. If the danger rating is greater than the danger rating threshold, the optimal movement action is evaluated based on the user's speed status. S40, the optimal movement action is dynamically displayed in the AR glasses, and the optimal movement action is broadcast to the user in real time through the speaker of the headphones.

[0023] Understandably, this invention elevates the traditionally predetermined collision time to a probability distribution by introducing a probabilistic collision risk model. This model can quantitatively express the uncertainties caused by sensor noise and unclear target movement intentions, thus enabling the system to "know itself." It provides accurate warnings when perception is clear and proactively enhances risk awareness and prevents problems when perception is uncertain, greatly improving the system's robustness and safety, and reducing false alarms and missed alarms.

[0024] This invention achieves a perfect balance between global optimization and local real-time performance by integrating an improved dynamic window method with a global path planning architecture. The dynamic window method ensures that each obstacle avoidance command generated conforms to the dynamics of pedestrian movement and is physically feasible; while its multi-objective evaluation function, by balancing safety distance, hazard avoidance, direction of travel, path fit, and movement efficiency, can select the comprehensive optimal solution from a range of possible actions. The guided behavior is natural, smooth, and intuitive, enabling decisive obstacle avoidance in critical situations and ensuring a rapid return to the globally optimal path after obstacle avoidance, avoiding short-sighted behavior and local oscillations, and greatly improving user comfort and trust in the system.

[0025] This invention improves safety by transmitting obstacle avoidance content from AR glasses to headphones, preventing users from becoming sluggish due to being engrossed in the content of AR glasses.

[0026] Furthermore, in step S10, the user's current motion state and vehicle motion state are evaluated using an inertial measurement device and a camera, which is consistent with existing technology and will not be elaborated here. By inputting the user's acceleration and angular velocity measured by the inertial measurement device and the feature point tracking data captured by the camera, Kalman filtering is used to perform data fusion to obtain the user's motion state and vehicle motion state.

[0027] Its own motion state is represented as: P p = [x p , y p vx p vy p , θ p ]; The vehicle's motion state is represented as: Car i = [ID, x i , y i vx i vy i , width i , length i ]; Where, x p and y p This represents the user's 2D position in the world coordinate system, vx p and vyp θ represents the user's velocity in the x and y directions, respectively. p Indicates the pedestrian's facing angle; ID represents the marker for each vehicle, x i and y i This indicates the 2D position of the vehicle in the world coordinate system, vx i and vy i These represent the vehicle's speed in the x and y directions, respectively, and width. i , length i These represent the width and length of the vehicle, respectively.

[0028] Furthermore, based on its own motion state and the vehicle's motion state, the relative motion state is calculated. Specific steps include: The user is defined as a cylindrical collider with a radius of R. In this step, the system detects the user's chosen mode of transportation in the navigation software. If the user chooses walking, R is smaller; if they choose cycling, R is larger. This R is based on the volume of a typical adult and a non-motorized vehicle; special users can adjust it themselves within the AR glasses system.

[0029] The vehicle is defined as a rectangular collision body, with the center of the rectangular collision body being x. i , y i The size is width i ,length i Orientation refers to the direction of the vehicle's speed; Calculate the shortest distance between the boundaries of the cylindrical collider and the rectangular collider to obtain the nearest point d. min ; Calculate the relative velocity scalar v between the user and the vehicle in the direction of vehicle velocity. r ; Based on the nearest point and the relative speed scalar, calculate the precise TTC for the user and the vehicle; ; Where t represents time.

[0030] Furthermore, step S20 specifically includes the following steps: Calculate the probability of a collision between a user and a vehicle; ; Where Δt represents the future time window (3 seconds in this example). Let ξ represent the probability density function, ξ represent the standard deviation of TTC, and P represent the probability density function. o Indicates the probability of collision; Calculate the distance cost between the user and the vehicle; ; Among them, C d d represents the distance cost. safe This indicates the safe distance threshold; the safe distance threshold can be dynamically adjusted according to speed. Calculate the speed cost between the user and the vehicle; ; Among them, C v V represents the speed cost. v V represents the speed of a vehicle. max This indicates the maximum vehicle speed in the current scenario; A hazard score C is calculated based on the collision probability, the distance cost, and the speed cost. o ; ; λ1, λ2, and λ3 are all weight coefficients, obtained by training logistic regression on a large amount of collision data.

[0031] Understandably, this step calculates the collision probability, distance cost, and speed cost between the user and each vehicle separately, and then considers the weights to calculate the total danger score. This allows the user to determine if there are any vehicles that pose a danger to them, and if so, proceed to the next step.

[0032] Furthermore, step S30 specifically includes the following steps: A preset risk score threshold is used to determine whether the risk score is greater than the risk score threshold. If so, then obtain the speed window and acceleration window based on the user's speed information, and generate a dynamic window based on the speed window and acceleration window; Speed ​​window Vs: ; Acceleration window Va: ; Dynamic Window Vd: Vd = Vs∩Va in,( v, w This represents the user's speed and angular velocity. v min and v max These represent the user's minimum and maximum speeds, respectively (the system uses the speeds applicable to both walking and cycling when the user selects their mode of transportation). w max This represents the user's maximum steering angular velocity (also assigned a value based on the mode of transportation). v p and wp These represent the user's current velocity and angular velocity, respectively. a max and b max These represent the user's maximum acceleration and maximum angular acceleration, respectively. Δt Indicates the time interval of the simulation; Extract each set of velocity and angular velocity from the dynamic window to simulate the future trajectory;

[0033] Where v and w are the velocity and angular velocity extracted from the dynamic window. x(t) , y(t) and θ(t) These represent the user's predicted x-direction position, predicted y-direction position, and predicted orientation angle at time t. x p , y p and θ p These represent the user's initial x-position, initial y-position, and initial orientation angle, respectively. Each of the aforementioned future trajectories is scored to select the optimal future trajectory, and the action in the optimal future trajectory is then taken as the optimal movement action.

[0034] Where G(v, w) represents the trajectory score at that velocity and angular velocity, and He represents the direction score. θ g The direction angle from the end point of the future trajectory to the navigation target point; Cl represents the safety distance score. p t (t) represents the position of the future trajectory at time t, p y This indicates the vehicle's location; Ve indicates the user's speed; Pa indicates the path alignment score. n represents the number of sampling points, di represents the maximum distance from the i-th sampling point to the global navigation path, σ represents the deviation tolerance (the smaller σ is, the less likely the maximum distance from the sampling point to the global navigation path can be, otherwise the score will decrease; this is determined based on the developer's design philosophy); Cg represents the avoidance cost score. Co represents the risk score obtained in step S20, and α, β, γ, δ, and ε are all weight coefficients. The system learns to make decisions based on reinforcement learning.

[0035] Furthermore, in step S40, once the optimal movement action is determined, the system will display the movement path in the AR glasses in real time. A dynamic arrow will indicate the direction in the AR glasses and prompt the user to accelerate or decelerate. At the same time, it will also broadcast through the headphones connected to the AR glasses. When the direction deviates too far, a louder sound will be emitted to prompt the user, and when the direction deviates slightly, a softer sound will be emitted to prompt the user. The speed prompt is the same. The user knows whether his movement action is correct by the volume of the sound in the headphones.

[0036] In summary, the path planning method based on AR glasses and headphones in the above embodiments of the present invention can calculate the danger of the user's current path in real time and feed back the optimal movement path to the user through AR glasses and headphones. Please refer to Figure 2 The image shows a path planning system based on AR glasses and headphones according to a second embodiment of the present invention, comprising: Evaluation module 11: used to estimate the user's own motion state and the vehicle's motion state in real time through the inertial measurement device and camera on the AR glasses, and calculate the relative motion state based on the user's own motion state and the vehicle's motion state; The self-motion state is represented by P. p = [x p , y p vx p vy p , θ p The vehicle's motion state is represented as: Car i = [ID, x i , y i vx i vy i , width i , length i ]; Where, x p and y p This represents the user's 2D position in the world coordinate system, vx p and vy p θ represents the user's velocity in the x and y directions, respectively. p Indicates the pedestrian's facing angle; ID represents the marker for each vehicle, x i and y i This indicates the 2D position of the vehicle in the world coordinate system, vx i and vy i These represent the vehicle's speed in the x and y directions, respectively, and width. i , length i These represent the width and length of the vehicle, respectively. The step of calculating the relative motion state based on its own motion state and the vehicle's motion state includes the following specific steps: The user is defined as a cylindrical collider with a radius of R. The vehicle is defined as a rectangular collision body, with the center of the rectangular collision body being x. i , y i The size is width i ,length i Orientation refers to the direction of the vehicle's speed; Calculate the shortest distance between the boundaries of the cylindrical collider and the rectangular collider to obtain the nearest point d. min ; Calculate the relative velocity scalar v between the user and the vehicle in the direction of vehicle velocity. r ; Based on the nearest point and the relative speed scalar, calculate the precise TTC for the user and the vehicle; Calculation module 12: used to calculate a hazard cost function for each vehicle based on the relative motion state, so as to score the hazard level of the user's current state; The calculation module 12 is specifically used for: Calculate the collision probability, distance cost, and speed cost between the user and the vehicle; A hazard score is calculated based on the collision probability, the distance cost, and the speed cost. Scoring module 13: used to preset a danger rating threshold. If the danger rating is greater than the danger rating threshold, the optimal movement action is evaluated based on the user's speed status. The scoring module is specifically used for: A preset risk score threshold is used to determine whether the risk score is greater than the risk score threshold. If so, then obtain the speed window and acceleration window based on the user's speed information, and generate a dynamic window based on the speed window and acceleration window; Extract each set of velocity and angular velocity from the dynamic window to simulate the future trajectory; Each of the aforementioned future trajectories is scored to select the optimal future trajectory, and the action in the optimal future trajectory is then taken as the optimal movement action. Feedback module 14: used to dynamically display the optimal movement action in the AR glasses and broadcast the optimal movement action to the user in real time through the speaker of the headphones; The speaker of the headphones dynamically adjusts the volume based on the user's current movement and the offset of the optimal movement.

[0037] The present invention also proposes an electronic device, please refer to [link to relevant documentation]. Figure 3 The image shows an electronic device according to a third embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, it implements the above-described path planning method based on AR glasses and headphones.

[0038] The memory 10 includes at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10 can be an internal storage unit of an electronic device, such as the hard disk of the electronic device. In other embodiments, the memory 10 can be an external storage device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the memory 10 can include both internal and external storage units of the electronic device. The memory 10 can be used not only to store application software and various types of data installed in the electronic device, but also to temporarily store data that has been output or will be output.

[0039] In some embodiments, the processor 20 may be an electronic control unit (ECU), a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run program code stored in the memory 10 or process data, such as executing access restriction programs.

[0040] It should be pointed out that, Figure 3 The structure shown does not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0041] This invention also proposes a readable storage medium storing a computer program that, when executed by a processor, implements the path planning method based on AR glasses and headphones as described above.

[0042] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0043] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0044] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0045] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0046] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A path planning method based on AR glasses and headphones, characterized in that, Includes the following steps: The user's own motion state and the vehicle's motion state are estimated in real time through the inertial measurement unit and camera on the AR glasses, and the relative motion state is calculated based on the user's own motion state and the vehicle's motion state. Based on the relative motion state, a hazard cost function is calculated for each vehicle to score the hazard level of the user's current state; A preset danger rating threshold is set. If the danger rating is greater than the danger rating threshold, the optimal movement action is evaluated based on the user's speed status. The optimal movement action is dynamically displayed in the AR glasses, and the optimal movement action is broadcast to the user in real time through the speaker of the headphones.

2. The path planning method based on AR glasses and headphones according to claim 1, characterized in that, The AR glasses use a camera and inertial measurement unit to estimate the user's own motion state and the vehicle's motion state in real time. Based on these motion states, the relative motion state is calculated, where the user's own motion state is represented by P. p =[x p , y p vx p vy p , θ p The vehicle's motion state is represented as: Car i = [ID, x i , y i vx i vy i , width i ,length i ]; Where, x p and y p This represents the user's 2D position in the world coordinate system, vx p and vy p θ represents the user's velocity in the x and y directions, respectively. p Indicates the pedestrian's facing angle; ID represents the marker for each vehicle, x i and y i This indicates the 2D position of the vehicle in the world coordinate system, vx i and vy i These represent the vehicle's speed in the x and y directions, respectively, and width. i , length i These represent the width and length of the vehicle, respectively.

3. The path planning method based on AR glasses and headphones according to claim 2, characterized in that, The step of calculating the relative motion state based on its own motion state and the vehicle's motion state includes the following specific steps: The user is defined as a cylindrical collider with a radius of R; The vehicle is defined as a rectangular collision body, with the center of the rectangular collision body being x. i , y i The size is width i , length i Orientation refers to the direction of the vehicle's speed; Calculate the shortest distance between the boundaries of the cylindrical collider and the rectangular collider to obtain the nearest point d. min ; Calculate the relative velocity scalar v between the user and the vehicle in the direction of vehicle velocity. r ; Based on the nearest point and the relative speed scalar, the precise TTC between the user and the vehicle is calculated.

4. The path planning method based on AR glasses and headphones according to claim 3, characterized in that, Based on the relative motion state, a hazard cost function is calculated for each vehicle to score the hazard level of the user's current state. Specific steps include: Calculate the collision probability, distance cost, and speed cost between the user and the vehicle; A hazard score is calculated based on the collision probability, the distance cost, and the speed cost.

5. The path planning method based on AR glasses and headphones according to claim 4, characterized in that, If the preset danger rating threshold is greater than the danger rating threshold, then the optimal movement action is evaluated based on the user's speed status. Specific steps include: A preset risk rating threshold is used to determine whether the risk rating is greater than the risk rating threshold. If so, then obtain the speed window and acceleration window based on the user's speed information, and generate a dynamic window based on the speed window and acceleration window; Extract each set of velocity and angular velocity from the dynamic window to simulate the future trajectory; Each of the aforementioned future trajectories is scored to select the optimal future trajectory, and the action in the optimal future trajectory is then taken as the optimal movement action.

6. The path planning method based on AR glasses and headphones according to claim 5, characterized in that, In the above step, the optimal movement action is dynamically displayed in the AR glasses, and the optimal movement action is broadcast to the user in real time through the speaker of the earphone. The speaker of the earphone dynamically adjusts the volume according to the user's current movement action and the offset of the optimal movement action.

7. A path planning system based on AR glasses and headphones, characterized in that, include: Evaluation module: used to estimate the user's own motion state and the vehicle's motion state in real time through the inertial measurement unit and camera on the AR glasses, and calculate the relative motion state based on the user's own motion state and the vehicle's motion state; Calculation module: used to calculate a hazard cost function for each vehicle based on the relative motion state, in order to score the hazard level of the user's current state; Scoring module: used to preset a danger score threshold. If the danger score is greater than the danger score threshold, the optimal movement action is evaluated based on the user's speed status. Feedback module: used to dynamically display the optimal movement action in the AR glasses and broadcast the optimal movement action to the user in real time through the speaker of the headphones.

8. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the path planning method based on AR glasses and headphones as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the path planning method based on AR glasses and headphones as described in any one of claims 1-6.

Images