Airport navigation method and related device

Abstract

The invention discloses an airport navigation method and a related device, and relates to the technical field of navigation, and the method comprises the following steps: obtaining a current airport realistic picture and user flight information; wherein the user flight information comprises a navigation destination and the emergency degree of a flight state; carrying out obstacle detection on the current airport realistic image to obtain a detection result; wherein the detection result comprises an obstacle position and a confidence score; generating a navigation grid according to the obstacle position; wherein the navigation grid comprises the passing cost of the area where the obstacle is located, and the passing cost is obtained according to the mapping relation between the passing cost and the confidence score; obtaining a navigation path according to the navigation grid and the navigation destination; and according to the emergency degree of the flight state, the confidence score, the traffic cost and the height variation of the navigation path, generating an identifier for indicating the navigation path, and displaying the identifier on the current airport realistic picture. The guiding effect of airport navigation can be improved, and the problem that passengers enter wrong routes by mistake or miss flights is reduced.

Description

Technical Field

[0001] This application relates to the field of navigation technology, and in particular to an airport navigation method and related apparatus. Background Technology

[0002] Airport navigation guides passengers to their destinations (such as boarding gates, baggage claim areas, and service facilities) within the airport through route planning. Currently, airport navigation largely relies on pre-set static maps for route planning, using route planning algorithms (such as A* or Dijkstra's algorithm) to generate navigation routes and guide users to their destinations. However, when navigation guidance is ineffective, such as with unclear or incorrect instructions, it can lead passengers astray or even cause them to miss their flights. Summary of the Invention

[0003] In view of the above problems, this application provides an airport navigation method and related device to improve the guidance effect of airport navigation and reduce the occurrence of passengers going astray or missing their flights. The specific solution is as follows:

[0004] The first aspect of this application provides an airport navigation method, including:

[0005] Obtain the current real-time airport view and user flight information; wherein, the user flight information includes the navigation destination and the urgency of the flight status;

[0006] Obstacle detection is performed on the current airport real-view image to obtain detection results; wherein, the detection results include obstacle locations and confidence scores;

[0007] A navigation grid is generated based on the location of the obstacles; wherein, the navigation grid includes the passage cost of passing through the area where the obstacles are located, and the passage cost is obtained based on the mapping relationship between the passage cost and the confidence score;

[0008] A navigation path is obtained based on the navigation grid and the navigation destination;

[0009] Based on the urgency of the flight status, the confidence score, the transit cost, and the altitude change of the navigation path, an identifier indicating the navigation path is generated and displayed on the current airport real-view map.

[0010] In one possible implementation, the obstacle detection on the current airport real-view image to obtain the detection result includes:

[0011] Feature extraction is performed on the current airport real-view image to obtain a multi-scale feature map;

[0012] Channel attention is applied to the multi-scale feature map to generate a channel attention map, and spatial attention is applied to the channel attention map to generate a spatial weight matrix. An enhanced feature map is then generated based on the channel attention map and the spatial weight matrix.

[0013] A heatmap of the spatial weight matrix is ​​generated, and heatmap values ​​are obtained; wherein, the heatmap values ​​reflect the attention paid to objects in the current airport real-world image;

[0014] The detection box is obtained based on the enhanced feature map;

[0015] A decay factor for the Soft-NMS algorithm is generated based on the heatmap values; wherein the decay factor decreases when the heatmap values ​​increase and increases when the heatmap values ​​decrease.

[0016] Based on the attenuation factor, the Soft-NMS algorithm is used to perform overlapping detection box deletion processing on the detection boxes to obtain the detection results.

[0017] In one possible implementation, the identifier includes an arrow;

[0018] The step of generating an identifier indicating the navigation path based on the urgency of the flight status, the confidence score, the transit cost, and the altitude change of the navigation path, and displaying the identifier on the current airport real-view map, includes:

[0019] The pulse frequency of the light strip at the edge of the arrow is determined based on the product of the urgency level and the heat map value;

[0020] The arrow color is determined based on saturation, where the saturation is the difference between 1 and the confidence score;

[0021] The arrow width is determined based on the aforementioned passage cost;

[0022] The arrow transparency is determined based on the height change of the navigation path;

[0023] Based on the pulse frequency, arrow color, arrow width, and arrow transparency, an arrow indicating the navigation path is generated via WebXR, and the arrow is displayed on the current airport real-view map.

[0024] In one possible implementation, the feature extraction from the current airport real-view image to obtain a multi-scale feature map includes:

[0025] The MobileNetV4-UIB model is used to extract features from the current airport real-view image to obtain the multi-scale feature map;

[0026] The step of generating a channel attention map by applying channel attention to the multi-scale feature map, generating a spatial weight matrix by applying spatial attention to the channel attention map, and generating an enhanced feature map based on the channel attention map and the spatial weight matrix includes:

[0027] The target channel is determined based on the multi-scale feature map; wherein, the target channel includes a pedestrian channel;

[0028] The CBAM module is used to apply channel attention to the target channel to generate a channel attention map, and spatial attention is applied to the channel attention map to generate a spatial weight matrix. An enhanced feature map is then generated based on the channel attention map and the spatial weight matrix.

[0029] The process of generating the heatmap of the spatial weight matrix, and obtaining heatmap values, includes:

[0030] A heatmap of the spatial weight matrix is ​​generated using a GLSL shader to obtain heatmap values;

[0031] The step of obtaining the detection box based on the enhanced feature map includes:

[0032] The enhanced feature map is processed by the neck network of YOLOv6 to obtain the processed feature map;

[0033] The processed feature map is detected using the YOLOv6 detection head to obtain the detection box.

[0034] In one possible implementation, the detection result further includes obstacle type, which includes dynamic obstacles and static obstacles;

[0035] The step of generating a navigation mesh based on the obstacle positions includes:

[0036] The passage cost is determined based on the confidence score; wherein, the higher the confidence score, the higher the passage cost, and the lower the confidence score, the lower the passage cost.

[0037] Predict the position of the dynamic obstacle at a target time; wherein, the target time is N seconds after the current time;

[0038] The navigation grid is generated using a navigation grid algorithm based on the location of the static obstacles, the predicted location of the dynamic obstacles, and the passage cost.

[0039] In one possible implementation, obtaining the navigation path based on the navigation grid and the navigation destination includes:

[0040] The navigation path is obtained using the Recast Navigation algorithm based on the navigation grid and the navigation destination.

[0041] A second aspect of this application provides an airport navigation system, comprising:

[0042] The data acquisition module is used to acquire the current real-time airport image and user flight information; wherein, the user flight information includes the navigation destination and the urgency of the flight status;

[0043] An obstacle detection module is used to detect obstacles in the current airport real-view image and obtain detection results; wherein, the detection results include obstacle locations and confidence scores;

[0044] A navigation mesh generation module is used to generate a navigation mesh based on the location of the obstacle; wherein, the navigation mesh includes the passage cost of passing through the area where the obstacle is located, and the passage cost is obtained based on the mapping relationship between the passage cost and the confidence score;

[0045] A navigation path generation module is used to obtain a navigation path based on the navigation grid and the navigation destination;

[0046] The navigation module is used to generate an identifier indicating the navigation path based on the urgency of the flight status, the confidence score, the transit cost, and the altitude change of the navigation path, and to display the identifier on the current airport real-view map.

[0047] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the airport navigation method of the first aspect or any implementation thereof.

[0048] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0049] The memory is used to store computer programs;

[0050] The processor is used to execute the computer program so that the electronic device can implement the airport navigation method of the first aspect or any implementation thereof.

[0051] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to perform the airport navigation method described in the first aspect or any implementation thereof.

[0052] By employing the aforementioned technical solutions, the airport navigation method and related devices provided in this application generate navigation paths by detecting obstacles using the current real-world airport map. This approach, combined with the actual airport environment, offers greater flexibility compared to generating navigation paths from preset static maps. Even when the map changes, it can still guide passengers to the correct destination. Furthermore, it considers user flight information when generating navigation paths, allowing for the regeneration of navigation paths based on changes in the destination. This reduces the risk of passengers entering the wrong route or missing their flights. Additionally, it generates navigation path indicators based on flight status urgency, confidence score, transit cost, and changes in navigation path elevation. These indicators can alert passengers based on flight status, road congestion, and floor changes, improving the guidance effect of airport navigation, enhancing the passenger navigation experience, and increasing navigation efficiency. Attached Figure Description

[0053] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0054] Figure 1 A flowchart of an airport navigation method provided in this application;

[0055] Figure 2 A structural diagram of an airport navigation system provided in this application;

[0056] Figure 3 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0057] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0058] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0059] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0060] Currently, airport navigation systems largely rely on pre-set static maps and isolated path planning algorithms (such as A* or Dijkstra's algorithm), failing to respond in real-time to dynamic changes within the terminal (such as passenger flow, obstacle movement, and flight delays). If navigation routes are not updated synchronously after gate changes, passengers may end up on the wrong route or even miss their flights. Furthermore, traditional airport navigation systems primarily use 2D displays, offering limited information presentation and making it difficult for passengers to intuitively understand complex spatial layouts (such as multi-level terminals and intersecting passageways). They also lack interactive methods such as AR (Augmented Reality), VR (Virtual Reality), and MR (Mixed Reality), reducing navigation efficiency. Airport terminals are vast and multi-dimensional, making it difficult for traditional 2D planes to intuitively represent complex spatial relationships. Given the dense crowds and complex environment within terminals, generating optimal routes by combining flight status (such as delays and gate adjustments) and real-time environmental perception (such as obstacle detection) is a pressing issue that needs to be addressed.

[0061] Based on this, this application provides an airport navigation method to solve the above problems.

[0062] The airport navigation method provided in this application is based on the WebXR API interface. WebXR (Web Extended Reality) is a browser-based JavaScript API used to implement virtual reality (VR), augmented reality (AR), and mixed reality (MR) experiences on web pages. Immersive 3D applications can be built directly in the browser without the need to install additional plugins or applications. Users can achieve immersive navigation simply by using compatible devices such as VR headsets, AR glasses, or smartphones.

[0063] Reference Figure 1 , Figure 1 This application provides a flowchart illustrating an airport navigation method as an embodiment of the present application. Figure 1As shown in the embodiment of this application, an airport navigation method may include steps 101 to 105, which are described in detail below.

[0064] Step 101: Obtain the current airport view and user flight information; the user flight information includes the navigation destination and the urgency of the flight status.

[0065] The current airport view image can be a real-time airport view image captured by the user through the camera of a mobile terminal such as a mobile phone or tablet. The airport view image is an image of the environment around the user's location, reflecting the user's location and surrounding environment information. The current airport view image can reflect information such as the density of people and airport facilities.

[0066] User flight information can be obtained through user identification information. This information includes the navigation destination, which can be the user-input destination or automatically obtained after changes to the navigation destination, such as the boarding gate. The information also includes the urgency level of the flight status, which may include, but is not limited to, gate closing time, flight delay, gate change, and flight cancellation. The urgency level is determined based on the flight status. For example, the urgency range is 0-1, with 1 representing the highest urgency and 0 representing the lowest. An urgency level of 1 is given when the gate is about to close (e.g., 30 minutes before closing). An urgency level of 0.7 is given when the flight is delayed beyond a preset time (e.g., more than one hour). An urgency level of 0.5 is given when the gate changes. An urgency level of 0.9 is given when the flight is cancelled. These values ​​are merely illustrative and can be adjusted according to actual needs; this application does not impose specific limitations on them.

[0067] Optionally, the system can retrieve the current airport view and the user's flight information after responding to the user's navigation command. User navigation commands can be generated via voice, gestures, or touch input. For example, a user can ask for the boarding gate by voice, swipe on the map by gesture, or directly input their navigation destination.

[0068] Step 102: Perform obstacle detection on the current airport real-view image to obtain the detection results; the detection results include obstacle locations and confidence scores.

[0069] The obstacles to be detected may include, but are not limited to, people, luggage, and airport facilities. These airport facilities may include passenger service facilities, operational support facilities, and commercial support facilities, such as security screening equipment and service counters.

[0070] Image detection methods can be used to detect objects and obtain bounding boxes and confidence scores. The bounding boxes reflect the location of the objects, and the confidence scores reflect the degree of confidence that the objects in the bounding boxes are obstacles.

[0071] In one possible implementation, obstacle detection is performed on the current real-world image of the airport to obtain the detection results, including:

[0072] Feature extraction is performed on the current real-view image of the airport to obtain a multi-scale feature map;

[0073] Channel attention maps are generated by applying channel attention to multi-scale feature maps, and spatial attention is applied to channel attention maps to generate spatial weight matrices. Enhanced feature maps are then generated based on the channel attention maps and spatial weight matrices.

[0074] A heatmap of the spatial weight matrix is ​​generated, and heatmap values ​​are obtained; the heatmap values ​​reflect the attention paid to objects in the current airport real-view image.

[0075] The detection box is obtained based on the enhanced feature map;

[0076] The decay factor of the Soft-NMS algorithm is generated based on the heatmap values; wherein the decay factor decreases when the heatmap values ​​increase and increases when the heatmap values ​​decrease.

[0077] Based on the attenuation factor, the Soft-NMS algorithm is used to remove overlapping detection boxes to obtain the detection results.

[0078] Optionally, the MobileNetV4-UIB model can be used to extract features from the current airport real-world image to obtain multi-scale feature maps.

[0079] This application replaces the YOLOv6 backbone with the MobileNetV4-UIB model, improving optimization for mobile device hardware such as smartphones, providing native WebGL, multimodal extension capabilities, and excellent compatibility with WebXR. After INT8 quantization, the size is only 2.1MB, improving the ability to recognize dense crowds. The introduction of UIB blocks (Universal Inverted Bottleneck) enhances the ability to recognize occluded objects while reducing scanning latency. Users use mobile device cameras (at least 12 megapixels) or tablets to capture images in real-time while facing forward. These images are input into the MobileNetV4-UIB backbone, which outputs multi-scale feature maps. UIB blocks can accurately segment occluded targets, preventing temporary obstacles such as cleaning vehicles from being missed.

[0080] When using the MobileNetV4-UIB model to extract features from real-world airport images, the images, such as photos of the terminal building, are compressed to a preset resolution. Brightness enhancement and anti-glare processing are also applied. Then, the image is split: a 3×9 convolutional kernel is used for vertical scanning to identify standing people, pillars, and other vertical objects; a 9×3 convolutional kernel is used for horizontal scanning to identify baggage carts, conveyor belts, and other horizontal objects. The images are then stitched together, combining the vertical and horizontal scan results. Channel augmentation is applied to enhance detail, depth scanning extracts features, and weighting emphasizes important features, resulting in a multi-scale feature map.

[0081] Optionally, channel attention is applied to the multi-scale feature map to generate a channel attention map, and spatial attention is applied to the channel attention map to generate a spatial weight matrix. An enhanced feature map is then generated based on the channel attention map and the spatial weight matrix, including:

[0082] Target channels are determined based on multi-scale feature maps; among which, pedestrian channels are included.

[0083] The CBAM module is used to generate a channel attention map by applying channel attention to the target channel, and a spatial weight matrix is ​​generated by applying spatial attention to the channel attention map. An enhanced feature map is then generated based on the channel attention map and the spatial weight matrix.

[0084] Enabling CBAM on target channels, such as pedestrian channels or pedestrian-related channels, can reduce energy consumption. For other channels that are not target channels, detection boxes can be obtained directly through the YOLOv6 model after obtaining multi-scale feature maps.

[0085] In the YOLOv6 model, the Convolutional Block Attention Module (CBAM) is introduced. In occlusion scenarios, the spatial attention of the CBAM module helps the model focus on the visible parts of pedestrians (such as heads or legs), increasing occlusion detection capabilities. However, to reduce the overhead of introducing this module, it needs to be improved. Specifically, the CBAM module should be enabled for target channels, such as pedestrian channels or pedestrian-related channels, to reduce energy consumption. The CBAM module has a dual-channel dimension. Channel attention determines which feature channels are more important; for example, in an airport terminal scene, the weight of the human detection channel should be higher than that of the ceiling texture channel. Spatial attention determines which locations in the image are more important. Spatial attention takes the output of channel attention as input, performs max pooling and average pooling to obtain two feature maps, concatenates the two feature maps, and then uses a 7x7 convolution to calculate spatial weights. This is equivalent to identifying key areas in the image; for example, a running person is more important than a stationary billboard. Channel attention first filters important feature types, and spatial attention then finds important regions within these features.

[0086] Optionally, after obtaining the spatial weight matrix, a heatmap of the spatial weight matrix can be generated using a GLSL shader to obtain heatmap values. These heatmap values ​​reflect the level of attention paid to objects in the current airport view.

[0087] A heat map is a visual representation of areas that require attention. The process of generating a heat map involves identifying the most prominent areas (such as a red suitcase) and areas with common characteristics (such as areas where crowds gather) in the image, combining and analyzing these two pieces of information to ultimately generate an "attention distribution map." The intensity of the color indicates the degree of attention required. The heat map can also label objects in the image with a level of danger, such as red for avoidance and green for passage.

[0088] In practical applications, red areas on a heatmap can indicate high-response areas (such as the head or luggage), while green areas can indicate medium-response areas (such as the legs or torso). Heatmaps can be used to identify areas of interest in terminal scene images, such as key parts of moving human bodies: head (high-response area), legs (medium-response area), potential obstacles, baggage cart handles, edges of security equipment, running children, and sliding luggage.

[0089] Optionally, the detection box is obtained based on the enhanced feature map, including:

[0090] The enhanced feature map is processed by the neck network of YOLOv6 to obtain the processed feature map;

[0091] The processed feature map is detected using the YOLOv6 detection head to obtain detection boxes.

[0092] After obtaining the detection boxes, the Soft-NMS algorithm can be used to remove overlapping detection boxes. When performing this removal process, an attenuation factor is generated based on the heatmap values. This attenuation factor decreases as the heatmap values ​​increase and increases as the heatmap values ​​decrease. Then, based on this attenuation factor, the Soft-NMS algorithm is used to remove overlapping detection boxes, yielding the detection results.

[0093] For pedestrian occlusion scenarios at airports, the attenuation factor sigma of the Soft-NMS algorithm can be dynamically adjusted. This attenuation factor sigma is related to the confidence of the detection box and the degree of occlusion. Combined with the attention heatmap information provided by the CBAM module, the attenuation function of the overlapping boxes is weighted and optimized to reduce the missed detection problem caused by dense pedestrian occlusion and further improve the pedestrian detection recall rate under occlusion conditions.

[0094] The heatmap values ​​affect the sigma parameter of the Soft-NMS algorithm, thus the attenuation factor sigma of the Soft-NMS algorithm is dynamically calculated from the CBAM heatmap response values. The sigma calculation formula can be sigma = 0.8 - 0.5 × M_cbam, where M_cbam is the heatmap value, M_cbam∈[0,1]. Then, the final score of Soft-NMS is obtained based on the sigma parameter.

[0095] For example, a heat value greater than or equal to 0.6 indicates that the area corresponds to a highly significant target (such as a head), and the soft-nms attenuation is weak, which can preserve the detection box; 0.3 < heat value < 0.6 indicates a passable area, and the path can pass normally; a heat value less than or equal to 0.3 indicates a low-risk area, and the soft-nms attenuation is strong, which can suppress false detections.

[0096] If the detection boxes output by Soft-NMS have a score of 0.3 ≤ score < 0.5, these may be occluded or blurred targets. Instead of simply ignoring them during path planning, they are converted into different levels of passage costs; finally, risk warnings are provided through AR visualization. For detection boxes with a score of 0.4 ≤ score < 0.5, a preventative path with a 1-meter avoidance radius can be generated in Recast Navigation. For detection boxes with a score of 0.3 ≤ score < 0.4, the passage cost can be increased, for example, by three times. For moving targets, a predictive avoidance area can be generated by predicting the target's trajectory. These adjustments can be made based on actual business needs, and this application does not impose specific limitations on them. Ultimately, these changes will correspond to changes in network parameters on the Recast Navigation algorithm, thus affecting path planning.

[0097] Soft-NMS outputs a series of bounding boxes (object locations) along with their confidence scores and object types (e.g., pedestrians, luggage carts, etc.). A confidence score greater than or equal to 0.5 indicates a real obstacle that needs to be avoided. A confidence score greater than or equal to 0.3 and less than 0.5 suggests a possible obstacle, but this is not entirely certain and can be marked as a risk area. A confidence score less than 0.3 indicates that the object is not an obstacle and can be ignored.

[0098] Optionally, the detection results may also include obstacle types, which include dynamic obstacles and static obstacles.

[0099] Step 103: Generate a navigation grid based on the location of obstacles; wherein, the navigation grid includes the passage cost of passing through the area where the obstacle is located, and the passage cost is obtained based on the mapping relationship between passage cost and confidence score.

[0100] In one possible implementation, the navigation mesh is generated based on the obstacle locations, including:

[0101] The passage cost is determined based on the confidence score; where the higher the confidence score, the higher the passage cost, and the lower the confidence score, the lower the passage cost.

[0102] Predict the position of dynamic obstacles at a target time; where the target time is N seconds after the current time.

[0103] A navigation grid is generated using a navigation grid algorithm based on the location of static obstacles, the predicted location of dynamic obstacles, and the passage cost.

[0104] The cost of passage is determined based on the confidence score. Areas with a confidence score greater than a threshold (e.g., 0.5) are considered solid obstacles (e.g., cylinders) and will be completely avoided during path planning. For example, a pedestrian (confidence score greater than the threshold) would be placed near a cylindrical obstacle with a radius of 1 meter, and the path would bypass this cylinder. Areas with a confidence score less than the threshold (e.g., confidence score between 0.3 and 0.5) are not considered solid obstacles but are marked as high-cost passage areas; the path can pass through them but will try to avoid them. For example, a vaguely defined suitcase (confidence score less than the threshold) would be marked as a circular area, and a path through this area would incur higher costs. During path planning, if the cost of detouring is lower, the detour will be chosen.

[0105] For moving obstacles (such as pedestrians or sliding suitcases), their future position can be predicted (e.g., 2 seconds later), and the position of the obstacle can be dynamically updated so that the path planning can avoid them in advance.

[0106] When updating the navigation mesh, the Recast algorithm can be used. Recast avoids rebuilding the entire map's navigation mesh, instead updating only a local area around obstacles (e.g., within a 20-meter radius), which significantly improves the navigation mesh generation speed. After updating, the navigation mesh includes information on obstacles and the passage cost area.

[0107] Step 104: Obtain the navigation path based on the navigation grid and navigation destination.

[0108] The Recast Navigation algorithm can be used to obtain the navigation path based on the navigation grid and the navigation destination.

[0109] As the airport's real-time map and obstacle locations change, or as boarding gates change, the navigation path is updated in real time. Specifically, based on the updated navigation grid, Recast recalculates the path from the origin to the destination. The new path automatically avoids solid obstacles and, as far as possible, high-cost areas. The navigation path is a series of 3D coordinate points, which can be output to the WebXR engine and displayed as a new navigation route on AR glasses or mobile phones. The entire process, from detection to path planning completion, takes only tens of milliseconds, ensuring users receive a safe navigation path in real time. For example, if Soft-NMS detects a pedestrian with a confidence score of 0.8, Recast places an obstacle at the pedestrian's location, and the path immediately avoids it. When Soft-NMS detects a suitcase with a confidence score of 0.4, Recast marks a high-cost area around the suitcase; the generated navigation path will take a slightly longer route, but will not completely avoid it. If a detour is less costly, it will be avoided.

[0110] Optionally, during flight delays, low-congestion routes can be recommended. If the delay exceeds a certain duration, the path weight for the dining area or rest area can be increased to guide passengers there. If the boarding gate changes, passengers can be guided to the new gate. In case of emergency evacuation, obstacle avoidance routes can be generated quickly.

[0111] This application generates navigation routes by detecting obstacles using the current real-world airport map. This method combines the actual airport environment with route navigation, making it more flexible than generating navigation routes using a preset static map. It can also guide passengers to the correct destination even when the map changes. Furthermore, it takes user flight information into account when generating navigation routes. If the navigation destination changes, the navigation route can be regenerated based on the changed destination, reducing the chances of passengers going astray or missing their flights.

[0112] Step 105: Based on the urgency of the flight status, confidence score, transit cost, and altitude change of the navigation path, generate an indicator for the navigation path and display the indicator on the current airport view map.

[0113] Optionally, the identifier can be an arrow.

[0114] In one possible implementation, a navigation path indicator is generated based on the urgency of the flight status, confidence score, transit cost, and altitude change of the navigation path. This indicator is then displayed on the current airport view map, including:

[0115] The pulse frequency of the light strip at the edge of the arrow is determined by the product of the urgency level and the heat map value;

[0116] The arrow color is determined based on saturation, where saturation is the difference between 1 and the confidence score;

[0117] The arrow width is determined based on the toll cost;

[0118] The arrow transparency is determined based on the height change of the navigation path;

[0119] Based on the pulse frequency, arrow color, arrow width, and arrow transparency, an arrow indicating the navigation path is generated using WebXR and displayed on the current airport view map.

[0120] This application uses RGBA spatiotemporal encoding to generate arrow colors. In a WebXR 3D model, RGBA represents the color values ​​that make up a 3D path indicator, and the four letters RGBA are related to algorithms and business logic. R represents the red component, G the green component, B the blue component, and A the transparency. The arrow's color is determined by the red, green, blue components, and transparency. Specifically, the red component is determined by the product of urgency and heatmap value; the green component is determined by the difference between 1 and the confidence score; the blue component is determined by the passage cost; and the transparency is determined by the change in altitude along the navigation path.

[0121] When determining the pulse frequency of the light strip along the arrow's edge, the higher the urgency and heatmap value, the faster the pulse frequency. This pulse frequency can be the red light pulse frequency. For example, the red light pulse frequency can vary from 0.3Hz to 3Hz. A flashing red light can indicate a gate change, delay, or impending gate closure to passengers. A red light pulse frequency of 0.3Hz can indicate a flight delay, while a red light pulse frequency of 3Hz can indicate an impending gate closure.

[0122] When determining arrow colors based on saturation, the saturation level is the difference between 1 and the confidence score. A higher confidence score indicates higher saturation, and a lower confidence score indicates lower saturation. Green has lower saturation than yellow, and yellow has lower saturation than red. A low confidence score (less than 0.4) indicates a lower probability of an obstacle, and is visualized as green. A confidence score between 0.4 and 0.8 indicates a possible obstacle, and is visualized as yellow. A high confidence score (greater than 0.8) indicates a higher probability of an obstacle, and is visualized as red. Color can provide a visual indication of congestion or danger areas. Red indicates greater congestion or danger than yellow, and yellow indicates greater congestion or danger than green. Green represents low risk, yellow represents medium risk (with auxiliary elements such as a semi-transparent light wall (1 meter high) can be added), and red represents high risk (with auxiliary elements such as a rotating warning cone (2 meters high) can be added).

[0123] When determining the arrow width based on the cost of passage, the higher the cost of passage, the wider the arrow, indicating a longer detour distance or higher time cost, which can also be represented by dark blue; the lower the cost of passage, the narrower the arrow, indicating a shorter detour distance or lower time cost, which can also be represented by light blue.

[0124] When determining arrow transparency based on changes in the height of the navigation path, if the height of the navigation path remains unchanged, assuming the current user is on the first floor of the terminal and the boarding gate is on the second or third floor, the arrow transparency is 1 (completely opaque) on the first floor. When the user is shown going up the escalator, the height of the navigation path changes, and the arrow transparency decreases, such as becoming 0.3 or even 0. The arrow becomes transparent to indicate that the user's height is changing. When the user reaches the second floor, the height of the navigation path no longer changes, and the arrow transparency becomes 1 again, returning to an opaque state.

[0125] This application generates navigation path indicators based on flight status urgency, confidence score, transit cost, and altitude changes along the navigation path. It can alert passengers based on flight status, road congestion, and floor changes, improving the guidance effect of airport navigation, enhancing the passenger navigation experience, and increasing navigation efficiency.

[0126] In practical applications, when the boarding gate changes, a collaborative reset of the MobileNetV4-UIB / CBAM / Soft-NMS / Recast modules is triggered. MobileNetV4-UIB replaces the YOLOv6 backbone network, making YOLOv6 lighter, increasing the detection capability of small targets, improving compatibility with terminal building pillars / escalators, and reducing scanning latency. Heatmaps are used to dynamically adjust the attenuation intensity of Soft-NMS to reduce the missed detection rate of occluded targets. The CBAM module focuses attention, thus influencing the Soft-NMS decision-making. The detection results processed by Soft-NMS are input as dynamic obstacles into Recast Navigation to update the navigation grid in real time and generate new paths. Areas with confidence scores below a threshold trigger "preventative avoidance paths," while areas with confidence scores above the threshold trigger avoidance. In WebXR, the planned path is color-coded (RGBA four-dimensional compilation) according to its status (e.g., normal, preventative avoidance, emergency avoidance) and flight information (e.g., delay level, boarding gate change). In the 3D model, the path is displayed with a corresponding coded color.

[0127] This application converts all algorithm models into JavaScript files for browser loading, avoiding navigation delays caused by frequent server interactions. Utilizing glTF format and texture compression, the terminal 3D model size is reduced to below 50MB. Level of Detail (LOD) technology is used to dynamically load models of varying precision, adapting to the limited bandwidth and computing power of mobile devices. Model quantization (FP32 → INT8) significantly reduces model size, ensuring fast loading of the terminal on slow network speeds. Simultaneously, accuracy loss is maintained at <1%, integer calculation speed is increased by 2-4 times, and energy consumption is reduced by 60%, making it suitable for mobile phones, AR devices, and GPU (Graphics Processing Unit) memory reuse strategies. The 3D model is bound to flight data (such as gate changes and delay notifications), allowing for display or voice announcements on the model when events are triggered.

[0128] This application presents a WebXR-based flight data-driven multi-algorithm collaborative dynamic path planning system for airports. It enables real-time dynamic path planning in airport environments, accurately identifies real-time passenger density, and achieves real-time synchronization of dynamic path planning with flight information. Replacing the YOLOv6 backbone network algorithm with the MobileNetV4-UIB model demonstrates superior performance in occlusion detection, small target recognition, and mobile GPU performance optimization. Combined with Recast Navigation's dynamic navigation mesh update capabilities, it can respond in real-time to changes in the airport environment (such as obstacle movement and equipment scheduling), ensuring the accuracy and adaptability of path planning. Simultaneously, the system utilizes WebXR technology to achieve high-precision 3D airport model AR / VR rendering and enhances user convenience through multimodal input methods such as voice and gestures.

[0129] The above describes an airport navigation method provided by an embodiment of this application. The following will describe a system that performs the above airport navigation method.

[0130] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an airport navigation system provided in an embodiment of this application. Figure 2 As shown, the airport navigation system includes:

[0131] The data acquisition module 201 is used to acquire the current real-time airport image and user flight information; among which, the user flight information includes the navigation destination and the urgency of the flight status.

[0132] The obstacle detection module 202 is used to perform obstacle detection on the current airport real-view image and obtain the detection results; the detection results include the obstacle location and confidence score.

[0133] The navigation mesh generation module 203 is used to generate a navigation mesh based on the location of obstacles; wherein, the navigation mesh includes the passage cost of passing through the area where the obstacle is located, and the passage cost is obtained according to the mapping relationship between passage cost and confidence score.

[0134] The navigation path generation module 204 is used to obtain the navigation path based on the navigation grid and the navigation destination.

[0135] The navigation module 205 is used to generate an indicator for the navigation path based on the urgency of the flight status, confidence score, transit cost, and altitude change of the navigation path, and to display the indicator on the current airport view map.

[0136] In one possible implementation, the obstacle detection module 202 is specifically used for:

[0137] Feature extraction is performed on the current real-view image of the airport to obtain a multi-scale feature map;

[0138] Channel attention maps are generated by applying channel attention to multi-scale feature maps, and spatial attention is applied to channel attention maps to generate spatial weight matrices. Enhanced feature maps are then generated based on the channel attention maps and spatial weight matrices.

[0139] A heatmap of the spatial weight matrix is ​​generated, and heatmap values ​​are obtained; the heatmap values ​​reflect the attention paid to objects in the current airport real-view image.

[0140] The detection box is obtained based on the enhanced feature map;

[0141] The decay factor of the Soft-NMS algorithm is generated based on the heatmap values; wherein the decay factor decreases when the heatmap values ​​increase and increases when the heatmap values ​​decrease.

[0142] Based on the attenuation factor, the Soft-NMS algorithm is used to remove overlapping detection boxes to obtain the detection results.

[0143] Optional, the identifier may include an arrow;

[0144] In one possible implementation, navigation module 205 is specifically used for:

[0145] The pulse frequency of the light strip at the edge of the arrow is determined by the product of the urgency level and the heat map value;

[0146] The arrow color is determined based on saturation, where saturation is the difference between 1 and the confidence score;

[0147] The arrow width is determined based on the toll cost;

[0148] The arrow transparency is determined based on the height change of the navigation path;

[0149] Based on the pulse frequency, arrow color, arrow width, and arrow transparency, an arrow indicating the navigation path is generated using WebXR and displayed on the current airport view map.

[0150] In one possible implementation, the obstacle detection module 202 is also used for:

[0151] The MobileNetV4-UIB model is used to extract features from the current airport real-view image to obtain multi-scale feature maps;

[0152] Channel attention maps are generated by applying channel attention to multi-scale feature maps, and spatial attention is applied to these channel attention maps to generate spatial weight matrices. Enhanced feature maps are then generated based on the channel attention maps and the spatial weight matrices, including:

[0153] Target channels are determined based on multi-scale feature maps; among which, pedestrian channels are included.

[0154] The CBAM module is used to generate a channel attention map by applying channel attention to the target channel, and a spatial weight matrix is ​​generated by applying spatial attention to the channel attention map. An enhanced feature map is then generated based on the channel attention map and the spatial weight matrix.

[0155] Generate a heatmap of the spatial weight matrix to obtain heatmap values, including:

[0156] A heatmap of the spatial weight matrix is ​​generated using a GLSL shader, and the heatmap values ​​are obtained.

[0157] The detection bounding box is obtained based on the enhanced feature map, including:

[0158] The enhanced feature map is processed by the neck network of YOLOv6 to obtain the processed feature map;

[0159] The processed feature map is detected using the YOLOv6 detection head to obtain detection boxes.

[0160] Optionally, the detection results may also include obstacle types, which include dynamic obstacles and static obstacles.

[0161] In one possible implementation, the navigation mesh generation module 203 is specifically used for:

[0162] The passage cost is determined based on the confidence score; where the higher the confidence score, the higher the passage cost, and the lower the confidence score, the lower the passage cost.

[0163] Predict the position of dynamic obstacles at a target time; where the target time is N seconds after the current time.

[0164] A navigation grid is generated using a navigation grid algorithm based on the location of static obstacles, the predicted location of dynamic obstacles, and the passage cost.

[0165] In one possible implementation, the navigation path generation module 204 is used for:

[0166] The navigation path is obtained using the Recast Navigation algorithm based on the navigation grid and the navigation destination.

[0167] This application also provides an electronic device in its embodiments. (See reference...) Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0168] like Figure 3 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. When the electronic device is powered on, the RAM 303 also stores various programs and data required for the operation of the electronic device. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0169] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, memory cards, hard drives, etc.; and communication devices 309. Communication device 309 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0170] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the airport navigation methods provided in this application.

[0171] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the airport navigation methods provided in this application.

[0172] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0174] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0175] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. An airport navigation method, characterized in that, include: Obtain the current real-time airport view and user flight information; wherein, the user flight information includes the navigation destination and the urgency of the flight status; Obstacle detection is performed on the current airport real-view image to obtain detection results; wherein, the detection results include obstacle locations and confidence scores; A navigation grid is generated based on the location of the obstacles; wherein, the navigation grid includes the passage cost of passing through the area where the obstacles are located, and the passage cost is obtained based on the mapping relationship between the passage cost and the confidence score; A navigation path is obtained based on the navigation grid and the navigation destination; Based on the urgency of the flight status, the confidence score, the transit cost, and the altitude change of the navigation path, an identifier indicating the navigation path is generated and displayed on the current airport real-view map.

2. The airport navigation method according to claim 1, characterized in that, The obstacle detection process on the current airport real-view image, to obtain the detection results, includes: Feature extraction is performed on the current airport real-view image to obtain a multi-scale feature map; Channel attention maps are generated by applying channel attention to the multi-scale feature maps, and spatial attention is applied to the channel attention maps to generate spatial weight matrices. Enhanced feature maps are then generated based on the channel attention maps and the spatial weight matrices. A heatmap of the spatial weight matrix is ​​generated, and heatmap values ​​are obtained; wherein, the heatmap values ​​reflect the attention paid to objects in the current airport real-world image; The detection box is obtained based on the enhanced feature map; A decay factor for the Soft-NMS algorithm is generated based on the heatmap values; wherein the decay factor decreases when the heatmap values ​​increase and increases when the heatmap values ​​decrease. Based on the attenuation factor, the Soft-NMS algorithm is used to perform overlapping detection box deletion processing on the detection boxes to obtain the detection results.

3. The airport navigation method according to claim 2, characterized in that, The identifier includes an arrow; The step of generating an identifier indicating the navigation path based on the urgency of the flight status, the confidence score, the transit cost, and the altitude change of the navigation path, and displaying the identifier on the current airport real-view map, includes: The pulse frequency of the light strip at the edge of the arrow is determined based on the product of the urgency level and the heat map value; The arrow color is determined based on saturation, where the saturation is the difference between 1 and the confidence score; The arrow width is determined based on the aforementioned passage cost; The arrow transparency is determined based on the height change of the navigation path; Based on the pulse frequency, arrow color, arrow width, and arrow transparency, an arrow indicating the navigation path is generated via WebXR, and the arrow is displayed on the current airport real-view map.

4. The airport navigation method according to claim 2, characterized in that, The step of extracting features from the current airport real-view image to obtain a multi-scale feature map includes: The MobileNetV4-UIB model is used to extract features from the current airport real-view image to obtain the multi-scale feature map; The step of generating a channel attention map by applying channel attention to the multi-scale feature map, generating a spatial weight matrix by applying spatial attention to the channel attention map, and generating an enhanced feature map based on the channel attention map and the spatial weight matrix includes: The target channel is determined based on the multi-scale feature map; wherein, the target channel includes a pedestrian channel; The CBAM module is used to apply channel attention to the target channel to generate a channel attention map, and spatial attention is applied to the channel attention map to generate a spatial weight matrix. An enhanced feature map is then generated based on the channel attention map and the spatial weight matrix. The process of generating the heatmap of the spatial weight matrix, and obtaining heatmap values, includes: A heatmap of the spatial weight matrix is ​​generated using a GLSL shader to obtain heatmap values; The step of obtaining the detection box based on the enhanced feature map includes: The enhanced feature map is processed by the neck network of YOLOv6 to obtain the processed feature map; The processed feature map is detected using the YOLOv6 detection head to obtain the detection box.

5. The airport navigation method according to any one of claims 1 to 4, characterized in that, The detection results also include obstacle types, which include dynamic obstacles and static obstacles; The step of generating a navigation mesh based on the obstacle positions includes: The passage cost is determined based on the confidence score; wherein, the higher the confidence score, the higher the passage cost, and the lower the confidence score, the lower the passage cost. Predict the position of the dynamic obstacle at a target time; wherein, the target time is N seconds after the current time; The navigation grid is generated using a navigation grid algorithm based on the location of the static obstacles, the predicted location of the dynamic obstacles, and the passage cost.

6. The airport navigation method according to any one of claims 1 to 4, characterized in that, The step of obtaining a navigation path based on the navigation grid and the navigation destination includes: The navigation path is obtained using the Recast Navigation algorithm based on the navigation grid and the navigation destination.

7. An airport navigation system, characterized in that, include: The data acquisition module is used to acquire the current real-time airport image and user flight information; wherein, the user flight information includes the navigation destination and the urgency of the flight status; An obstacle detection module is used to perform obstacle detection on the current airport real-view image and obtain detection results; wherein, the detection results include obstacle location and confidence score; A navigation mesh generation module is used to generate a navigation mesh based on the location of the obstacle; wherein, the navigation mesh includes the passage cost of passing through the area where the obstacle is located, and the passage cost is obtained according to the mapping relationship between the passage cost and the confidence score; A navigation path generation module is used to obtain a navigation path based on the navigation grid and the navigation destination; The navigation module is used to generate an identifier indicating the navigation path based on the urgency of the flight status, the confidence score, the transit cost, and the altitude change of the navigation path, and to display the identifier on the current airport real-view map.

8. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the airport navigation method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the airport navigation method as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the airport navigation method as described in any one of claims 1 to 6.

Images