A multi-modal intelligent guide route planning method and system

Abstract

This invention discloses a multimodal intelligent guided tour route planning method and system, belonging to the field of intelligent path planning. The method includes: acquiring static GIS network topology data of the target geographical area; dynamically updating the environmental map data structure based on environmental parameters to generate a multidimensional dynamic weight matrix; receiving start-point and end-point requests, and calculating the comprehensive environmental cost of candidate nodes based on the multidimensional dynamic weight matrix using an improved multi-objective heuristic algorithm; optimizing candidate nodes according to the comprehensive environmental cost to generate an optimal guided tour route containing a node sequence and its corresponding travel timestamps; and generating a navigation instruction set based on the optimal guided tour route. This invention improves the overall traffic efficiency of scenic areas and the balance of visitor experience.

Description

Technical Field

[0001] This invention relates to the field of intelligent route planning, and in particular to a multimodal intelligent tour guide route planning method and system. Background Technology

[0002] With the rapid development of the tourism industry and the increasing demand for personalized experiences from tourists, scenic area tour route planning has become an important research topic in the field of smart tourism. How to plan an efficient and comfortable tour route for tourists in a complex scenic area road network directly affects the tourist experience and the operational efficiency of the scenic area. Currently, existing tour route planning schemes mainly rely on path search based on static geographic information system data. Their core technology uses the physical distance of road segments as the sole or primary cost evaluation indicator, employing Dijkstra's algorithm or traditional A / B algorithm. The algorithm solves the shortest path on the road network graph model. This type of solution is relatively mature in implementation and can quickly output a route with the shortest physical distance or the shortest travel time given a starting point and a destination.

[0003] However, existing tour route planning schemes have significant shortcomings in terms of multidimensional environmental perception, forward-looking nature of time-varying environments, response efficiency to sudden events, fault tolerance of sensor failures, and traffic balancing for multiple concurrent users, making it difficult to meet the actual needs of high-quality and robust tour guide services in smart scenic area scenarios. Summary of the Invention

[0004] One of the objectives of this invention is to provide a multimodal intelligent tour guide route planning method and system to solve the problem that existing tour guide route planning relies solely on static physical distance as a single cost indicator and cannot perceive and comprehensively evaluate multidimensional environmental factors in real time.

[0005] This invention is achieved through the following technical solution: a multimodal intelligent navigation route planning method, comprising the following steps: acquiring static GIS network topology data of a target geographical area, and a first data stream collected in real time by multiple sensor nodes distributed within the target geographical area, wherein the first data stream includes real-time video frame sequences, environmental audio sound pressure level data, and meteorological parameter data; performing multi-channel feature extraction logic on the first data stream to obtain environmental parameters including at least a shadow occlusion coefficient, a congestion penalty variable, a logarithmic mean, a current illuminance value, and a temperature gradient value; and dynamically updating the environmental map data structure based on the environmental parameters. The system generates a multi-dimensional dynamic weight matrix; receives start-point and end-point requests; and, based on the multi-dimensional dynamic weight matrix, uses an improved multi-objective heuristic algorithm to calculate the comprehensive environmental cost of candidate nodes, whereby the comprehensive environmental cost is determined by physical distance, congestion, acoustic environment, and meteorological conditions. Based on the comprehensive environmental cost, the system optimizes the candidate nodes to generate an optimal guided route containing a node sequence and its corresponding travel timestamps. Based on the optimal guided route, a navigation instruction set is generated, and environmental parameters are processed along the node sequence to generate an experience comfort curve. The navigation instruction set and the experience comfort curve are then output to a display terminal.

[0006] Furthermore, the environmental map data structure includes a static topology connection table and a multi-channel environmental state array that establishes a mapping relationship with the static topology connection table; the static topology connection table is used to record the physical connectivity and physical distance attributes between nodes within the target geographical area; the multi-channel environmental state array is used to store the current environmental parameters of the corresponding road segments of each edge according to different environmental perception dimensions.

[0007] Further, generating the multidimensional dynamic weight matrix includes: reading the current values ​​of multiple independent memory channels corresponding to each edge in the multi-channel environment state array according to the edge index in the static topology connection table; organizing the physical distance attribute, shadow occlusion coefficient, congestion penalty variable, logarithmic mean, current illumination lumen value and temperature gradient value corresponding to the same edge into multi-channel state data for that edge; and combining the multi-channel state data of all edges into the multidimensional dynamic weight matrix.

[0008] Furthermore, the guided route planning method also includes: local recalculation logic, monitoring the rate of change of the comprehensive environmental cost of any side in the multi-channel environmental state array between adjacent timestamps; when the rate of change exceeds a preset mutation threshold, determining that the side has undergone an environmental mutation; constructing a spatial bounding box based on the location of the side where the environmental mutation occurred, and performing the re-evaluation of the improved multi-objective heuristic algorithm only on route nodes located within the spatial bounding box that have not yet been passed.

[0009] Furthermore, constructing the spatial bounding box includes: determining the radius of the spatial bounding box based on the extent to which the rate of change exceeds the mutation threshold; determining the range of nodes participating in the re-evaluation based on the spatial location of the edge where the environmental mutation occurs, using the spatial bounding box radius as the center; and locking route nodes outside the node range to a read-only state.

[0010] Furthermore, the guided route planning method also includes: adaptive interpolation fallback logic, which stops reading the corresponding memory channel of the target node when an interruption signal occurs in the real-time video frame sequence or environmental audio sound pressure level data corresponding to the target node in the static GIS network topology data; selects effective neighbor nodes from the first-order neighbor nodes that have physical connectivity with the target node based on the sensor status flag bit; reads the effective environmental parameters of the effective neighbor nodes in the multi-channel environmental state array; performs distance-inverse weighted averaging on the effective environmental parameters based on the physical distance between the effective neighbor nodes and the target node to generate alternative weight values; and uses the alternative weight values ​​as environmental parameters of the target node for optimization.

[0011] Furthermore, the adaptive interpolation fallback logic also includes: when none of the first-order neighbor nodes with physical connectivity to the target node are selected as valid neighbor nodes, searching for second-order neighbor nodes with indirect physical connectivity to the target node; when there are valid second-order neighbor nodes, generating the alternative weight value based on the valid environmental parameters of the second-order neighbor nodes; when there are no valid second-order neighbor nodes, using the historical environmental parameters of the target node that were last successfully updated before failure as the alternative weight value.

[0012] Furthermore, the guided tour route planning method also includes: resource anti-conflict control logic: when multiple start-point and end-point requests are received simultaneously, the optimal guided tour route corresponding to each start-point and end-point request is pre-mapped to a global spatiotemporal occupancy matrix; based on the global spatiotemporal occupancy matrix, it is detected whether different optimal guided tour routes overlap and conflict at the same node within the same time window; when the number of occupancy corresponding to the overlapping conflict exceeds the upper limit of the passage bandwidth, a time delay penalty term is added to the start-point and end-point requests with lower priority, and the start-point and end-point requests are re-entered into the improved multi-objective heuristic algorithm for route solving.

[0013] Furthermore, the additional time delay penalty includes: when the number of slots occupied by a node within the estimated time window does not exceed the upper limit of the bandwidth, no additional cost is imposed on the node; when the number of slots occupied by a node within the estimated time window exceeds the upper limit of the bandwidth, a time delay penalty is generated based on the number of slots occupied exceeding the upper limit of the bandwidth, which is positively correlated with the degree of over-occupancy; and the time delay penalty is added to the cost evaluation of candidate nodes corresponding to the lower priority start and end requests.

[0014] Further, the shadow occlusion coefficient is obtained through the following steps: converting the video frames in the real-time video frame sequence to the HSV color space; identifying the set of pixels that meet the preset shadow brightness threshold and hue range in the HSV color space; determining the area ratio of the pixel set to the preset region of interest corresponding to the road segment; and using the area ratio as the shadow occlusion coefficient of the corresponding road segment.

[0015] Further, the congestion penalty variable is obtained through the following steps: performing motion analysis on adjacent video frames in the real-time video frame sequence to obtain the foreground pixel displacement vector located within the preset region of interest; statistically analyzing the displacement amplitude of the foreground pixel displacement vector within a preset time sampling window to obtain the average density of the displacement vector; and proportionalizing the average density of the displacement vector with a preset physical carrying capacity parameter to obtain the congestion penalty variable for the corresponding road segment.

[0016] Furthermore, the logarithmic average, the current illuminance lumen value, and the temperature gradient value are obtained through the following steps: within a preset time sliding window, the ambient audio sound pressure level data is smoothed and statistically analyzed to obtain the logarithmic average; the current illuminance is read from the meteorological parameter data to obtain the current illuminance lumen value; and the local temperature status is read from the meteorological parameter data to obtain the temperature gradient value.

[0017] Furthermore, the dynamic update of the environmental map data structure includes: writing the shadow occlusion coefficient, the crowding penalty variable, the logarithmic average, the current illumination lumen value, and the temperature gradient value into independent memory channels in the multi-channel environmental state array; when the data refresh cycles of different sensor nodes are inconsistent, only the independent memory channel corresponding to the refreshed data is updated, while keeping the current values ​​in other independent memory channels unchanged.

[0018] Furthermore, the calculation of the comprehensive environmental cost value of the candidate node includes: assigning normalized weights to physical distance, congestion, acoustic environment, and meteorological conditions based on the user preference vector; generating a single-step environmental cost value based on the physical distance, the congestion penalty variable, the acoustic cost value mapped from the logarithmic mean, and the meteorological cost value jointly evaluated by the temperature gradient value and the shading coefficient; and using the single-step environmental cost value as the basis for calculating the comprehensive environmental cost value of the candidate node.

[0019] Furthermore, the meteorological cost value is obtained through the following steps: when the temperature gradient value is higher than a preset high temperature trigger threshold and the shadow occlusion coefficient is lower than a preset minimum safe shadow threshold, the meteorological cost value of the corresponding candidate node is set as the maximum cost value used to characterize non-priority passage; when the temperature gradient value is not higher than the high temperature trigger threshold, or the shadow occlusion coefficient is not lower than the minimum safe shadow threshold, a continuously changing meteorological cost value is generated based on the degree of deviation of the temperature gradient value relative to the preset comfort baseline temperature and the degree of shadow insufficiency.

[0020] Furthermore, the method of using an improved multi-objective heuristic algorithm to calculate the comprehensive environmental cost of candidate nodes includes: accumulating the single-step environmental cost of each path segment based on the explored path from the starting node to the current candidate node to obtain the known cumulative cost; obtaining the heuristic estimated cost based on the spatial distance from the current candidate node to the target node and the future congestion trend of the current candidate node; and prioritizing the candidate nodes based on the known cumulative cost and the heuristic estimated cost.

[0021] Furthermore, the heuristic cost estimation is obtained through the following steps: determining the minimum straight-line physical distance cost from the current candidate node to the target node based on the spatial coordinates of the current candidate node and the spatial coordinates of the target node; determining the estimated timestamp of arrival at the current candidate node based on the path length from the starting node to the current candidate node and the preset average walking speed; determining the congestion change rate of the current candidate node using the changes in the congestion penalty variable of the current candidate node in the most recent historical sampling periods; and determining the expected environmental penalty increment corresponding to the future congestion trend based on the current congestion, the congestion change rate, and the estimated timestamp of arrival at the current candidate node.

[0022] Furthermore, the determination of the congestion change rate includes: reading the congestion penalty variable of the current candidate node at multiple adjacent historical sampling times; determining the ratio between the congestion change magnitude and the time interval between adjacent historical sampling times; and averaging the multiple ratios to obtain the congestion change rate.

[0023] Furthermore, the generation of the experience comfort curve includes: sequentially reading the environmental parameters of the passed road segments in the multi-channel environmental state array along the node sequence of the optimal tour route; obtaining the cumulative environmental cost value sequence from the starting node to each intermediate node according to the calculation method of the comprehensive environmental cost value; performing a monotonically decreasing inverse mapping on the cumulative environmental cost value sequence to generate an experience comfort curve that changes with the passage timestamp.

[0024] Another aspect of the present invention provides a multimodal intelligent tour guide route planning system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements any of the multimodal intelligent tour guide route planning methods described above.

[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0026] 1. This invention constructs an environmental graph data structure containing a static topology connection table and a multi-channel environmental state array. It models the road network environment of the target geographical area as a multi-channel graph model that changes dynamically over time. It extracts shadow occlusion coefficients and congestion penalty variables from real-time video frame sequences, logarithmic averages from environmental audio sound pressure level data, and illuminance lumen values ​​and temperature gradient values ​​from meteorological parameter data. Each channel is calculated, stored, and does not interfere with each other. By normalizing the weight coefficients, the physical distance step size is weighted and summed with multiple environmental cost components to generate a comprehensive environmental cost value as a unified cost index for path search. This allows the planning results to simultaneously reflect the comprehensive impact of multiple environmental dimensions such as distance, shading, congestion, noise, and meteorology, overcoming the limitations of existing technologies that rely solely on a single physical distance index.

[0027] 2. The improved multi-objective heuristic algorithm of this invention introduces a congestion prediction component based on the congestion change rate through time linear extrapolation into the heuristic cost estimation. It obtains the first-order difference mean of the congestion penalty variable within the most recent historical sampling periods as the congestion change rate, and combines this with the estimated arrival timestamp to perform first-order linear extrapolation to predict the congestion at future times. This allows the algorithm to predict the congestion level of target nodes at the time of visitor arrival during path search. When the congestion level of a node shows an upward trend, the extrapolation result increases the heuristic estimation cost of that node, driving the algorithm to actively avoid soon-to-be-congested road segments. Conversely, when the congestion level shows a downward trend, the algorithm can identify and utilize road segments that will become less congested in the future, effectively alleviating the congestion of traditional A / B algorithms. The algorithm suffers from path quality degradation due to state lag in time-varying environments.

[0028] 3. This invention designs a local recalculation mechanism based on dynamic bounding boxes. When the rate of change of the comprehensive environmental cost on either side exceeds a preset mutation threshold, local recalculation is triggered. The radius of the spatial bounding box is adaptively determined based on the difference between the magnitude of the rate of change and the mutation threshold. This mechanism significantly reduces the computational complexity from the scale of the global graph to the scale of local active nodes. In typical scenic road networks, it can complete the replanning of affected routes with millisecond-level latency, meeting the real-time response requirements under environmental mutation scenarios. At the same time, the design that the bounding box radius is proportional to the mutation amplitude allows the recalculation range to adaptively match the actual impact range of the mutation event, achieving a reasonable balance between computational accuracy and computational efficiency.

[0029] 4. This invention designs an adaptive interpolation fallback mechanism based on graph topological spatial correlation. When the sensor data stream of the target node is interrupted, the system automatically traverses the nodes with normal sensor status among its first-order neighbor nodes and uses a distance-inverse weighted average algorithm to generate alternative weight values ​​using the effective environmental parameters of the neighbor nodes. This mechanism utilizes the physical correlation of environmental status between spatially adjacent road segments. When all first-order neighbor nodes fail, the system can further search for second-order neighbor nodes or use historical data as a downgrade alternative, forming a multi-layered fault-tolerant guarantee system and avoiding the impact of single-point sensor failure on the overall path planning results.

[0030] 5. The multi-channel environmental state array of this invention adopts a design of independent storage by channel. The data updates of each channel do not interfere with each other and support asynchronous updates. When it is necessary to add a new environmental perception dimension, only the corresponding calculation channel and storage channel need to be added. There is no need to modify the calculation logic and data structure of the existing channels. At the same time, the weighted linear combination calculation method of the comprehensive environmental cost value makes the contribution of each environmental factor to the final cost independently quantified and interpreted, which is convenient for system operation and maintenance personnel to analyze and optimize the planning results. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0032] Figure 1 This is a flowchart of the method provided in Embodiment 1 of the present invention.

[0033] Figure 2 The curve showing the dynamic change of the shadow occlusion coefficient over time is provided in Embodiment 1 of the present invention.

[0034] Figure 3 This is a comparison chart of the discriminative power of the crowding penalty variable provided in Embodiment 1 of the present invention.

[0035] Figure 4The conventional A provided in Embodiment 1 of the present invention A comparison of path quality between heuristic functions and spatiotemporal prediction heuristic functions.

[0036] Figure 5 This is a schematic diagram of the experience comfort curve provided in Embodiment 1 of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0038] Example 1

[0039] This embodiment discloses a multimodal intelligent navigation route planning method. Figure 1 The overall method flowchart of this embodiment is shown. As can be seen from the figure, this embodiment includes the following steps:

[0040] Step 1: Obtain static GIS network topology data of the target geographic area, as well as the first data stream collected in real time by multiple sensor nodes distributed within the target geographic area; the first data stream includes real-time video frame sequences, environmental audio sound pressure level data, and meteorological parameter data.

[0041] The target geographical area refers to the physical space range for which guided tour routes need to be planned. For example, the target geographical area can be a natural scenic area, theme park, historical and cultural block, large exhibition venue, or urban pedestrian landscape belt, etc., which has multiple accessible routes and tourists have route selection needs.

[0042] Static GIS network topology data refers to geographic information system data used to describe the spatial structure of road networks within a target geographic area. It expresses the geometry and connectivity of the scenic area's road network in the form of a graph.

[0043] In this embodiment, the static GIS network topology data contains at least two basic elements: a node set and an edge set. Each node in the node set represents a key intersection, path fork, or point of interest (such as a scenic spot entrance, rest stop, or restroom) within the scenic area, and each node has a unique identifier and its corresponding geographic coordinates (e.g., longitude and latitude, or planar projected coordinates). Each edge in the edge set represents an actual traversable path connecting two adjacent nodes, and each edge contains at least a starting node identifier, an ending node identifier, and the physical distance attribute of the path segment.

[0044] It is understandable that the "static" in static GIS network topology data means that the physical structure of the road network described by the data (i.e., which nodes have passable paths and the physical distance between road segments) will not change in a short time scale, which is different from the environmental perception data that changes dynamically over time as described later.

[0045] Sensor nodes refer to data acquisition terminal devices deployed at key locations within a target geographical area. Each sensor node integrates at least one or more of the following: video acquisition module, audio acquisition module, and meteorological acquisition module, to acquire environmental status information of the road section covered by the node in real time.

[0046] In this embodiment, the sensor nodes may include, but are not limited to: video surveillance equipment equipped with a high-definition camera, an ambient sound pressure level acquisition instrument, and a weather station integrating a temperature sensor, a humidity sensor, and a light sensor.

[0047] The first data stream refers to the multi-source heterogeneous environmental perception data stream that is collected in real time by multiple sensor nodes and continuously transmitted to the system's central processing unit.

[0048] The real-time video frame sequence refers to a sequence of video images continuously captured at a preset frame rate by camera devices deployed at key locations along the road. Each frame in the video frame sequence is a two-dimensional image containing road surface, pedestrians, and environmental elements within the camera's field of view, and its pixel values ​​are stored in RGB or other preset color space formats.

[0049] Understandably, real-time video frame sequences can be used for subsequent shadow occlusion analysis and pedestrian motion vector extraction, serving as a fundamental data source for the visual dimension in multi-channel environmental perception.

[0050] Ambient audio sound pressure level data refers to acoustic environmental measurements continuously acquired by sound pressure level acquisition devices deployed at key locations along a road segment at a preset sampling rate. This data is usually expressed in decibels (dB) and reflects the acoustic energy level of the surrounding environment at the time of acquisition.

[0051] Meteorological parameter data refers to atmospheric environmental physical quantities collected in real time by meteorological sensor equipment deployed at key locations along the road segment. In this embodiment, the meteorological parameter data includes at least the current illuminance lumen value and the temperature gradient value. The current illuminance lumen value reflects the natural light intensity of the road segment at the time of data collection; the temperature gradient value reflects the local air temperature level of the road segment at the time of data collection, and can be an absolute temperature value or a deviation relative to a comfort baseline temperature.

[0052] In this embodiment, meteorological parameter data may also include other meteorological physical quantities such as relative humidity, wind speed, and wind direction to further enrich the dimensions of environmental description.

[0053] Step 2: Perform multi-channel feature extraction logic on the first data stream, and dynamically update the preset environmental map data structure based on the extraction results to generate a multi-dimensional dynamic weight matrix.

[0054] Among them, the multi-channel feature extraction logic refers to a set of calculation processes that perform independent and deterministic signal processing and numerical mapping operations on the raw sensor data of different modalities in the first data stream, and transform the high-dimensional raw sensor readings into low-dimensional normalized scalar coefficients or structured numerical parameters.

[0055] It is understandable that the multi-channel approach in this feature extraction logic refers to the system allocating an independent computational and storage channel for each independent environmental perception dimension (such as shadow occlusion, crowd density, acoustic environment level, and weather conditions). The computation processes between each channel are independent of each other, and there is no cross-channel data dependency, thus ensuring the parallelism of the system.

[0056] The environmental map data structure refers to the core data structure pre-established by the system for storing and managing the environmental status information of all road segments within the target geographic area in the form of a graph topology. This data structure models the road network environment of the target geographic area as a dynamically changing graph model over time, enabling subsequent path planning algorithms to access multiple environmental factors simultaneously within a unified computational framework.

[0057] In this embodiment, the environmental map data structure includes a static topology connection table and a multi-channel environmental state array that is mapped to the static topology connection table via pointers.

[0058] The static topology connection table is a data table used to record the physical connectivity relationships between nodes in the scenic area's road network. The content of this connection table is directly derived from the static GIS network topology data obtained in step 100, storing all directed or undirected connections between node pairs and their corresponding physical distance attributes in the form of an adjacency list or adjacency matrix. It is understood that the static topology connection table remains unchanged during system operation, only describing the physical structure of the road network and not containing any dynamic environmental information.

[0059] The multi-channel environment state array refers to a multi-dimensional data storage space that maps each edge in the static topology connection table to a pointer. For each edge, the multi-channel environment state array allocates several independent memory channels, and each memory channel is used to store the current value of the road segment corresponding to that edge in a specific environmental perception dimension.

[0060] Understandably, by associating the static topology connection table with the multi-channel environment state array through pointer mapping, when the system needs to query the environment state of a certain road segment, it only needs to use the edge index in the static topology connection table to directly jump to the corresponding memory area in the multi-channel environment state array for reading, avoiding the overhead of traversal search and realizing constant-time random access to environmental data.

[0061] For example, in this embodiment, the scenic environment can be modeled as a time-parameterized dynamic tensor graph. Its definition can be shown in the following formula:

[0062]

[0063] in, The set of all nodes in the graph represents key intersections or points of interest in the scenic area; Let be the set of all edges in the graph, representing the actual paths connecting the nodes; This is a time-varying edge weight tensor.

[0064] It is understandable that in the above formula, the set of nodes... Sum of edges The contents of the static topology connection table remain unchanged during system operation; while the edge weight tensor... The contents of the multi-channel environmental state array are continuously updated as sensor data continues to flow in.

[0065] The multidimensional dynamic weight matrix is ​​a global numerical matrix composed of the current values ​​from the multi-channel environmental state arrays of all edges after the environmental graph data structure has been updated. Each row of this matrix corresponds to an edge in the graph, and each column corresponds to an independent environmental perception channel. Each element in the matrix is ​​the current value of a certain edge in a certain environmental channel. Subsequent path planning algorithms use this matrix as input for cost calculation, achieving a comprehensive consideration of multidimensional environmental factors.

[0066] For example, in this embodiment, for any connection node With nodes The path, its edge weight It is no longer a scalar, but a multi-channel state vector, as shown in the following equation:

[0067]

[0068] in, For nodes With nodes The physical distance between them is a static component, and its value comes from static GIS network topology data, which remains unchanged during system operation. For this section of road at the time The shadow occlusion coefficient reflects the degree to which sunlight is blocked by buildings or vegetation on the road surface, and belongs to the dynamic visual perception component. For this section of road at the time The congestion penalty variable reflects the degree to which the density of people and their movement hinder passage, and belongs to the dynamic visual perception component. For this section of road at the time The acoustic sound pressure level reflects the level of environmental noise and is a component of dynamic auditory perception. For this section of road at the time The temperature gradient value reflects local meteorological conditions and belongs to the dynamic meteorological sensing component.

[0069] Understandably, by organizing the aforementioned environmental parameters with different physical dimensions into a unified vector structure, and mapping each component of this vector to an independent memory channel in a multi-channel environmental state array for storage, the subsequent path planning algorithm can simultaneously consider multiple environmental factors within the same computational framework, rather than relying solely on a single physical distance metric as in traditional schemes. This multi-channel separate storage design also ensures that data updates in each environmental channel do not interfere with each other. When the sensor data refresh rate of one channel differs from that of other channels, the system can still correctly update only the value of the corresponding channel.

[0070] In this embodiment, the step of performing multi-channel feature extraction logic on the first data stream may further include the following sub-steps:

[0071] Sub-step 2.1: Extraction of shadow occlusion coefficient:

[0072] For real-time video frame sequences, a set of pixels that meet the preset shadow brightness threshold and hue range is extracted in the HSV color space. The proportion of the pixel set in the preset region of interest is calculated, and the shadow occlusion coefficient of the corresponding road segment is generated.

[0073] The HSV color space is a color model that represents the color of image pixels as three components: hue, saturation, and value. Compared to the RGB color space, the HSV color space can more intuitively separate the lightness and darkness information of a color from its tonal information, making it more suitable for pixel classification tasks based on lighting conditions.

[0074] The preset shadow brightness threshold refers to a pre-defined set of HSV component values ​​used to define shadow pixels. Specifically, this threshold defines a joint range in the HSV three-dimensional color space. A pixel is considered a shadow pixel only when the HSV color value of a pixel falls within the joint range.

[0075] It is understandable that shadow areas are visually characterized by lower brightness, lower saturation, and a cooler hue. By manually calibrating shadow samples under different lighting conditions and ground materials beforehand, a set of HSV threshold parameters that can effectively distinguish shadow pixels from non-shadow pixels can be determined.

[0076] A pre-defined Region of Interest (ROI) refers to a range of pixel coordinates in the camera's view that corresponds to a specific road segment's ground area. During the deployment phase, the system needs to manually or semi-automatically mark the ground area in the view corresponding to each road segment as the ROI for each camera's field of view, and then associate the coordinate range of this ROI with the corresponding edge in the static topology connection table.

[0077] The shadow occlusion coefficient is a normalized scalar coefficient obtained by integrating the area of ​​all pixels in the region of interest that meet the shadow determination criteria, and then dividing by the total area of ​​the region of interest. The value range of this coefficient is [0,1]. The closer its value is to 1, the higher the proportion of the road segment that is occluded by shadow, that is, more of the road segment is currently in a shaded state; the closer its value is to 0, the more likely the road segment is to be exposed to direct sunlight.

[0078] For example, in this embodiment, the road segment At any moment Shadow occlusion coefficient It can be calculated using the following formula:

[0079]

[0080] in, For the road segment in the camera footage Region of interest; This represents the area of ​​the region of interest, and its value remains fixed after being pre-calibrated during the system deployment phase. For at any time Screen coordinates The color value of the pixel at that location in the HSV color space is obtained by transforming the pixel data of the corresponding frame in the real-time video frame sequence into the color space. The pre-calibrated HSV multi-channel joint threshold range is used to define the color feature range of shadow pixels; This is an indicator function. It takes the value 1 when the condition in parentheses is true, that is, when the color value of the pixel falls within the shadow feature range, and otherwise takes the value 0. Figure 2The graph showing the dynamic change of the shadow occlusion coefficient over time in this embodiment is illustrated. Figure 2 In the diagram, the X-axis represents the time of day (6:00~18:00), and the Y-axis represents the shadow occlusion coefficient. The curves drawn for three road sections with different orientations / shading conditions demonstrate the typical diurnal variation of road sections with high shading in the morning and low shading at noon, as well as the stable characteristics of tree-lined road sections with high shading throughout the day.

[0081] The above formula essentially means that the area of ​​all pixels satisfying the shadow color condition is integrated on the two-dimensional pixel plane of the region of interest, and then the integration result is divided by the total area of ​​the region of interest to obtain a normalized coefficient reflecting the shadow coverage ratio. This pixel-level area integration method can directly compress high-dimensional visual image information into a low-dimensional scalar index, greatly reducing the memory usage and computational complexity of subsequent path cost calculations, and providing a foundation for fast table lookups.

[0082] Sub-step 2.2: Generation of the crowding penalty variable:

[0083] Calculate the foreground pixel displacement vector between adjacent video frames, divide the mean density of the displacement vector by the preset physical load-bearing capacity parameter, and generate a deterministic crowding penalty variable.

[0084] The foreground pixel displacement vector refers to the motion displacement vector calculated for each pixel region identified as a foreground moving target in the image after optical flow analysis or frame difference processing of adjacent frames in a real-time video frame sequence. This displacement vector describes the displacement direction and distance of the foreground moving target from the previous frame to the current frame in the two-dimensional coordinate system of the image.

[0085] Understandably, foreground moving targets typically correspond to pedestrians or moving objects on the road segment. By extracting the displacement vectors of these targets, rather than simply counting the number of pedestrians in the area, the system can effectively distinguish between two distinct traffic conditions: a large number of people moving smoothly and a large number of people stagnating and congested. When pedestrians are stationary or encounter opposing forces, the magnitude distribution of the displacement vectors exhibits different statistical characteristics compared to smooth flow, thus making the quantification of congestion more accurate.

[0086] The mean density of displacement vectors refers to the flux per unit time obtained by summing the displacement vector magnitudes of all foreground moving targets in the region of interest within a preset time sampling window and then dividing by the length of the time window. This flux value reflects the overall motion activity of foreground targets in the region of interest per unit time.

[0087] The preset physical carrying capacity parameter refers to the upper limit scalar value that is pre-set for each road segment, representing the amount of foreground moving targets that the road segment can accommodate per unit time. This parameter is usually calibrated based on the physical width of the road segment, road surface conditions, and safe passage standards.

[0088] The congestion penalty variable is a normalized value obtained by dividing the mean density of the displacement vector by the physical bearing capacity parameter. This value is a deterministic calculation result and does not depend on any stochastic process or machine learning model inference.

[0089] For example, in this embodiment, the road segment At any moment Crowding penalty variable It can be calculated using the following formula:

[0090]

[0091] in, This represents the total number of foreground moving targets detected in the current frame. For the first A prospective moving target at time The motion displacement vector is obtained through optical flow analysis of adjacent video frames; The magnitude of the displacement vector is used to reflect the amplitude of the target's motion. For the characteristic function, when the first... The moving target is located on the road section. The value is 1 when the corresponding region of interest is selected, and 0 otherwise. The sampling time window length is used to convert the cumulative amount into the flux per unit time. This is the upper limit parameter for physical bearing capacity, used to normalize the results to a comparable numerical range. Figure 3 This diagram shows a comparison of the discriminative power of the crowding penalty variable in this embodiment. Figure 3 In the graph, the X-axis represents the time series, and the Y-axis represents the congestion index value; it contains two sub-graphs. Figure 3 (a) A scene with many people but smooth flow; Figure 3 (b) A scenario with many people and stagnation; it shows that the simple headcount method gives an approximate congestion value (indistinguishable) in both scenarios, while the motion vector flux method can give a significantly different congestion value.

[0092] Understandably, the above formula, by introducing the time-cumulative flux of the motion vector magnitude rather than a simple headcount, can effectively distinguish between smooth flow and stagnant congestion. When pedestrians are actively moving and flowing smoothly on a road segment, although the vector magnitude is relatively large, its distribution is uniform, and the normalized congestion value is at a moderate level. However, when pedestrians are stagnant and gathering or encounter obstacles, although the individual vector magnitude may be small, the high density of targets causes the cumulative flux to increase sharply, resulting in a larger congestion penalty value, which more accurately reflects the actual traffic difficulty of the road segment.

[0093] Sub-step 2.3: Extraction of acoustic and meteorological parameters:

[0094] Extract the logarithmic average of the ambient audio sound pressure level data within a preset time sliding window, and extract the current illuminance lumen value and temperature gradient value from the meteorological parameter data.

[0095] The preset time sliding window refers to a continuous time interval with the current time as the end point and a preset time length as the window width. Within this window, the system performs statistical calculations on the sensor data to eliminate the impact of instantaneous noise fluctuations on environmental parameters and obtain more stable and reliable environmental description values.

[0096] Understandably, a sliding window approach is used instead of instantaneous sampling because environmental audio sound pressure level data typically exhibits significant instantaneous fluctuations (e.g., sudden bird calls, brief shouts from tourists). Directly using instantaneous values ​​as input for path costs would lead to drastic fluctuations in the cost values, negatively impacting the stability of path planning. By taking the logarithmic average within the sliding window, the influence of short-term impulse noise can be effectively suppressed while preserving the acoustic environmental trend information of the road segment over a longer timescale.

[0097] The logarithmic mean is the arithmetic mean of ambient audio sound pressure level data, expressed in decibels, within a preset time sliding window. Since the sound pressure level itself is already the result of taking the logarithm of the acoustic energy, the arithmetic mean of the sound pressure level sequence is equivalent to the logarithmic mean of the original acoustic energy within the time window.

[0098] The current illumination lumen value refers to the reading of natural light intensity collected by a light sensor at the current moment in meteorological parameter data, and its unit is usually lux.

[0099] The temperature gradient value refers to the local air temperature reading collected by a temperature sensor at the current moment in meteorological parameter data. In this embodiment, the temperature gradient value can be directly an absolute temperature value in degrees Celsius; in other embodiments, the temperature gradient value can also be the offset of the current temperature relative to a preset comfort baseline temperature.

[0100] Sub-step 2.4: Writing and updating the multi-channel environment state array:

[0101] The shadow occlusion coefficient, crowding penalty variable, logarithmic mean, illumination lumen value, and temperature gradient value are written into the independent memory channels of the multi-channel environment state array.

[0102] In this context, an independent memory channel refers to a pre-allocated storage space within the multi-channel environment state array that is physically or logically isolated from each other for each environment awareness dimension. Each memory channel stores only the current value of one environment parameter, and write operations between different channels do not interfere with each other.

[0103] Understandably, after sub-steps S2.1 to S2.3 complete the calculation of each channel parameter, the system writes the calculation results sequentially into the corresponding storage location in the multi-channel environment state array according to the predefined channel indices. Specifically, the shadow occlusion coefficient... Write to the first channel, congestion penalty variable Write the value to the second channel, the logarithmic average to the third channel, the illuminance lumen value to the fourth channel, and the temperature gradient value. Write to the fifth channel. Each write operation overwrites the old value of that edge in that channel, thus achieving dynamic updates to the environment graph data structure.

[0104] In this embodiment, since different sensors may have different sampling frequencies (e.g., the video frame rate is 25fps, while the meteorological parameter sampling frequency is once per minute), the update cycle of each channel can be asynchronous. The system uses the latest written value of each channel as the current value for subsequent cost calculations, and does not require all channels to be updated synchronously at the same time.

[0105] Step 3: Based on the multidimensional dynamic weight matrix, an improved multi-objective heuristic algorithm is used to process the preset start and end point requests, calculate the comprehensive environmental cost of each candidate node, generate and output the best tour route containing the node sequence and its corresponding access timestamp.

[0106] Among them, the improved multi-objective heuristic algorithm refers to the traditional A An improved optimization algorithm based on the path search algorithm. Traditional A The algorithm uses static spatial distance as the sole cost and heuristic basis, while the improved algorithm in this embodiment extends the cost calculation to a weighted comprehensive evaluation of multi-dimensional environmental factors, and introduces a time-dimensional extrapolation prediction mechanism into the heuristic function. This enables the algorithm to consider not only spatial distance when searching for the optimal path, but also multiple environmental dimensions such as congestion, noise, and weather conditions, and to have the ability to predict future environmental state changes.

[0107] A start-point and destination request refers to a route planning request submitted by a user through the system client. This request includes at least a start node identifier and a destination node identifier. In this embodiment, the start-point and destination request may also include a user preference vector, which specifies the user's level of attention to different environmental factors.

[0108] The comprehensive environmental cost value refers to the single scalar cost value obtained by calculating the corresponding road segment's multi-channel environmental parameters in the multi-dimensional dynamic weight matrix for a candidate node in the graph using a cost mapping formula. This cost value comprehensively reflects the physical distance cost and multi-dimensional environmental comfort loss required to move from the current node to the candidate node.

[0109] In this embodiment, the calculation process of the comprehensive environmental cost value includes two levels: the first level is to reduce the dimensionality of the multi-channel state vector to a single-step environmental cost value; the second level is to incorporate the single-step environmental cost value into the improved A Accumulation and heuristic estimation are performed in the cost evaluation function of the algorithm.

[0110] Calculation of single-step integrated environmental cost value:

[0111] In this embodiment, the first step in path planning is to reduce the multi-channel state vector to a single scalar value. This dimensionality reduction process needs to consider two key factors: first, the relative importance between different environmental channels, i.e., how much weight users or system administrators assign to factors such as distance, congestion, noise, and temperature; and second, the nonlinear coupling relationship between certain environmental factors, such as the negative impact on the tourist experience when high temperature and lack of shade occur simultaneously, which is far greater than the simple superposition of the individual effects of the two.

[0112] In this embodiment, the current node is calculated using a preset multidimensional cost mapping formula. Actual environmental costs :

[0113]

[0114] in, The physical distance step size is the distance from the current node to the candidate node. Physical distance components of the corresponding road segment ; The congestion penalty variable refers to the dynamic congestion penalty value of this road segment at the current moment. ; The acoustic cost based on the logarithmic mean mapping is the subjective discomfort value obtained by transforming the logarithmic mean of the sound pressure level of the road section through a noise mapping function. The meteorological cost is calculated based on the temperature gradient value and the shading coefficient, which is the meteorological environmental penalty value obtained after processing by the joint evaluation function of temperature and shading. to These are normalized weight coefficients dynamically assigned based on user preference vectors. Each component represents the user's level of concern regarding physical distance, congestion, noise, and weather conditions, and satisfies the following conditions: Normalization constraints.

[0115] Understandably, the core design idea of ​​the above cost mapping formula is to reduce the information from multi-dimensional environmental channels into a single scalar cost value through a weighted linear combination, thereby making subsequent A... The search algorithm can directly compare and select the best candidate node. Weighting coefficients. to The introduction of this feature enables the system to flexibly adjust the relative weights of various environmental factors in cost calculation based on the personalized preferences of different users.

[0116] For example, in this embodiment, the above formula can be equivalently expressed as the inner product of the user preference weight vector and the environmental cost component vector:

[0117]

[0118] in, For user preference weight vectors; This is a noise mapping function used to convert sound pressure level readings into subjective discomfort. This is a joint evaluation function for temperature and shading.

[0119] The specific calculation logic of meteorological value:

[0120] In this embodiment, meteorological value The computational logic adopts a piecewise function design to achieve rigid physical constraints under extreme weather conditions.

[0121] Specifically, when the temperature gradient value in the meteorological parameter data exceeds the preset high-temperature trigger threshold, the heat avoidance logic branch is activated. In this logic branch, the system jointly evaluates the shading coefficient and the temperature gradient value: if the shading coefficient of a candidate node is lower than the minimum safety threshold, it indicates that the road section is neither shaded nor heat-prone under high-temperature conditions, and is physically unsuitable for tourists. Therefore, the meteorological cost value of this node is assigned the maximum integer value MAX_INT, thereby performing hard physical pruning of unshaded road sections in the optimal route search, making A... The search algorithm directly truncates the reachability of the road segment at the mathematical level.

[0122] Under non-extreme conditions, i.e. when the temperature does not exceed the high temperature trigger threshold or the shadow occlusion is higher than the minimum safety threshold, the system uses the product of the temperature offset and the insufficient shadow amount to perform a smooth penalty calculation, reflecting the continuous penalty logic that the higher the temperature and the less occlusion, the greater the cost.

[0123] For example, in this embodiment, the joint evaluation function of temperature and shadow It can be represented as follows:

[0124]

[0125] in, This is the real-time temperature of the road segment, which is the current value read from the temperature gradient channel of the multi-channel environmental state array; This is the shadow occlusion coefficient for this road segment, which is the current value read from the shadow occlusion channel of the multi-channel environment state array; This is the high-temperature trigger threshold. When the temperature exceeds this threshold, it is considered a high-temperature state. This threshold can be pre-calibrated according to the climate characteristics of different regions. This is the minimum safe shadow threshold; when the shadow occlusion is below this threshold, it is considered to have no effective occlusion. This is the baseline value for temperature comfort, used to calculate the temperature offset; This is the thermal penalty scaling factor, used to adjust the numerical magnitude of the meteorological cost value under non-extreme conditions; It is the largest integer value that the system can represent.

[0126] It is understandable that in the above formula, when and At the same time, the value was directly set to a maximum number. Because A During the optimization process, the algorithm consistently selects the candidate node with the lowest cost value for expansion. Setting the cost value of a node to a maximum number is mathematically equivalent to removing that node from the search space, allowing the optimization algorithm to absolutely avoid unshaded road sections under intense sunlight. Under other conditions, the cost value is the temperature offset. Insufficient shadow The product of the products is scaled by the scaling factor. The adjusted results demonstrate a continuous penalty logic: higher temperatures and less shading incur greater costs. Specifically, when the temperature is below the comfort baseline, A value of 0 means no weather penalty is applied; when the shading level is 1 (i.e., complete shading), A value of 0 also means no weather penalty is imposed, which aligns with physical intuition.

[0127] Improved A Algorithm cost evaluation function

[0128] In this embodiment, the improved multi-objective heuristic algorithm employs a cost evaluation function. Perform node optimization.

[0129] Among them, the cost evaluation function It refers to the improved A The core numerical basis for the algorithm to prioritize each candidate node during the search process. A The basic working principle of the algorithm is to maintain a priority queue (usually called an open list). In each search iteration, the node with the smallest cost evaluation function value is taken from the priority queue and expanded. The cost evaluation function consists of the sum of two parts: the known cumulative cost. and heuristic cost estimation .

[0130] This refers to starting from the starting point and following the explored path to reach the current node. The cumulative real environmental costs Its value is the cumulative sum of the individual environmental costs at each step along the path.

[0131] Understandable The accumulation is in A The algorithm's search process is completed step by step, as the algorithm moves from one expanded node... Start by examining one of its neighboring nodes. hour, The value is Add from arrive Single-step environmental cost .

[0132] This refers to starting from the current node. Heuristic cost estimation to the destination. Traditional A The heuristic function of the algorithm usually only uses the Euclidean distance or Manhattan distance from the current node to the target node as an estimate. This estimate only contains static spatial information and does not consider the possible changes in the environmental state at future times.

[0133] In this embodiment, the heuristic function Designed as a multimodal prediction function, its computational logic includes not only spatial distance estimation but also an extrapolation prediction mechanism in the time dimension. Specifically, The computational logic consists of two components: the first component is the computation node. The minimum straight-line physical distance cost to the destination; the second component is the cost of utilizing the nearest... The expected environmental penalty increment is calculated by the linear extrapolation slope of the congestion penalty variable for each time period.

[0134] The motivation for introducing a time-dimensional extrapolation mechanism is understandable: in scenic areas, tourists need time to walk along paths, and during this time, the congestion level of certain sections may change significantly. If the algorithm's heuristic function lacks the ability to predict future states, it may guide tourists to a path that currently appears unobstructed but becomes congested upon arrival. By incorporating a linear extrapolation prediction of future congestion into the heuristic function, the algorithm gains the ability to predict future congestion trends during path search, thereby effectively mitigating the problems associated with traditional algorithmic approaches. The problem of path quality degradation caused by state lag in time-varying environments.

[0135] For example, in this embodiment, the heuristic function It can be calculated using the following formula:

[0136]

[0137] in, For the current node Spatial coordinates; The spatial coordinates of the target node (end point); This is the Euclidean distance from the current node to the target node, which represents the shortest spatial distance from the current node to the destination under ideal conditions (ignoring road network topology constraints); This represents the weighting coefficient for the spatial distance term; The weighting coefficients for the congestion prediction item; For nodes At the present moment The congestion value, which is read from the congestion channel of the multi-channel environment state array; To estimate the arrival node The timestamp is estimated by dividing the path length from the starting point to the node by the preset average walking speed; For nodes The rate of change of congestion is used to characterize the evolution trend of the congestion of the node over time.

[0138] It is understandable that the second term of the above heuristic function... The meaning is: using the current congestion level as a baseline, and adding the product of the congestion rate of change and the time difference, a first-order linear extrapolation estimate of the congestion level at the time tourists arrive at that node is made. When the rate of change... When the value is positive, it indicates that the congestion of the node is increasing, and the extrapolated result will be greater than the current value, thus increasing the estimated cost of the node in the heuristic function, making A... The algorithm tends to avoid the node; conversely, when the rate of change is negative, it indicates that the crowding is decreasing, and the extrapolated result will be less than the current value, making the node more attractive in the future.

[0139] In this embodiment, the rate of change in crowding It can be calculated using the following formula:

[0140]

[0141] in, This is the number of historical sampling periods used to calculate the rate of change; and For adjacent sampling times; and This represents the congestion level at the corresponding time.

[0142] Understandably, this rate of change is essentially the most recent The arithmetic mean of the first-order differences of congestion over several periods. By averaging the differences over multiple periods, rather than using only the difference from the most recent period, the estimation bias of the rate of change caused by instantaneous fluctuations can be effectively suppressed, making the extrapolation forecast more robust. Parameters The value of needs to be balanced between response sensitivity and robustness: The larger the value, the smoother and more robust the rate of change estimate, but the more delayed the response to sudden changes; The smaller the value, the more sensitive the response, but the more susceptible the estimated value is to noise interference. Figure 4 This embodiment illustrates conventional A. A comparison of path quality between heuristic functions and spatiotemporal prediction heuristic functions. Figure 4 The X-axis represents the node number on the path, and the Y-axis represents the actual congestion level upon reaching each node; it can be seen that traditional A The planned route encountered high congestion in the latter half (due to unforeseen circumstances), while the planned route in this scheme had stable and low congestion throughout (because the heuristic function predicted the increasing congestion trend and detoured in advance).

[0143] Step 4: Route output and generation of experience comfort curve.

[0144] In step 3, through the improved A After the algorithm completes the optimization, the system obtains an optimal guided route consisting of an ordered sequence of nodes and their corresponding access timestamps. In the output stage, the system needs to further process the original route data into visual information for display on the visitor's terminal.

[0145] Specifically, the node sequence in the optimal navigation route is parsed into a set of navigation instructions with directional features.

[0146] The navigation instruction set refers to a set of directional guidance text or graphic instructions for tourists generated by parsing the node sequence of the optimal tour route one by one in sequence. For each pair of adjacent nodes in the node sequence, the system calculates the direction angle based on the geographical coordinates of these two nodes, and calculates the turning angle by combining it with the direction angle of the previous path, thereby generating natural language navigation instructions or corresponding directional arrow icons, such as going straight for about 120 meters, turning left into the lotus pond path, and turning right to the viewing pavilion.

[0147] In this embodiment, the navigation instruction set may also include auxiliary information such as the estimated travel time for each segment of the route, the names of the attractions along the way, and the estimated time of arrival at each intermediate node.

[0148] Meanwhile, the system accumulates and sums the environmental parameters in the multi-channel environmental state array along the node sequence and maps them in reverse to generate an experience comfort curve that changes with the time axis.

[0149] The cumulative summation refers to reading the current values ​​of each environmental channel in the multi-channel environmental state array for each road segment along the node sequence of the optimal navigation route, calculating the single-step environmental cost value of each path according to the cost mapping formula defined in step 3, and gradually accumulating it along the route to obtain the cumulative environmental cost value sequence from the starting point to each intermediate node on the route.

[0150] Reverse mapping refers to the process of converting the aforementioned cumulative environmental cost sequence into an experience comfort score through a preset mapping function. This mapping function is typically a monotonically decreasing function, meaning that the higher the environmental cost, the lower the corresponding comfort score.

[0151] Understandably, the motivation behind the reverse mapping design is that environmental cost value is an internal optimization metric for algorithms; a higher value indicates worse accessibility. However, directly presenting this cost value to tourists lacks intuitiveness. By converting the cost value into a comfort rating (e.g., represented by a score from 0 to 100, with higher scores indicating a more comfortable experience), the reverse mapping presents information about the route's experience quality in a way that is easier for tourists to understand.

[0152] The comfort level curve is a continuous curve plotted with travel time or distance on the horizontal axis and comfort level rating on the vertical axis. This curve visually illustrates the trend of predicted environmental comfort at various times or locations as tourists travel along the optimal guided route.

[0153] In this embodiment, the navigation instruction set and the user experience comfort curve are output to the display terminal simultaneously.

[0154] Among them, the display terminal refers to the electronic device interface used by tourists to receive and view guide information, including but not limited to smartphone application interfaces, tablet application interfaces, and scenic area self-guided tour terminal screens.

[0155] Understandably, by presenting the navigation instruction set and the experience comfort curve simultaneously on the same interface, tourists can not only obtain directional guidance information on where to go, but also understand in advance the environmental expectations of what the experience will be like after they arrive, thus making more comprehensive and proactive route selection decisions. Figure 5 This embodiment shows a schematic diagram of the experience comfort curve. Figure 5 The comfort curves of three candidate routes are plotted: the shortest route (short distance but large fluctuations in comfort with troughs), the most comfortable route (high comfort throughout but longer distance), and the balanced route recommended by this solution (moderate distance and no obvious troughs in comfort); this demonstrates the intuitive value of the comfort curve as output information, and shows that this embodiment can achieve the optimal balance between distance and comfort.

[0156] Step 5: Local Recalculation Logic for Scenarios of Sudden Environmental Changes. Considering that environmental conditions may change abruptly in actual operation, such as a sudden increase in congestion or a sharp deterioration in traffic conditions on a road segment due to unforeseen events (e.g., temporary road closures, sudden crowd gatherings, heavy rain), if a global replanning of the entire graph is performed each time, the computational cost will be proportional to the size of all nodes in the graph, potentially leading to unacceptably high latency in large-scale scenic road networks. To address the high latency issue of global replanning, this embodiment designs a local recalculation mechanism based on dynamic bounding boxes.

[0157] Specifically, when the rate of change of the weight of any directed edge in the multi-channel environment state array within adjacent timestamps exceeds a preset mutation threshold, local recalculation is triggered.

[0158] The rate of change refers to the absolute value of the rate of change of the comprehensive environmental cost of a certain edge between two adjacent update times. After each update of the multi-channel environmental state array, the system automatically calculates the time rate of change of the cost of each edge and compares it with a preset mutation threshold.

[0159] The mutation threshold is a pre-defined reference value used to distinguish between normal environmental fluctuations and abnormal mutation events. This threshold can be calibrated based on the statistical fluctuation characteristics of environmental cost values ​​in historical operational data. When the rate of change of a certain edge exceeds this threshold, the system determines that an environmental mutation has occurred on that edge, requiring a local recalculation.

[0160] For example, in this embodiment, the mutation rate of edge weights Defined as the absolute value of the rate of change of environmental costs with respect to time:

[0161]

[0162] in, For the edge The comprehensive environmental value; For time. When Exceeding the preset mutation threshold When this occurs, a local recalculation mechanism is triggered.

[0163] In this embodiment, after triggering local recalculation, the system constructs a spatial bounding box based on the location of the edge where the weight changes abruptly. The radius of the spatial bounding box is determined by the magnitude of the rate of change.

[0164] The spatial bounding box refers to a spatial region defined by the spatial location of the edge where the abrupt change occurs and the calculated radius. Only graph nodes located inside the spatial bounding box are unlocked to participate in this recalculation, while all nodes located outside the bounding box retain their state determined in the previous calculation and do not participate in this recalculation.

[0165] For example, in this embodiment, the radius of the active bounding box and the set of nodes that need to be activated and recalculated It can be calculated using the following formula:

[0166]

[0167]

[0168] in, The bounding box expansion coefficient is used to control the mapping ratio of abrupt change magnitude to spatial range. This coefficient can be calibrated based on the average road segment length of the scenic area's road network. The mutation threshold; The edge where the mutation occurs; For nodes To the mutation edge Spatial distance.

[0169] Understandable This ensures that only mutations exceeding a threshold contribute to the bounding box radius, while normal fluctuations below the threshold do not trigger recalculation, thus avoiding unnecessary computational overhead. Furthermore, the design logic of the bounding box radius being proportional to the mutation amplitude is based on the fact that the more severe the mutation, the larger its impact range may be (for example, a large-scale gathering of people may affect multiple surrounding road sections), thus requiring nodes within a larger area to be included in the recalculation scope.

[0170] In this embodiment, the improved multi-objective heuristic algorithm is re-evaluated only for route nodes that pass through the spatial bounding box and have not yet been traversed, and route nodes outside the topology connection table are locked in a read-only state.

[0171] Understandably, by using the aforementioned local recalculation mechanism, the computational complexity is reduced from that of the global graph. Significantly reduced to the size of locally active nodes In a typical scenic area road network, the number of local nodes affected by a single mutation event is usually much smaller than the total number of nodes in the entire map. Therefore, this mechanism can significantly reduce computing power and response latency, ensuring that the system can complete the replanning of affected routes with a latency of milliseconds after an environmental mutation occurs.

[0172] Step 6: Adaptive Interpolation Fallback Logic for Sensor Failure. In actual scenic area deployment environments, sensor devices may experience data flow interruptions due to hardware failures, communication outages, power supply anomalies, etc. If the pathfinding algorithm lacks fault tolerance for sensor failures, the loss of sensor data at a node may prevent the corresponding channel values ​​in the node's multi-channel environmental state array from being updated. This would render the node and its associated paths incalculable and invalid, affecting the overall path planning results and potentially causing system anomalies. To provide the system with engineering robustness to continue operating even under partial sensor failure conditions, this embodiment designs an adaptive interpolation fallback mechanism based on graph topological spatial correlation.

[0173] Specifically, when an interruption signal occurs in the real-time video frame sequence or environmental audio sound pressure level data corresponding to a target node in the static GIS network topology data, the system stops reading the corresponding memory channel of that target node.

[0174] The interrupt signal refers to a sensor anomaly indication detected by the system through a preset data stream status monitoring mechanism. In this embodiment, the interrupt signal can be detected in one or more of the following ways: sensor heartbeat signal timeout (i.e., the sensor does not send any data packets within a preset time interval), the error rate of the received data packet checksum exceeds a preset threshold, or the data value shows an obviously physically impossible value (such as a temperature sensor reading below -273°C), etc.

[0175] The target node is the node where the sensor data stream is interrupted.

[0176] In this embodiment, the system maintains a sensor status flag bit for each node. This is used to indicate whether the sensor at that node is in normal working order. When, it indicates a node The sensor is functioning normally, and the data in its multi-channel environmental state array is reliable and usable; when When, it indicates a node The sensor has failed, and the system no longer reads data from the corresponding memory channel of that node.

[0177] In this embodiment, the system automatically traverses nodes that are physically connected to the target node. Extract the first-order neighbor nodes. The effective environmental parameters of each first-order neighbor node in the multi-channel environmental state array are used to generate interpolation parameters using a weighted average algorithm based on the inverse physical distance, which are then used as alternative weight values ​​for the target node in the optimization process.

[0178] In this context, first-order neighbor nodes refer to all adjacent nodes in the static topology connection table that are directly connected to the target node via an edge. These nodes are physically adjacent to the target node and typically have a strong correlation with their environmental state.

[0179] Valid environmental parameters refer to the sensor status flag bits in the first-order neighbor nodes being normal (i.e., The current value of the node in the multi-channel environment state array. Only data from neighboring nodes whose sensors are functioning normally are included in the interpolation calculation to avoid interpolating with invalid data from another failed node.

[0180] The weighted average algorithm based on inverse physical distance refers to a calculation method that uses the inverse power function value of the physical distance from each healthy neighbor node to the target failed node as weights to calculate the weighted average of the environmental parameters of these neighbor nodes. The rationale for this method is that road segments that are spatially closer usually have a stronger correlation in their environmental conditions; therefore, the closer the neighbor nodes are, the greater their contribution to the interpolation result should be.

[0181] For example, in this embodiment, when node When the sensor data stream is interrupted (record its status flag bit as ), ), its alternative weight It can be calculated using the following formula:

[0182]

[0183] in, For nodes The set of first-order physically connected neighbors on the graph topology; For neighboring nodes At any moment The multi-channel environment state vector is a vector composed of the current values ​​of the neighboring node in each environment channel. For nodes with neighboring nodes The physical distance between them, a value read directly from the static topology connection table; This is the distance decay exponent, used to control the rate at which the contribution of distance to the weight decays. For neighboring nodes The sensor status flag bit has a value of 1 indicating that the sensor of the neighboring node is working normally, and a value of 0 indicating that it is not working.

[0184] It is understandable that in the above formula, both the numerator and denominator are multiplied by... This ensures that only neighboring nodes with normal sensor status participate in the weighted calculation, thus avoiding the use of invalid data from another failed node for interpolation. Meanwhile, the distance decay index... The value is typically set to 2, indicating that the influence decreases with the square of the distance, which follows the inverse square law spatial decay model. This value is chosen based on physical intuition: in open spaces, the spatial correlation of many environmental physical quantities (such as temperature field and sound pressure field) usually decreases with the square of the distance.

[0185] In this embodiment, when all first-order neighbor nodes of the target node have failed (i.e., all neighbors' first-order neighbor nodes have failed), When the target node fails, the system can further search for healthy nodes among the second-order neighbor nodes (i.e., the neighbors of the neighbors) for interpolation, or use the historical data of the last successful update of the target node before failure as a degradation replacement.

[0186] Step 7: Resource conflict prevention control logic for multiple concurrent requests.

[0187] In scenarios where multiple users utilize the system simultaneously, if each user's path planning is performed independently, the system may simultaneously direct a large number of concurrent users to the same seemingly optimal path, causing rapid congestion on that path. This means that the system's own planning behavior itself becomes a new source of congestion. To avoid this problem at its source, this embodiment introduces a global spatiotemporal occupancy awareness mechanism in the algorithm backend.

[0188] Specifically, when multiple tour route planning requests are received simultaneously, the optimal tour route output by each request is pre-mapped into a global spatiotemporal occupancy matrix.

[0189] The global spatiotemporal occupancy matrix is ​​a globally shared data structure maintained by the system. It records the occupancy status of all active user requests that have completed path planning and are still in progress, and the occupancy of each node within each time window. The row dimension of this matrix corresponds to the set of nodes in the graph, and the column dimension corresponds to the discretized time window sequence. The value of each element in the matrix represents how many planned paths are scheduled to pass through the corresponding node within the corresponding time window.

[0190] For example, in this embodiment, the node At the estimated time Spatiotemporal occupancy It can be calculated using the following formula:

[0191]

[0192] in, This is the set of all active user requests that have completed route planning but have not yet reached their destination. For user requests The planned path, i.e. the sequence of nodes the user will pass through; For user requests Expected to pass through nodes A time window that is determined by the user's request. The timestamps of the corresponding nodes and the preset time tolerances are calculated from the route planning results; For the characteristic function, when the node Located in the request The value is 1 if the path is on the planned path and the estimated arrival time falls within its time window, and 0 otherwise.

[0193] It is understandable that the above formula means: for a certain node in the graph At some point in the future This counts how many of the currently active users plan to pass through the node around a given time. This statistical value represents the node's expected occupancy density at that moment.

[0194] In this embodiment, if multiple routes are detected to overlap and conflict at the same node at the same timestamp, and the number of overlaps exceeds the node's bandwidth limit, a time delay penalty is forcibly added to the route request with lower priority, and the route is made to re-enter step S3 to solve for the suboptimal route.

[0195] The upper limit of bandwidth refers to the physical capacity parameter that is pre-set for each node, representing the maximum number of users who can comfortably pass through the corresponding road segment within a unit time window. This parameter is usually calibrated based on the physical width of the road segment, the passage space, and safety standards.

[0196] Priority refers to the processing priority level assigned by the system to each route planning request. In this embodiment, priority can be determined based on rules such as request submission time (first come, first served), user level, or special needs identifiers (such as priority for the elderly, children, and disabled persons).

[0197] The time delay penalty is an additional cost added to the cost function for subsequent lower-priority requests when the bit position of a node exceeds its bandwidth limit. The effect of this penalty is to increase the value of that node in the cost evaluation of that request, thereby driving A... The search algorithm automatically selects an alternative path with a lighter load for the request.

[0198] For example, in this embodiment, a delay penalty term based on placeholders is used. It can be represented as follows:

[0199]

[0200] in, For nodes The physical bandwidth limit represents the maximum number of users that can comfortably pass through the corresponding road segment of this node per unit of time; The penalty scaling factor is used to control the strength of the mapping from excess occupancy to cost increments.

[0201] Understandably, when the number of digits is... Not exceeding the physical bandwidth limit When no penalty is applied, the cost of the node remains unchanged; however, when the number of occupants exceeds the upper limit, the penalty value is linearly proportional to the excess, with a heavier penalty for larger excesses. This penalty term is in A During the search process, the cost is added to the cost evaluation function value of the corresponding node, so that the node whose bandwidth is about to be saturated presents a higher passage cost for subsequent requests. This drives the optimization algorithm to automatically select the alternative path with a lighter load, achieving a balanced distribution of user traffic at the global level and solving the problem of clustering and blocking in high-concurrency scenarios from the source.

[0202] Example 2

[0203] This embodiment discloses a multimodal intelligent tour guide route planning system. Specifically, this system can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster consisting of multiple servers. When the electronic device is running, it can implement the multimodal intelligent tour guide route planning method described in Embodiment 1 of this application.

[0204] In some embodiments, the system can also be integrated into multiple electronic devices. For example, the multimodal intelligent tour guide route planning system can be integrated into multiple servers, and the multimodal intelligent tour guide route planning method of this application can be implemented by multiple servers. When the electronic device is running, it can implement the multimodal intelligent tour guide route planning method in embodiment 1 of this application.

[0205] In some embodiments, the server may also be implemented as a terminal.

[0206] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal intelligent tour guide route planning method, characterized in that, The guided tour route planning method includes: Acquire static GIS network topology data of the target geographic area, as well as a first data stream collected in real time by multiple sensor nodes distributed within the target geographic area. The first data stream includes real-time video frame sequences, environmental audio sound pressure level data, and meteorological parameter data. Multi-channel feature extraction logic is performed on the first data stream to obtain environmental parameters including at least the shadow occlusion coefficient, crowding penalty variable, logarithmic mean, current illumination lumen value, and temperature gradient value. The environmental map data structure is dynamically updated based on the environmental parameters to generate a multi-dimensional dynamic weight matrix. Upon receiving the start and end point requests, and based on the multi-dimensional dynamic weight matrix, an improved multi-objective heuristic algorithm is used to calculate the comprehensive environmental cost of candidate nodes. The comprehensive environmental cost is determined by physical distance, congestion, acoustic environment, and meteorological conditions. Based on the comprehensive environmental cost, candidate nodes are optimized to generate the best tour route, which includes the node sequence and its corresponding access timestamp. A navigation instruction set is generated based on the optimal guide route, and The environmental parameters are processed along the node sequence to generate an experience comfort curve, and the navigation instruction set and the experience comfort curve are output to the display terminal.

2. The multimodal intelligent tour route planning method according to claim 1, characterized in that, The shadow occlusion coefficient is obtained through the following steps: Convert the video frames in the real-time video frame sequence to the HSV color space; Identify a set of pixels in the HSV color space that satisfy a preset shadow brightness threshold and hue range; Within a preset region of interest corresponding to a road segment, determine the area ratio of the pixel set to the preset region of interest; The area ratio is used as the shadow occlusion coefficient of the corresponding road segment; The crowding penalty variable is obtained through the following steps: Motion analysis is performed on adjacent video frames in the real-time video frame sequence to obtain the foreground pixel displacement vector located within the preset region of interest; Within a preset time sampling window, the displacement amplitude of the foreground pixel displacement vector is statistically analyzed to obtain the average density of the displacement vector. The average density of the displacement vector is proportionalized to a preset physical bearing capacity parameter to obtain the congestion penalty variable for the corresponding road segment. The logarithmic average, the current illumination lumen value, and the temperature gradient value are obtained through the following steps: Within a preset time sliding window, the environmental audio sound pressure level data is smoothed and statistically analyzed to obtain the logarithmic average value; The current light intensity is read from the meteorological parameter data to obtain the current light lumen value; The local temperature status is read from the meteorological parameter data to obtain the temperature gradient value.

3. The multimodal intelligent tour route planning method according to claim 1, characterized in that, The dynamic updating of the environmental map data structure includes: The shadow occlusion coefficient, the crowding penalty variable, the logarithmic mean, the current illumination lumen value, and the temperature gradient value are respectively written into independent memory channels in the multi-channel environment state array; When the data refresh cycles of different sensor nodes are inconsistent, only the independent memory channel corresponding to the refreshed data is updated, while the current values ​​in other independent memory channels remain unchanged.

4. The multimodal intelligent tour guide route planning method according to claim 1, characterized in that, The calculation of the comprehensive environmental cost of candidate nodes includes: Based on user preference vectors, normalized weights are assigned to physical distance, congestion, acoustic environment, and meteorological conditions, respectively. A single-step environmental cost is generated based on the physical distance, the crowding penalty variable, the acoustic cost value mapped from the logarithmic mean, and the meteorological cost value jointly evaluated by the temperature gradient value and the shading coefficient. The single-step environmental cost value is used as the basis for calculating the comprehensive environmental cost value of candidate nodes.

5. The multimodal intelligent tour guide route planning method according to claim 4, characterized in that, The meteorological value is obtained through the following steps: When the temperature gradient value is higher than the preset high temperature trigger threshold, and When the shadow occlusion coefficient is lower than the preset minimum safe shadow threshold, the meteorological value of the corresponding candidate node is set as the value used to characterize the non-priority passage. When the temperature gradient value is not higher than the high temperature trigger threshold, or When the shadow occlusion coefficient is not lower than the minimum safe shadow threshold, a continuously changing meteorological cost is generated based on the degree of deviation of the temperature gradient value from the preset comfort baseline temperature and the degree of shadow insufficiency.

6. The multimodal intelligent tour guide route planning method according to claim 1, characterized in that, The method of calculating the comprehensive environmental cost of candidate nodes using an improved multi-objective heuristic algorithm includes: Based on the explored paths from the starting node to the current candidate node, the single-step environmental cost of each segment is accumulated to obtain the known cumulative cost; Based on the spatial distance between the current candidate node and the target node, and the future congestion trend of the current candidate node, a heuristic cost estimate is obtained; Based on the known cumulative cost and the heuristic estimated cost, the candidate nodes are prioritized.

7. The multimodal intelligent tour guide route planning method according to claim 6, characterized in that, The heuristic cost estimation is obtained through the following steps: Based on the spatial coordinates of the current candidate node and the spatial coordinates of the target node, determine the minimum straight-line physical distance cost from the current candidate node to the target node; Based on the path length from the starting node to the current candidate node and the preset average walking speed, determine the estimated timestamp of arrival at the current candidate node; The congestion rate of the current candidate node is determined by using the changes in the congestion penalty variable of the current candidate node in the most recent historical sampling periods. Based on the current congestion level, the rate of change of congestion level, and the estimated timestamp of arrival at the current candidate node, the expected environmental penalty increment corresponding to the future congestion trend is determined.

8. The multimodal intelligent tour guide route planning method according to claim 1, characterized in that, The determination of the congestion change rate includes: reading the congestion penalty variable of the current candidate node at multiple adjacent historical sampling times; determining the ratio between the congestion change magnitude and the time interval between adjacent historical sampling times; and averaging the multiple ratios to obtain the congestion change rate.

9. The multimodal intelligent tour guide route planning method according to claim 1, characterized in that, The generation of the experience comfort curve includes: Along the node sequence of the optimal navigation route, the environmental parameters of the passed road segments in the multi-channel environmental state array are read sequentially. Based on the calculation method of the comprehensive environmental cost value, the cumulative environmental cost value sequence from the starting node to each intermediate node is obtained; A monotonically decreasing inverse mapping is performed on the cumulative environmental cost sequence to generate an experience comfort curve that varies with the passage timestamp.

10. A multimodal intelligent tour guide route planning system, characterized in that, The guided tour route planning system includes: processor; The memory stores a computer program that, when executed by a processor, implements the multimodal intelligent navigation route planning method as described in any one of claims 1 to 9.

Images