Map rendering method and device

Abstract

The invention provides a map rendering method and device, and relates to the technical field of artificial intelligence, in particular to the technical field of intelligent traffic, intelligent navigation and deep learning. A specific embodiment of the method comprises the following steps: constructing a multi-dimensional sensing matrix based on vehicle information, navigation information and environment information; generating a time sequence matrix based on the multi-dimensional sensing matrix of a plurality of continuous time steps; inputting the time sequence matrix into a long short-term memory (LSTM) network model to obtain a probability distribution diagram of the region of interest of the display screen in future preset time; and rendering the region of interest based on the probability distribution diagram and the current driving scene.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of smart transportation, intelligent navigation and deep learning technology. Background Art

[0002] With the development of intelligent driving technology, electronic maps have become a core vehicle for human-machine interaction. However, due to the complexity of road networks, traditional map rendering methods often lead to information overload, making it difficult for users to quickly identify key road features. For example, in highway scenarios, excessive secondary information such as side roads and shops can hinder the identification of main roads. In urban areas, complex structures such as elevated roads and ramps make it difficult for users to accurately determine the travel path.

[0003] Currently, in order to simplify maps, fixed rules (such as speed thresholds triggering map simplification) or static layering strategies (such as fixed adjustment of display according to road levels) are usually used to render maps. Summary of the Invention

[0004] The embodiments of the present disclosure provide a map rendering method, apparatus, device, storage medium, and program product.

[0005] In a first aspect, an embodiment of the present disclosure proposes a map rendering method, including: constructing a multidimensional perception matrix based on vehicle information, navigation information, and environmental information; generating a time series matrix based on the multidimensional perception matrix of multiple consecutive time steps; inputting the time series matrix into a long short-term memory network (LSTM) model to obtain a probability distribution map of the focus area of ​​the display screen within a preset future time; and rendering the focus area based on the probability distribution map and the current driving scene.

[0006] In a second aspect, an embodiment of the present disclosure proposes a map rendering device, including: a construction module, configured to construct a multidimensional perception matrix based on vehicle information, navigation information and environmental information; a generation module, configured to generate a time series matrix based on the multidimensional perception matrix of multiple consecutive time steps; a prediction module, configured to input the time series matrix into a long short-term memory network (LSTM) model to obtain a probability distribution map of the focus area of ​​the display screen within a preset time in the future; and a rendering module, configured to render the focus area based on the probability distribution map and the current driving scene.

[0007] In a third aspect, an embodiment of the present disclosure proposes an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute the method described in the first aspect.

[0009] In a fifth aspect, an embodiment of the present disclosure proposes a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0010] The key or important features of the embodiments of the present disclosure are not intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects, and advantages of the present disclosure will become more apparent upon reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are provided for a better understanding of the present disclosure and do not constitute a limitation of the present disclosure. Among them: Figure 1 is a flowchart of an embodiment of a map rendering method according to the present disclosure; Figure 2 is a flowchart of another embodiment of a map rendering method according to the present disclosure; Figure 3 is a structural diagram of an embodiment of a map rendering device according to the present disclosure; Figure 4 It is a block diagram of an electronic device used to implement the map rendering method of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0013] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0014] Figure 1 A process 100 of an embodiment of a map rendering method according to the present disclosure is shown. The map rendering method includes the following steps: Step 101: construct a multi-dimensional perception matrix based on vehicle information, navigation information and environmental information.

[0015] In this embodiment, the execution subject of the map rendering method can construct a multi-dimensional perception matrix based on vehicle information, navigation information and environmental information.

[0016] The map rendering method is typically executed by a server. The server can be either hardware or software. If the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers or as a single server. If the server is software, it can be implemented as multiple software programs or software modules (for example, to provide distributed services) or as a single software program or software module. This is not specifically limited here.

[0017] The multi-dimensional perception matrix can include information of multiple dimensions such as vehicle information, navigation information and environmental information.

[0018] In some embodiments, the vehicle information may include the vehicle status and the remaining range L. The vehicle status may include the vehicle speed v, the position (x, y) and the lane type. Lane Type This may include but is not limited to main roads ( )、Auxiliary Road( ) and ramps ( ) etc. Navigation information includes navigation stage . Navigation phase This may include but is not limited to high speed ( ), urban area ( ) and congestion ( ), etc. Environmental information can include lighting environment and weather conditions . Lighting environment Can include daytime ( ) and night ( ). Weather conditions Can include sunny days ( ),rain( ) and foggy days ( ) etc. At this time, the multi-dimensional perception matrix M can be [v, x, y, , , L, , ].

[0019] For example, a car is traveling at a speed of 60 km / h (v=60), with local coordinates (x=120.5, y=34.8), and is traveling on the main road ( ), the navigation phase is high speed ( ), the remaining mileage is 30km (L=30), and it is daytime ( ) and sunny ( ), then the multidimensional perception matrix can be: M=[60, 120.5, 34.8, 1, 1, 30, 1, 1].

[0020] Step 102: Generate a time series matrix based on the multi-dimensional perception matrix of multiple consecutive time steps.

[0021] In this embodiment, the execution entity may generate a time series matrix based on a multi-dimensional perception matrix of a plurality of consecutive time steps.

[0022] For example, by selecting the multidimensional perception matrix of 10 consecutive time steps before time t, the time series matrix can be constructed =[ , ,…, ].in, is the multidimensional perception matrix at time t, It is the multi-dimensional perception matrix at time t-8, It is the multi-dimensional perception matrix at time t-9.

[0023] Step 103 : Input the time series matrix into a long short-term memory network (LSTM) model to obtain a probability distribution map of the focus area of ​​the display screen within a preset time in the future.

[0024] In this embodiment, the execution entity may input the time series matrix into a long short-term memory network (LSTM) model to obtain a probability distribution map of the focus area of ​​the display screen within a preset time in the future.

[0025] The LSTM (Long Short-term Memory Networks) model can be used to predict a driver's visual focus in the future, such as the area where the driver may focus on the navigation interface on the display screen in the next 5 seconds.

[0026] An LSTM model can be composed of LSTM layers. To enable the LSTM model to learn more complex time series patterns, it can employ a structure with at least two LSTM layers. By performing gated calculations on each LSTM layer within the LSTM model, a probability distribution map of the display's focus area within a preset future timeframe can be obtained. The time series matrix can be the input to the first LSTM layer. The output of the previous LSTM layer can serve as the input to the next LSTM layer. The output of the final LSTM layer can be a probability distribution map of the display's focus area within a preset future timeframe. The gated calculations for each LSTM layer can include, but are not limited to, input gates, forget gates, candidate memory cells, target memory cells, and output gates.

[0027] Step 104 : Rendering the focus area based on the probability distribution map and the current driving scene.

[0028] In this embodiment, the execution entity can adaptively render the focus area of ​​the display screen based on the probability distribution map and the current driving scene. The probability in the probability distribution map can be used to represent the possibility that the corresponding position of the display screen is the focus area.

[0029] Driving scenarios may include but are not limited to tunnel scenarios, overpass layered scenarios, highway ramp scenarios, complex urban intersection scenarios, bad weather scenarios, service area scenarios, construction section detour scenarios, narrow car avoidance scenarios on rural roads, and night roads without street lights. Different driving scenarios can correspond to different rendering styles, thereby achieving targeted rendering of various driving scenarios. For all driving scenarios, these rendering styles meet the following rendering rules: enhance the rendering of the focus areas with large probability values ​​in the probability distribution map, and weaken the rendering of the focus areas with small probabilities in the probability distribution map. For example, highlight the key focus areas, weaken other non-key focus areas, hide non-key labels, enhance the display of traffic lights, increase the brightness of lane lines, and thicken the main road outline.

[0030] The disclosed embodiment provides a map rendering method that constructs a multidimensional perception matrix based on vehicle information, navigation information, and environmental information, and combines it with an LSTM model to predict the driver's possible areas of concern in the future time period. The rendering optimization of the navigation interface is then dynamically adjusted accordingly to improve the readability of the map and navigation efficiency.

[0031] Continue to refer Figure 2 , which shows a process 200 of another embodiment of a map rendering method according to the present disclosure. The map rendering method includes the following steps: Step 201: construct a multi-dimensional perception matrix based on vehicle information, navigation information and environmental information.

[0032] Step 202: Generate a time series matrix based on the multi-dimensional perception matrix of multiple consecutive time steps.

[0033] In this embodiment, the specific operations of steps 201-202 are already described in Figure 1 In the illustrated embodiment, steps 101-102 are described in detail and will not be repeated here.

[0034] Step 203: Perform gate calculation on each LSTM layer in the LSTM model to obtain a probability distribution graph.

[0035] In this embodiment, the execution body of the map rendering method can perform gated calculations on each LSTM layer in the LSTM model to obtain a probability distribution map.

[0036] The LSTM model can be used to predict the driver's visual focus in the future, such as the driver's possible focus area on the navigation interface on the display screen in the next 5 seconds.

[0037] An LSTM model can be composed of LSTM layers. To enable the LSTM model to learn more complex time series patterns, it can employ a structure of at least two LSTM layers. By performing gated calculations on each LSTM layer within the LSTM model, a probability distribution map of the display's focus area within a preset future timeframe can be obtained. The time series matrix can be the input to the first LSTM layer. The output of the previous LSTM layer can serve as the input to the next LSTM layer. The output of the final LSTM layer can be a probability distribution map of the display's focus area within a preset future timeframe.

[0038] The gated calculations of each LSTM layer may include but are not limited to: input gate, forget gate, candidate memory unit, target memory unit and output gate.

[0039] For the input gate, the weight matrix corresponding to the input gate can be used to perform a weighted summation of the LSTM layer's input and the previous hidden state, and then add the bias vector corresponding to the input gate to obtain the initial output value of the input gate. The activation function is then used to compress the initial output value of the input gate to a first preset numerical range to obtain the target output value of the input gate. The target output value of the input gate can be used to represent the amount of information entering the candidate memory unit.

[0040] Input Gate The calculation formula can be: σ ( WiXt + Uiht 1+ bi ).in, is the input of the LSTM layer, ht 1 is the previous hidden state, and its hidden layer dimension is h (such as h=128), yes The weight matrix of is of dimension (input_size, h). yes The weight matrix of is of shape (h, h). is the bias vector with dimension (h,). σ is the sigmoid activation function, which compresses the output value to the interval [0, 1] and is used to control how much new information can enter the candidate memory unit. .

[0041] For the forget gate, the weight matrix corresponding to the forget gate can be used to perform weighted summation on the input and current hidden state of the LSTM layer, and then the bias vector corresponding to the forget gate can be added to obtain the initial output value of the forget gate. The activation function is used to compress the initial output value of the forget gate to a first preset numerical interval to obtain the target output value of the forget gate, where the target output value of the forget gate is used to represent the amount of forgotten information of the previous target memory unit.

[0042] Forget Gate The calculation formula can be: = σ ( Wxya + Ufht 1+ bf ).in, Wf yes The weight matrix of is of dimension (input_size, h). Uf yes ht The weight matrix of 1 has the shape (h, h). bf is the bias vector with dimension (h,). σ is the sigmoid activation function, which compresses the output value to the interval [0, 1] and is used to determine the previous target memory unit. How much information needs to be forgotten.

[0043] For the candidate memory unit, the weight matrix corresponding to the candidate memory unit is used to perform weighted summation on the input and the previous hidden state of the LSTM layer, and then the bias vector corresponding to the candidate memory unit is added to obtain the initial output value of the candidate memory unit; the hyperbolic tangent function is used to compress the initial output value of the candidate memory unit to a second preset numerical interval to obtain the target output value of the candidate memory unit, wherein the target output value of the candidate memory unit is used to represent the candidate memory content.

[0044] Candidate memory units The calculation formula can be: =tanh( Wlq + Ucht 1+ bc ).in, Wc yes The weight matrix of is of dimension (input_size, h). Uc yes ht The weight matrix of 1 has the shape (h, h). bc is a bias vector with dimension (h,). tanh is a hyperbolic tangent function that compresses the output value to the interval [-1, 1] to generate candidate memory content.

[0045] For the target memory unit, calculate the product of the target output value of the forget gate and the previous target memory unit, and the sum of the product of the target output value of the output gate and the target output value of the candidate memory unit, and update the current target memory unit.

[0046] The update formula of the current target memory unit can be: = + .in, is the target memory unit, the target output through the forget gate and the target output of the input gate , update the current target memory unit , retaining important information and adding new information.

[0047] For the output gate, the weight matrix corresponding to the output gate is used to perform weighted summation on the input and the previous hidden state of the LSTM layer, and then the bias vector corresponding to the output gate is added to obtain the initial output value of the output gate; the activation function is used to compress the initial output value of the output gate to a first preset numerical interval to obtain the target output value of the output gate, wherein the target output value of the output gate is used to represent the amount of information output from the current target memory unit to the current hidden state.

[0048] Output Gate The calculation formula can be: = σ ( WoXt + Uoht 1+ bo ).in, Wo yes The weight matrix of is of dimension (input_size, h). Uo yes ht The weight matrix of 1 has the shape (h, h). bo is the bias vector with dimension (h,). σ is the sigmoid activation function, which compresses the output value to the interval [0, 1] and is used to determine the current target memory unit. How much information should be output to the current hidden state ht .

[0049] For the final output, the product of the target output value of the output gate and the hyperbolic tangent value of the current target memory unit can be calculated to obtain the current hidden state.

[0050] Current hidden state The calculation formula can be: = ot tanh( Ct ).

[0051] The hidden state of the last time step of the last LSTM layer can be input into the activation function to obtain the probability distribution map of the attention area.

[0052] The calculation formula of probability distribution graph α can be: α=sigmoid( Wpht + bp ).in, Wp yes ht The weight matrix of , with dimension (h, 1), bp is a bias vector with a dimension of (1,). α is a probability value with a dimension of (batch_size, 1), which indicates the probability of each sample corresponding to the region of interest in the future.

[0053] For example, the navigation interface of the display screen is divided into a 3×3 grid (a total of 9 areas). If these 9 areas are predicted by the LSTM model, the output will be the [0,1] interval for each area, such as: α = ; That is to say, in the next 5-10 seconds, the driver is most likely to pay attention to the areas of α=0.9, α=0.6, and α=0.4. The higher the value, the higher the possibility of attention.

[0054] Step 204: Determine the position of the area of ​​interest based on the resolution of the display screen.

[0055] In this embodiment, the execution entity may determine the position of the area of ​​interest based on the resolution of the display screen.

[0056] Based on the resolution of the display, the focus area can be converted into the actual position at the display resolution, thereby adapting to the display screens of various devices.

[0057] For example, the navigation interface of the display is divided into a 3×3 grid, and the LSTM model is used to predict: α = .

[0058] If the display resolution is 1920 1280, then for α= The position of this grid at the display resolution is: start =640, x end =1280; y start =360, y end =720. Similarly, the positions of other grids can also be divided by resolution, so as to achieve different grids. αMapping to display resolution.

[0059] Step 205 : Rendering the location of the focus area based on the probability distribution map and the current driving scene.

[0060] In this embodiment, the execution entity may render the location of the focus area based on the probability distribution map and the current driving scene.

[0061] Driving scenarios may include but are not limited to tunnel scenarios, overpass layered scenarios, highway ramp scenarios, complex urban intersection scenarios, bad weather scenarios, service area scenarios, construction section detour scenarios, narrow car avoidance scenarios on rural roads, and night roads without street lights. Different driving scenarios can correspond to different rendering styles, thereby achieving targeted rendering of various driving scenarios. For all driving scenarios, these rendering styles meet the following rendering rules: enhance the rendering of the focus areas with large probability values ​​in the probability distribution map, and weaken the rendering of the focus areas with small probabilities in the probability distribution map. For example, highlight the key focus areas, weaken other non-key focus areas, hide non-key labels, enhance the display of traffic lights, increase the brightness of lane lines, and thicken the main road outline.

[0062] For tunnel scenes, at least one of the following rendering operations can be performed: brightness compensation for locations with a probability greater than a first preset threshold (e.g., α > 0.8), transparency reduction for locations with a probability less than a second preset threshold (e.g., α < 0.5), rendering lanes in a first enhanced color, rendering lane markings in a second enhanced color, reducing the map scale, etc. For example, in a tunnel scene (α > 0.8) that meets the trigger conditions of navigation prediction of tunnel entry within 5 seconds and a sudden increase in α in the central lane area, dynamic brightness compensation can be performed in the α > 0.8 area; the α < 0.5 area can be gradually dimmed to 50% transparency 3 seconds in advance; the lane can be rendered as a fluorescent green band (RGB: 0, 255, 50); the lane markings on both sides can be upgraded to a 3px amber halo; a two-channel stereo prompt ("Tunnel ahead, stay in the middle lane") can be triggered; and the map scale can be simultaneously reduced to 50 meters.

[0063] For overpass layered scenes, at least one of the following rendering operations can be performed: rendering non-current layer roads as the first weakened color, projecting arrows extending along the correct path for a preset length, etc. For example, in an overpass layered scene (α>0.7) that meets the trigger conditions of a three-dimensional intersection (such as an elevated / ground level switch) on the navigation path and α remaining high in the layered decision area, a semi-transparent blue mask (30% transparency) can be used to cover the non-current layer roads; and a dynamic arrow (at a frequency of 2Hz) can be projected to extend a virtual 10-meter extension along the correct path.

[0064] For highway ramp entrances, at least one of the following rendering options can be performed: compressing non-target lanes, expanding target lanes, rendering target lanes in a third, enhanced color, etc. For example, for a highway ramp entrance (α > 0.9) that meets the trigger conditions of reaching the divergence point within 8 seconds and α forming a gradient distribution across the two right lanes, the non-target lane width can be compressed to 60% of its original value; the correct exit lane can be expanded by 120%, and a gold particle flow effect (particle density of 200 / ㎡) can be overlaid.

[0065] For complex urban intersections, at least one of the following rendering methods can be performed: using Gaussian blur to de-emphasize non-critical elements, rendering the target lane in a fourth enhanced color, etc. For example, in a complex urban intersection (α mutation monitoring) where the α value fluctuates wildly between 0.2 and 0.8 (with a standard deviation of σ > 0.15), indicating a trigger condition of visual focus confusion, Gaussian blur (radius of 15px) can be used to de-emphasize non-critical elements, and a radial gradient highlight can be applied to the target lane.

[0066] For inclement weather scenarios, at least one of the following rendering options can be performed: rendering lane markings in the fifth enhanced color, increasing lane line width, etc. For example, if the inclement weather scenario meets the trigger conditions of the rain sensor or camera detecting heavy rain or fog, and the overall alpha average decreases by more than 40%, the lane markings can be changed to a strobing dual-color (alternating blue and white at a frequency of 4Hz) with a line width increased to 8px. The navigation prompt sound source point can also be virtually positioned in the target direction, using HRTF (Head Related Transfer Function) to enhance direction perception.

[0067] For service area scenarios, at least one of the following rendering options can be performed: displaying the number of idle charging piles and / or gas stations within the service area with labels, using different colors to mark the cost-effectiveness of charging piles and / or gas stations, etc. For example, if the service area scenario meets the trigger conditions of the navigation prediction of approaching the service area entrance within 10 seconds and α linearly increasing in the right lane, the number of idle charging piles and / or gas stations in the service area can be displayed as floating labels (with the font bolded by 150%); and the cost-effectiveness can be marked with color blocks based on fluctuations in electricity and / or fuel prices (green for low prices, red for high prices).

[0068] For construction detour scenarios, at least one of the following rendering options can be performed: rendering the construction area in the sixth intensified color, rendering the drivable lane in the seventh intensified color, etc. For example, if the construction detour scenario meets the trigger conditions of V2X receiving the construction area coordinates and α suddenly decreasing in the construction direction, the construction area can be covered with a flashing red diagonal line (45° intersection, line width 10px) with 70% transparency; and a flowing blue arrow is generated in the drivable lane.

[0069] For narrow vehicle avoidance scenarios on rural roads, at least one of the following rendering options can be performed: compressing the width of the non-travel lane, rendering the non-travel lane in a second, weakened color, projecting a warning circle at the candidate meeting point, and displaying a historical accident heat map within the warning circle. For example, for a narrow vehicle avoidance scenario on a rural road that meets the trigger conditions of a lane width less than 3.5 meters and α greater than 0.4 in both the left and right oncoming lanes, the width of the non-travel lane can be compressed to 30% and overlaid with diagonal shading. A yellow warning circle (5 meters in diameter) can be projected at the potential meeting point, and a historical accident heat map can be displayed within the circle.

[0070] For scenes with no streetlights at night, at least one of the following rendering operations can be performed: generating dynamic lines in areas with missing lane markings, etc. For example, in a scene with no streetlights at night (global alpha decay), if the ambient light sensor detects an illuminance of less than 10 lux and the mean alpha value decreases by more than 50%, dynamic dashed lines (at a frequency of 2 Hz and a random length perturbation of ±20%) can be generated in areas with missing lane markings.

[0071] The disclosed embodiment provides a map rendering method that constructs a multidimensional perception matrix based on vehicle information, navigation information, and environmental information, and combines it with an LSTM model to predict the driver's possible areas of concern in the future time period. The rendering optimization of the navigation interface is then dynamically adjusted accordingly to improve the readability of the map and navigation efficiency.

[0072] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a map rendering device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0073] like Figure 3 As shown, the map rendering device 300 of this embodiment may include: a construction module 301, a generation module 302, a prediction module 303, and a rendering module 304. The construction module 301 is configured to construct a multidimensional perception matrix based on vehicle information, navigation information, and environmental information; the generation module 302 is configured to generate a time series matrix based on the multidimensional perception matrix at multiple consecutive time steps; the prediction module 303 is configured to input the time series matrix into a long short-term memory (LSTM) model to obtain a probability distribution map of the display screen's focus area within a preset future time period; and the rendering module 304 is configured to render the focus area based on the probability distribution map and the current driving scenario.

[0074] In this embodiment, the map rendering device 300 includes the construction module 301, the generation module 302, the prediction module 303, and the rendering module 304. The specific processing and technical effects thereof can be referred to in the respective Figure 1The relevant descriptions of steps 101-104 in the corresponding embodiment are not repeated here.

[0075] In some optional implementations of this embodiment, the vehicle information includes the vehicle state and the remaining mileage L, and the vehicle state includes the vehicle speed v, position (x, y) and lane type. , navigation information includes navigation stage , environmental information includes lighting environment and weather conditions , the multidimensional perception matrix M is [v, x, y, , , L, , ].

[0076] In some optional implementations of this embodiment, the LSTM model includes at least two LSTM layers; and the prediction module 403 is further configured to: perform gate calculation on each LSTM layer in the LSTM model to obtain a probability distribution map, wherein the gate calculation includes: an input gate, a forget gate, a candidate memory unit, a target memory unit and an output gate, the time series matrix is ​​the input of the first layer of LSTM, the output of the previous layer of LSTM is used as the input of the next layer of LSTM, and the output of the last layer of LSTM is a probability distribution map.

[0077] In some optional implementations of this embodiment, the prediction module 403 is further configured to: use the weight matrix corresponding to the input gate to perform weighted summation on the input and the previous hidden state of the LSTM layer, and add the bias vector corresponding to the input gate to obtain the initial output value of the input gate; use the activation function to compress the initial output value of the input gate to a first preset numerical interval to obtain the target output value of the input gate, wherein the target output value of the input gate is used to represent the amount of information entering the candidate memory unit.

[0078] In some optional implementations of this embodiment, the prediction module 403 is further configured to: use the weight matrix corresponding to the forget gate to perform weighted summation on the input and current hidden state of the LSTM layer, and add the bias vector corresponding to the forget gate to obtain the initial output value of the forget gate; use the activation function to compress the initial output value of the forget gate to a first preset numerical interval to obtain the target output value of the forget gate, wherein the target output value of the forget gate is used to represent the amount of forgotten information of the previous target memory unit.

[0079] In some optional implementations of this embodiment, the prediction module 403 is further configured to: use the weight matrix corresponding to the candidate memory unit to perform weighted summation on the input and the previous hidden state of the LSTM layer, and add the bias vector corresponding to the candidate memory unit to obtain the initial output value of the candidate memory unit; use the hyperbolic tangent function to compress the initial output value of the candidate memory unit to a second preset numerical interval to obtain the target output value of the candidate memory unit, wherein the target output value of the candidate memory unit is used to represent the candidate memory content.

[0080] In some optional implementations of this embodiment, the prediction module 403 is further configured to: calculate the product of the target output value of the forget gate and the previous target memory unit, and the sum of the products of the target output value of the output gate and the target output value of the candidate memory unit, and update the current target memory unit.

[0081] In some optional implementations of this embodiment, the prediction module 403 is further configured to: use the weight matrix corresponding to the output gate to perform weighted summation on the input and the previous hidden state of the LSTM layer, and add the bias vector corresponding to the output gate to obtain the initial output value of the output gate; use the activation function to compress the initial output value of the output gate to a first preset numerical interval to obtain the target output value of the output gate, wherein the target output value of the output gate is used to represent the amount of information output to the current hidden state in the current target memory unit.

[0082] In some optional implementations of this embodiment, the prediction module 403 is further configured to: calculate the product of the target output value of the output gate and the hyperbolic tangent value of the current target memory unit to obtain the current hidden state.

[0083] In some optional implementations of this embodiment, the prediction module 403 is further configured to: input the hidden state of the last time step of the last LSTM layer into the activation function to obtain a probability distribution graph.

[0084] In some optional implementations of this embodiment, the rendering module 404 is further configured to: determine the position of the focus area based on the resolution of the display screen; and render the position of the focus area based on the probability distribution map and the current driving scene.

[0085] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a tunnel scene, perform at least one of the following rendering operations: brightness compensation at positions where the probability is greater than a first preset threshold, reducing transparency at positions where the probability is less than a second preset threshold, rendering lanes as a first enhanced color, rendering lane lines as a second enhanced color, and reducing the map scale.

[0086] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a layered overpass scene, perform at least one of the following renderings: render the non-current layer road as a first weakened color, and project an arrow extending a preset length along the correct path.

[0087] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a highway ramp scene, perform at least one of the following renderings: compressing the non-target lane, expanding the target lane, and rendering the target lane as a third enhanced color.

[0088] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a complex urban intersection scene, perform at least one of the following renderings: use Gaussian blur to weaken non-critical factors, and render the target lane as a fourth enhanced color.

[0089] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a bad weather scene, perform at least one of the following renderings: rendering the lane line as a fifth enhanced color, increasing the line width of the lane line.

[0090] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a service area scene, perform at least one of the following renderings: display the number of idle charging piles and / or gas stations in the service area with labels, and use different colors to mark the cost-effectiveness of charging piles and / or gas stations.

[0091] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a detour scene of a construction section, perform at least one of the following renderings: render the construction area into the sixth enhanced color, and render the drivable lane into the seventh enhanced color.

[0092] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a narrow car avoidance scene on a rural road, perform at least one of the following renderings: compress the width of the non-driving direction lane, render the non-driving direction lane as a second weakened color, project a warning circle at the candidate meeting point, and display a historical accident heat map within the warning circle.

[0093] In some optional implementations of this embodiment, the rendering module 404 is further configured to: if the current driving scene is a night road scene without street lights, perform at least one of the following renderings: generate a dynamic line in the missing lane line area.

[0094] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0095] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0096] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0097] like Figure 4 As shown, device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. RAM 403 may also store various programs and data required for the operation of device 400. Computing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0098] Various components in device 400 are connected to I / O interface 405, including an input unit 406, such as a keyboard, mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, optical disk, etc.; and a communication unit 409, such as a network card, modem, wireless communication transceiver, etc. Communication unit 409 allows device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0099] Computing unit 401 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 401 performs the various methods and processes described above, such as the map rendering method. For example, in some embodiments, the map rendering method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed onto device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by computing unit 401, one or more steps of the map rendering method described above may be performed. Alternatively, in other embodiments, computing unit 401 may be configured to perform the map rendering method via any other suitable means (e.g., via firmware).

[0100] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0104] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0105] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0106] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved. This is not limited herein.

[0107] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A map rendering method, comprising: Construct a multi-dimensional perception matrix based on vehicle information, navigation information and environmental information; Generate a time series matrix based on the multi-dimensional perception matrix of multiple consecutive time steps; Input the time series matrix into a long short-term memory network (LSTM) model to obtain a probability distribution map of the focus area of ​​the display screen within a preset future time period; The focus area is rendered based on the probability distribution map and the current driving scene.

2. The method according to claim 1, wherein The vehicle information includes vehicle status and remaining mileage L. The vehicle status includes speed v, position (x, y) and lane type. , the navigation information includes navigation stage , the environmental information includes the lighting environment and weather conditions , the multidimensional perception matrix M is [v, x, y, , , L, , ].

3. The method according to claim 1, wherein The LSTM model includes at least two LSTM layers; as well as The time series matrix is ​​input into the long short-term memory network (LSTM) model to obtain a probability distribution diagram of the focus area of ​​the display screen within a preset time in the future, including: In the LSTM model, gate calculation is performed on each LSTM layer to obtain the probability distribution map, wherein the gate calculation includes: an input gate, a forget gate, a candidate memory unit, a target memory unit and an output gate, the time series matrix is ​​the input of the first layer of LSTM, the output of the previous layer of LSTM is used as the input of the next layer of LSTM, and the output of the last layer of LSTM is the probability distribution map.

4. The method according to claim 3, wherein: The gate calculation is performed on each LSTM layer in the LSTM model, including: Performing a weighted summation on the input and the previous hidden state of the LSTM layer using the weight matrix corresponding to the input gate, and adding the bias vector corresponding to the input gate to obtain the initial output value of the input gate; An activation function is used to compress the initial output value of the input gate into a first preset numerical interval to obtain a target output value of the input gate, wherein the target output value of the input gate is used to represent the amount of information entering the candidate memory unit.

5. The method according to claim 4, wherein The gate calculation is performed on each LSTM layer in the LSTM model, including: Performing a weighted summation on the input of the LSTM layer and the previous hidden state using the weight matrix corresponding to the forget gate, and adding the bias vector corresponding to the forget gate to obtain an initial output value of the forget gate; The activation function is used to compress the initial output value of the forget gate into the first preset numerical range to obtain the target output value of the forget gate, wherein the target output value of the forget gate is used to represent the amount of forgotten information of the previous target memory unit.

6. The method according to claim 5, wherein: The gate calculation is performed on each LSTM layer in the LSTM model, including: Performing a weighted summation on the input of the LSTM layer and the previous hidden state using the weight matrix corresponding to the candidate memory unit, and adding the bias vector corresponding to the candidate memory unit to obtain an initial output value of the candidate memory unit; The initial output value of the candidate memory unit is compressed into a second preset numerical interval using a hyperbolic tangent function to obtain a target output value of the candidate memory unit, wherein the target output value of the candidate memory unit is used to represent the candidate memory content.

7. The method according to claim 6, wherein: The gate calculation is performed on each LSTM layer in the LSTM model, including: Calculate the product of the target output value of the forget gate and the previous target memory unit, and the sum of the products of the target output value of the output gate and the target output value of the candidate memory unit, and update the current target memory unit.

8. The method according to claim 7, wherein: The gate calculation is performed on each LSTM layer in the LSTM model, including: Performing a weighted summation on the input of the LSTM layer and the previous hidden state using the weight matrix corresponding to the output gate, and adding the bias vector corresponding to the output gate to obtain an initial output value of the output gate; The activation function is used to compress the initial output value of the output gate to the first preset numerical interval to obtain the target output value of the output gate, wherein the target output value of the output gate is used to represent the amount of information output to the current hidden state in the current target memory unit.

9. The method according to claim 8, wherein The gate calculation is performed on each LSTM layer in the LSTM model, including: The product of the target output value of the output gate and the hyperbolic tangent value of the current target memory unit is calculated to obtain the current hidden state.

10. The method according to claim 9, wherein: The gate calculation is performed on each LSTM layer in the LSTM model, including: The hidden state of the last time step of the last LSTM layer is input into the activation function to obtain the probability distribution graph.

11. The method according to claim 1, wherein The rendering of the focus area based on the probability distribution map and the current driving scene includes: Determining a position of the region of interest based on a resolution of the display screen; Rendering is performed at a location of the focus area based on the probability distribution map and the current driving scene.

12. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a tunnel scene, at least one of the following rendering operations is performed: brightness compensation at locations where the probability is greater than a first preset threshold, reducing transparency at locations where the probability is less than a second preset threshold, rendering lanes as a first enhanced color, rendering lane lines as a second enhanced color, and reducing the map scale.

13. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a layered overpass scene, at least one of the following rendering operations is performed: rendering a non-current layer road as a first weakened color, and extending a projected arrow along a correct path by a preset length.

14. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a highway ramp scene, at least one of the following rendering is performed: compressing the non-target lane, expanding the target lane, and rendering the target lane as a third enhanced color.

15. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a complex urban intersection scene, at least one of the following rendering steps is performed: using Gaussian blur to weaken non-critical factors, and rendering the target lane as a fourth enhanced color.

16. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a bad weather scene, at least one of the following rendering operations is performed: rendering the lane line as a fifth enhanced color, and increasing the line width of the lane line.

17. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a service area scene, at least one of the following renderings is performed: displaying the number of idle charging piles and / or gas stations in the service area with labels, and marking the cost-effectiveness of charging piles and / or gas stations with different colors.

18. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a detour scene of a road construction section, at least one of the following renderings is performed: rendering the construction area into a sixth enhanced color, and rendering the drivable lane into a seventh enhanced color.

19. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a narrow car avoidance scene on a rural road, at least one of the following rendering operations is performed: compressing the width of the non-driving lane, rendering the non-driving lane as a second weakened color, projecting a warning circle at the candidate meeting point, and displaying a historical accident heat map within the warning circle.

20. The method according to claim 11, wherein The rendering of the position of the focus area based on the probability distribution map and the current driving scene includes: If the current driving scene is a night road scene without street lights, at least one of the following rendering steps is performed: generating a dynamic line in the missing lane line area.

21. A map rendering device, comprising: A building module is configured to build a multi-dimensional perception matrix based on vehicle information, navigation information, and environmental information; A generation module is configured to generate a time series matrix based on the multi-dimensional perception matrix of a plurality of consecutive time steps; A prediction module is configured to input the time series matrix into a long short-term memory network (LSTM) model to obtain a probability distribution map of the focus area of ​​the display screen within a preset time in the future; A rendering module is configured to render the focus area based on the probability distribution map and the current driving scene.

22. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 20.

23. A non-transitory computer-readable storage medium storing computer instructions, the computer instructions being configured to cause the computer to execute the method of any one of claims 1 to 20.

24. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 20.

Images