Navigation route planning method and device based on multi-modal interaction

Abstract

The invention relates to the field of intelligent navigation, and provides a navigation route planning method and device based on multi-modal interaction. The method comprises the following steps: determining a first expected travel time period and a first travel preference corresponding to each departure position, and a second expected travel time period and a second travel preference corresponding to each destination position; acquiring first historical traffic flow data of a first driving road section from the current position of the vehicle to each departure position in a first expected travel time period; acquiring second historical traffic flow data of a second driving road section from the current position of the vehicle to each destination position in a second expected travel time period; and performing navigation route planning based on the first expected travel time period, the first travel preference, the first historical traffic flow data, the second expected travel time period, the second travel preference and the second historical traffic flow data. According to the method, the efficient and convenient travel requirements of each user in a multi-user and multi-destination / departure place scene can be met, and the satisfaction degree and travel efficiency of the user on navigation service are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent navigation, and in particular to a navigation route planning method and device based on multimodal interaction. Background Art

[0002] With the rapid development of science and technology, intelligent navigation has become an indispensable part of people's daily travel.

[0003] Existing navigation systems can meet the user's requirements for efficient and convenient travel in a navigation scenario with a single user and a single destination. However, in a navigation scenario with multiple users and multiple destinations / departure points (for example, multiple users travel from the same departure point to different destinations, or multiple users travel from different departure points to different destinations, or multiple users travel from different departure points to the same destination), it is difficult to meet the requirements of each user for efficient and convenient travel. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a navigation route planning method and device based on multimodal interaction to solve the problem that the existing technology is difficult to meet the requirements of efficient and convenient travel for each user in a navigation scenario with multiple users and multiple destinations / departures.

[0005] According to a first aspect of an embodiment of the present application, a navigation route planning method based on multimodal interaction is provided, comprising:

[0006] Creating a navigation position group, the navigation position group including the vehicle's current position and a plurality of different departure positions and a common destination position, or including the vehicle's current position and a common departure position and a plurality of different destination positions, or including the vehicle's current position and a plurality of different departure positions and a plurality of different destination positions;

[0007] Determine a first desired travel time period and a first travel preference corresponding to each departure location, and a second desired travel time period and a second travel preference corresponding to each destination location;

[0008] For each departure position, obtaining first historical traffic flow data of a first driving section from the current position of the vehicle to the departure position during a first expected travel period;

[0009] For each destination location, obtaining second historical traffic flow data of a second driving section from the current location of the vehicle to the destination location during a second expected travel period;

[0010] Navigation route planning is performed based on a first expected travel time period, a first travel preference, a first historical traffic flow data, a second expected travel time period, a second travel preference, and a second historical traffic flow data, and a target navigation route is output.

[0011] According to a second aspect of an embodiment of the present application, a navigation route planning device based on multimodal interaction is provided, comprising:

[0012] A creation module is configured to create a navigation position group, the navigation position group including the vehicle's current position and a plurality of different departure positions and a same destination position, or including the vehicle's current position and a same departure position and a plurality of different destination positions, or including the vehicle's current position and a plurality of different departure positions and a plurality of different destination positions;

[0013] A determination module is configured to determine a first expected travel time period and a first travel preference corresponding to each departure location, and a second expected travel time period and a second travel preference corresponding to each destination location;

[0014] A first acquisition module is configured to acquire, for each departure position, first historical traffic flow data of a first driving section from a current position of the vehicle to the departure position in a first expected travel period;

[0015] A second acquisition module is configured to acquire, for each destination location, second historical traffic flow data of a second driving section from the current location of the vehicle to the destination location in a second expected travel period;

[0016] The navigation planning module is configured to plan a navigation route based on a first expected travel period, a first travel preference, a first historical traffic flow data, a second expected travel period, a second travel preference and a second historical traffic flow data, and output a target navigation route.

[0017] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0018] According to a fourth aspect of an embodiment of the present application, a readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0019] Compared with the prior art, the beneficial effects of the embodiments of the present application include at least: by creating a navigation location group, and comprehensively considering the first expected travel time period and the first travel preference corresponding to each departure location in the navigation location group, the second expected travel time period and the second travel preference corresponding to each destination location, and the first historical traffic flow data of the first driving section from the current location of the vehicle to each departure location, and the second historical traffic flow data of the second driving route from the current location of the vehicle to each destination location, a navigation route is planned that can meet the efficient and convenient travel requirements of each user in a multi-user and multi-destination / departure scenario, thereby improving user satisfaction with the navigation service and effectively improving travel efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a schematic diagram of a first navigation application scenario provided in an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a second navigation application scenario provided in an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a third navigation application scenario provided in an embodiment of the present application;

[0024] Figure 4 It is a flowchart of a navigation route planning method provided in an embodiment of the present application;

[0025] Figure 5 is a structural diagram of a multimodal interaction system provided in an embodiment of the present application;

[0026] Figure 6 is a structural diagram of another multimodal interaction system provided in an embodiment of the present application;

[0027] Figure 7 It is a structural diagram of a navigation route planning system provided by an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of a fourth navigation application scenario provided in an embodiment of the present application;

[0029] Fig. 9 It is a flowchart of a method for calculating a single-source shortest path provided in an embodiment of the present application;

[0030] Fig.10 is a schematic diagram of a fifth navigation application scenario provided in an embodiment of the present application;

[0031] Fig.11 It is a structural schematic diagram of a navigation route planning device based on multimodal interaction provided in an embodiment of the present application;

[0032] Fig.12 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0034] A navigation route planning method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0035] Figure 1 1 is a schematic diagram of a first navigation application scenario provided in an embodiment of the present application. The application scenario may include a vehicle 101, first users 102, 103 and 104, and a server 105.

[0036] The first users 102 , 103 and 104 may be users who are located at different departure locations (departure addresses), want to go to the same destination location (destination address), and are waiting to be picked up by the vehicle 101 .

[0037] Figure 2 1 is a schematic diagram of a second navigation application scenario provided in an embodiment of the present application. The application scenario may include a vehicle 101 , a server 105 , and second users 106 , 107 , and 108 .

[0038] Among them, the second users 106 , 107 and 108 may be users who are located at the same departure location (departure address), want to go to different destination locations (destination addresses), and are waiting for the vehicle 101 to pick them up.

[0039] Figure 3 1 is a schematic diagram of a third navigation application scenario provided in an embodiment of the present application. The application scenario may include a vehicle 101 , a server 105 , and third users 109 , 110 , and 111 .

[0040] Among them, the third users 109, 110 and 111 may be users who are located at different departure locations (departure addresses), want to go to different destination locations (destination addresses), and are waiting to be picked up by the vehicle 101.

[0041] Vehicle 101 may be a new energy vehicle. A new energy vehicle refers to a vehicle that uses new energy (non-traditional petroleum and diesel energy) and has advanced technology. These vehicles use a new power system that can effectively reduce vehicle emissions, reduce environmental impact, and improve energy efficiency. The new energy vehicles of the embodiments of the present application include but are not limited to the following types of vehicles: electric vehicles (EV), battery electric vehicles (BEV), fuel cell electric vehicles (FCEV), plug-in hybrid electric vehicles (PHEV), and hybrid electric vehicles (HEV), etc.

[0042] The server 105 may be a server that provides various services, for example, it may be a single server, or a server cluster consisting of several servers, or it may be a cloud computing service center, which is not limited in the embodiments of the present application.

[0043] It should be noted that the specific types, quantities and combinations of vehicle 101, first users 102, 103 and 104, second users 106, 107 and 108, third users 109, 110 and 111 and server 105 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present application are not limited to this.

[0044] At present, the existing navigation system can handle the navigation task of a single user going to a single destination, but in the case of multiple users and multiple destinations / departures, the driver is usually required to manually input each destination / departure and determine the navigation order of each destination / departure based on experience. However, the driver has limited knowledge of factors such as the distance between different destinations and departures (such as the current location of the vehicle), the direct road conditions, the traffic congestion between destinations, and the efficiency of each road section at different times. Therefore, the navigation order of each destination / departure determined by the driver based on experience is not accurate. After that, it may be necessary to repeatedly compare and adjust the navigation order of the destination / departure to determine a navigation route with better road conditions and more efficiency. The whole process is very time-consuming and requires the driver to have a high degree of understanding of the geographical location of each destination / departure, which affects the user experience of the navigation service and the travel efficiency is relatively low.

[0045] In view of this, the embodiment of the present application provides a navigation route planning method, which realizes intelligent planning of navigation routes for multi-user and multi-destination / departure scenarios by integrating advanced multimodal interaction technology and AI (Artificial Intelligence) optimization algorithm. It can comprehensively consider multiple factors such as the user's expected travel time, traffic conditions, user preferences, etc., and provide users with the best travel plan (navigation route). This innovation not only improves the user's travel efficiency and experience, but also injects new vitality into the development of smart travel.

[0046] Furthermore, an embodiment of the present application also provides a navigation route planning method, which creates a navigation location group and comprehensively considers the first expected travel time period and the first travel preference corresponding to each departure location in the navigation location group, the second expected travel time period and the second travel preference corresponding to each destination location, and the first historical traffic flow data of the first driving section from the current location of the vehicle to each departure location, and the second historical traffic flow data of the second driving route from the current location of the vehicle to each destination location, so as to plan a navigation route that can meet the efficient and convenient travel requirements of each user in a multi-user and multi-destination / departure scenario, thereby improving user satisfaction with the navigation service and effectively improving travel efficiency.

[0047] Figure 4 It is a flowchart of a navigation route planning method provided in an embodiment of the present application. Figure 3 The navigation route planning method can be Figure 1 to Figure 3 The server 105 executes. Figure 4 As shown, the navigation route planning method specifically includes the following steps:

[0048] Step S401, creating a navigation position group, the navigation position group includes the vehicle's current position and multiple different departure positions and a same destination position, or includes the vehicle's current position and a same departure position and multiple different destination positions, or includes the vehicle's current position and multiple different departure positions and multiple different destination positions.

[0049] Figure 5 It is a structural diagram of a multimodal interaction system provided in an embodiment of the present application.

[0050] See also Figure 5, the multimodal interaction system of the embodiment of the present application includes a cockpit software system 501, a T-BOX (telematics processor) 502, a TSP (Telematics Service Provider, automotive teleservice provider) 503, a cloud information platform 504, a map navigation system 505, a GPS positioning system 506 and a smart voice assistant 507. Among them, the cockpit software system 501, the T-BOX 502, the TSP 503, the map navigation system 505, the GPS positioning system 506 and the smart voice assistant 507 can be deployed in the vehicle 101. The cockpit software system 501, the T-BOX 502, the TSP 503, the map navigation system 505, the GPS positioning system 506, the smart voice assistant 507, and the cloud information platform 504 can communicate through the CAN (Controller Area Network) bus. The vehicle 101 can establish a communication connection with the server 105 through the cloud information platform 504.

[0051] A user (such as a driver) can turn on or off the multi-destination / departure smart navigation function (mode) through the cockpit software system 501 of the vehicle 101 , and bind the multi-destination / departure smart navigation function (mode) to the vehicle system account of the vehicle 101 .

[0052] The current position of the vehicle 101 may be collected by the GPS positioning system 506 and transmitted to the map navigation system 505 via the CAN bus.

[0053] As an example, see Figure 1 and Figure 5 , when multiple users (such as the first user 102, 103, and 104) are in the vehicle 101, the first users 102, 103, and 104 can input the destination they want to go to through the touch operation of the central control screen of the cockpit software system 501 of the vehicle 101, or can input the destination they want to go to through voice interaction through the intelligent voice assistant 507 of the vehicle 101. Both the cockpit software system 501 and the intelligent voice assistant 507 support multiple users to input different destinations at the same time.

[0054] Figure 6 It is a structural diagram of another multimodal interaction system provided in an embodiment of the present application.

[0055] As an example, see Figure 6, the user (such as the driver) can open the multi-destination / departure smart navigation mode through the cockpit software system 501 of the vehicle 101, and establish a communication connection with the cloud information platform 504 through the T-BOX 502 or TSP 503 of the vehicle 101. User devices (such as smart phones, etc.) can establish a communication connection with the cloud information platform 504 through the 4G / 5G network. Each user can use his user device to input the destination location / departure location he wants to go to, and then transmit it to the vehicle 101 through the cloud information platform 504 through the T-BOX 502 or TSP 503. The vehicle 101 transmits the received destination location / departure location to the cockpit software system 501 and the map navigation system 505 through the CAN bus.

[0056] As an example, in a scenario where multiple users travel from the same departure point to different destinations, the driver can create a multi-destination navigation itinerary by performing touch operation on the central control display screen of the cockpit software system 501 of the vehicle 101, or by performing voice interaction through the intelligent voice assistant 507. The cockpit software system 501 generates a QR code based on the multi-destination navigation itinerary and displays it on the central control display screen. Each user in the vehicle (such as the first user 102, 103, and 104) can scan the QR code using their respective user devices (such as smart phones), and enter the destination location (destination address) they want to go to on the information entry page that jumps out after scanning the QR code by text input, or share the location information of the destination location they want to go to in other software (such as instant messaging software such as WeChat and QQ) to the information entry page.

[0057] As another example, in a scenario where multiple users travel from different departure locations to different destinations or the same destination, the driver can remotely share the QR code of the multi-destination navigation itinerary created in the cockpit software system 501 with each user, so that each user can remotely enter their departure location and the destination they want to go to by scanning the QR code shared by the driver.

[0058] Figure 7 It is a structural diagram of a navigation route planning system provided in an embodiment of the present application.

[0059] See also Figure 7 The navigation route planning system of the embodiment of the present application includes a map navigation system 505 and a map cloud database 701. The map navigation system 505 can communicate with the map cloud database 701 through a 4G / 5G network to transmit the collected current position of the vehicle, the departure position of each first user, and the destination position of each second user to the map cloud database 701. The map cloud database 701 can be deployed in the server 105, wherein the server 105 can be a cloud server.

[0060] The above-mentioned departure location (departure address) and destination location (destination address) can be collected in the address library of the map navigation system 505 of the vehicle 101, and the address library can be bound to the vehicle system account of the vehicle 101, so that it can be directly called when needed later.

[0061] Before creating a navigation location group, you can first perform preliminary processing on each departure location and each destination location to ensure the accuracy and validity of the data. Among them, the preliminary processing mainly includes data cleaning, format conversion and verification steps. Specifically, the above-mentioned departure location and destination location can be subjected to address extraction, administrative district parsing, address structuring, address error correction and completion, address standardization, address anomaly detection, multi-address similarity judgment, longitude and latitude query and coordinate system conversion.

[0062] For the departure and destination locations input in text form, named entity recognition algorithms (NER) and deep learning-based named entity recognition models such as BiLSTM-CRF (bidirectional long short-term memory network with conditional random field) can be used to identify address entities in the text, and slot filling technology can be used to identify and classify the various components of the address, such as province, city, district, etc. Spelling correction algorithms (such as Levenshtein distance (edit distance) or BK-Tree (inverted keyword tree) algorithm) can be used to identify and correct spelling errors in the destination and departure locations.

[0063] In the case of supervised learning, a classification algorithm (Support Vector Machine (SVM) or neural network) can be trained through the data set to identify abnormal addresses in the destination and departure locations. In the case of unsupervised learning, a clustering algorithm (such as K-means or DBSCAN) is used to identify abnormal patterns of input destination and departure locations.

[0064] Geocoding technology is used to convert the input destination and departure locations into longitude and latitude geographic coordinates; dynamic time warping (DTW) and hidden Markov model (HMM) are used for map matching to ensure the correct matching of GPS track points with the road network map.

[0065] The above-mentioned algorithms and intelligent big models can learn geospatial data through deep learning (such as convolutional neural networks (CNN) and recurrent neural networks (RNN)), as well as deep learning and optimization through reinforcement learning and other methods.

[0066] Step S402: determining a first expected travel time period and a first travel preference corresponding to each departure location, and a second expected travel time period and a second travel preference corresponding to each destination location.

[0067] The first expected travel time period indicates the time period when the user expects the vehicle to arrive at the departure location provided by the user.

[0068] The second expected travel time period indicates the time period during which the user expects the vehicle to deliver him to the destination location provided by him.

[0069] The first travel preference and the second travel preference indicate the user's daily preferred travel habits, such as avoiding toll roads or preferring highways. A collaborative filtering method can be used to learn the user's travel preferences by analyzing the user's historical travel behavior. A matrix decomposition method (such as SVD (singular value decomposition)) can be used to predict the user's travel preferences for different routes.

[0070] As an example, a user can use a smartphone to scan the QR code of a multi-destination / departure navigation itinerary shared by a driver, and enter the departure location, the first desired travel time, and the first travel preference on the jump page; enter the destination location, the second desired travel time, and the second travel preference.

[0071] Step S403: for each departure position, obtaining first historical traffic flow data of a first driving section from the current position of the vehicle to the departure position during a first expected travel period.

[0072] Figure 8 is a schematic diagram of the fourth navigation application scenario provided by the embodiment of the present application. Figure 8 In a scenario where multiple users (including the first user 102, 103, and 104) travel to the same destination from different departure locations, it is assumed that the departure location provided by the first user 102 is point B, the departure location provided by the first user 103 is point C, the departure location provided by the first user 104 is point D, the current location of the vehicle 101 is point A, and the destination locations that the first users 102, 103, and 104 want to travel to are all point E. Then, the server 105 can interact with the traffic management system to obtain the first historical traffic flow data 1 of the first driving section 1 of the vehicle 101 from point A to point B as the starting position provided by the first user 102 (i.e., A→B) during the first expected travel period (e.g., T1) of the first user 102; obtain the first historical traffic flow data 2 of the first driving section 2 of the vehicle 101 from point A to point C as the starting position provided by the first user 103 (i.e., A→C) during the first expected travel period (e.g., T2) of the first user 103; and obtain the first historical traffic flow data 3 of the first driving section 3 of the vehicle 101 from point A to point D as the starting position provided by the first user 104 (i.e., A→D) during the first expected travel period (e.g., T3) of the first user 104.

[0073] The first historical traffic flow data includes road congestion, traffic accidents, construction information, etc. of the first driving section within the first expected time period.

[0074] Step S404: for each destination location, obtaining second historical traffic flow data of a second driving section from the current location of the vehicle to the destination location during a second expected travel period.

[0075] Specifically, the second historical traffic flow data of each second driving section in the second expected travel period may be obtained in the same manner as the above-mentioned first historical traffic flow data acquisition method, which will not be described in detail herein.

[0076] Step S405 , performing navigation route planning based on the first expected travel time period, the first travel preference, the first historical traffic flow data, the second expected travel time period, the second travel preference and the second historical traffic flow data, and outputting a target navigation route.

[0077] The technical solution provided in the embodiment of the present application creates a navigation position group and comprehensively considers the first expected travel time period and the first travel preference corresponding to each departure position in the navigation position group, the second expected travel time period and the second travel preference corresponding to each destination position, and the first historical traffic flow data of the first driving section from the current position of the vehicle to each departure position, and the second historical traffic flow data of the second driving route from the current position of the vehicle to each destination position, so as to plan a navigation route that can meet the efficient and convenient travel requirements of each user in a multi-user and multi-destination / departure scenario, thereby improving user satisfaction with the navigation service and effectively improving travel efficiency.

[0078] In some embodiments, performing navigation route planning based on the first expected travel time period, the first travel preference, the first historical traffic flow data, the second expected travel time period, the second travel preference, and the second historical traffic flow data, and outputting a target navigation route includes:

[0079] Based on the first historical traffic flow data of each first driving section, predicting the first traffic congestion state of each first driving section in the first expected occurrence time period;

[0080] Based on the second historical traffic flow data of each second driving section, predicting the second traffic congestion state of each second driving section in the second expected travel period;

[0081] sorting the departure locations based on the first traffic congestion state and the first travel preference to obtain a first navigation sequence;

[0082] sorting the destination locations based on the second traffic congestion status and the second travel preference to obtain a second navigation sequence;

[0083] A navigation route is planned based on the first navigation sequence and the second navigation sequence, and a target navigation route is output.

[0084] As an example, see Figure 8 , taking the first driving section 1 (A (starting point) → B (end point)) as an example, assuming that the first expected travel time period T1 corresponding to the first driving section 1 is 9:00-10:00, then the map cloud database 701 of the server 105 can retrieve the first historical traffic flow data corresponding to the first expected travel time period T1 of the previous w days (w is an integer ≥1), and then input the first historical traffic flow data of the previous w days into a pre-trained prediction model (which can be a convolutional neural network model or other deep learning network model), and output the first traffic congestion state of the first driving section 1 in the first expected travel time period T1. The first traffic congestion state includes m (m is an integer ≥1) forward driving paths from the starting point (point A) to the end point (point B), n reverse driving paths from the end point (point B) to the starting point (point A), and the vehicle flow, path node topology, driving distance (Euclidean distance) and other information corresponding to each forward driving path and each reverse path.

[0085] According to the above method, the second traffic congestion status of the first driving section 2 (A (starting point) → C (end point)), the first driving section 3 (A (starting point) → D (end point)), and each second driving section during the second expected travel period can be predicted, which will not be repeated here.

[0086] In some embodiments, based on the first traffic congestion state and the first travel preference, each departure location is sorted to obtain a first navigation sequence, including:

[0087] For each departure location, based on the first traffic congestion state and the first travel preference, determine a first shortest driving path from the current location of the vehicle to a first driving section of the departure location; wherein a first shortest driving time of the first shortest driving path is within a first expected travel time period;

[0088] The departure locations are sorted based on the first shortest travel time to obtain a first navigation sequence.

[0089] As an example, see Figure 8, taking the first driving section 1 (A (starting point) → B (end point)) as an example, assuming that the first travel preference of the first user 102 is to prefer express routes and avoid toll sections, then k forward candidate driving paths that meet the first travel preference of the first user 102 among the m driving paths from A (starting point) to B (end point) and g reverse candidate driving paths that meet the first travel preference of the first user 102 among the n driving paths from B (end point) to A (starting point) can be searched first, where k≤m, g≤n. The overlapping candidate driving paths among the k forward candidate driving paths and the g reverse candidate driving paths are eliminated to obtain s entries of candidate driving paths. Then, according to the first traffic congestion state, the travel time of each target candidate driving path among the s target candidate driving paths is calculated respectively, and a target candidate driving path with the shortest travel time and within the first expected travel time period is determined as the first shortest driving path 1 of the first driving section 1 (A (starting point) → B (end point)).

[0090] By searching for the forward candidate driving paths from the starting point to the end point in combination with the user's travel preferences, searching for the reverse candidate driving paths from the end point to the starting point, and eliminating the overlapping candidate driving paths between the forward candidate driving paths and the reverse candidate driving paths, and then combining the first traffic congestion status and the user's expected travel time to perform path screening, the global optimal driving path that meets the user's travel preferences and expected travel time can be searched in the shortest time, thereby improving the user's experience of navigation services.

[0091] Fig. 9 It is a flowchart of a method for calculating a single-source shortest path provided in an embodiment of the present application.

[0092] See also Fig. 9 First, the starting path node, the intermediate path node, and the terminal path node of each target candidate driving path can be determined according to the first traffic congestion state. Then, the following is performed: Fig. 9 The process shown determines the first shortest driving route 1 of the first driving section 1 (A (starting point) → B (end point)).

[0093] Specifically, step S901, initialize the distance of all vertices (including intermediate path nodes and terminal path nodes) to infinity. Step S902, set the distance of the source point (such as the starting path node) to zero. Step S903, create a priority queue and add the source point. Step S903, create a priority queue and add the source point. Step S904, determine whether the priority queue is empty. If the priority queue is empty, end the process. If the priority queue is not empty, jump to step S905, pop the vertex U and the distance from the priority queue. Step S906, determine whether the neighbors V of all vertex U have been traversed. If so, return to step S904. If not, jump to step S907, iterate each neighbor V of the vertex. Step S908, calculate the distance from the source point to the neighbor V. Step S909, determine whether the newly calculated distance from the source point to the neighbor V is less than the distance of the known neighbor V. If not, return to step S906. If yes, jump to step S910 and update the distance of neighbor V. S911, add neighbor V and the new distance to the priority queue, and return to step S906.

[0094] The first shortest driving path 2 of the first driving section 2 (A (starting point) → C (end point)) and the first shortest driving path 3 of the first driving section 3 (A (starting point) → D (end point)) can be calculated in a similar manner.

[0095] Next, the departure locations are sorted according to the first shortest driving times corresponding to the first shortest driving path 1, the first shortest driving path 2, and the first shortest driving path 3 to obtain a first navigation sequence.

[0096] As an example, assuming that the first shortest driving times corresponding to the first shortest driving path 1, the first shortest driving path 2, and the first shortest driving path 3 are arranged in ascending order as follows: first shortest driving path 3 < first shortest driving path 1 < first shortest driving path 2, then the first navigation sequence is [D, B, C], indicating that the order in which vehicle 101 arrives at the starting positions provided by the first users 102, 103, and 104 is D→B→C.

[0097] Furthermore, D can be used as the starting point to calculate the second shortest driving time 1 from D to B and the second shortest driving time 2 from D to C. If the second shortest driving time 1> the second shortest driving time 2, the first navigation sequence can be adjusted to obtain an adjusted first navigation sequence [D, C, B].

[0098] In a similar manner, various destination locations may be sorted based on the second traffic congestion status and the second travel preference to obtain a second navigation sequence, which will not be described in detail herein.

[0099] See also Figure 8In combination with the above example, assuming that the first navigation sequence obtained through the above steps is [D, C, B] and the second navigation sequence is [E], then based on the first navigation sequence and the second navigation sequence, the navigation route is planned, and the target navigation route obtained is [A, D, C, B, E], which means that the pick-up and drop-off sequence for vehicle 101 to arrive at the departure location provided by the first user 102, 103, 104 and send them to the destination is A→D→C→B→E.

[0100] The above implementation method can take into account the user's travel preferences, expected time periods, and traffic congestion conditions from the vehicle's current location to each departure location / destination location during the navigation route planning process, so that the obtained navigation route can meet the requirements of each user for efficient and convenient travel in a multi-user and multi-destination / departure location scenario as much as possible, thereby improving the user's satisfaction with the navigation service and effectively improving the overall travel efficiency.

[0101] In some embodiments, determining a first shortest travel time of a first travel segment from a current location of the vehicle to a starting location based on a first traffic congestion state and a first travel preference includes:

[0102] Determine the navigation application scenario;

[0103] Determining a first preference weight of the first travel preference based on the navigation application scenario;

[0104] Performing navigation route planning based on the first preference weight and the first traffic congestion status to obtain an optimal driving trajectory for the first driving section;

[0105] According to the optimal driving trajectory and the vehicle driving speed, the first shortest driving time is calculated.

[0106] Among them, navigation application scenarios include but are not limited to urban commuting scenarios, long-distance travel scenarios, and short-distance ride scenarios.

[0107] For urban commuting scenarios, users generally focus more on minimizing commuting time and avoiding congested roads. For long-distance travel scenarios, users generally focus more on factors such as driving distance, fuel consumption, and rest stops along the way. For short-distance rides, users generally focus more on quick arrival and convenience.

[0108] In view of this, the embodiment of the present application pre-sets the corresponding relationship between the user's general preferences and their weight factors corresponding to different navigation application scenarios, and stores the corresponding relationship in the map cloud database 701 of the server 105.

[0109] As an example, the corresponding relationship between navigation application scenarios, user general preferences and weight factors is shown in Table 1.

[0110] Table 1. Correspondence between navigation application scenarios, user preferences and weight factors

[0111]

[0112] Fig.10 is a schematic diagram of the fifth navigation application scenario provided by the embodiment of the present application. Figure 2 and Fig.10 In a scenario where multiple users (such as second users 106, 107, and 108) travel from the same departure point to different destinations, assuming that the departure locations provided by the second users 106, 107, and 108 are all point A, the current location of the vehicle 101 is also point A, the destination location that the second user 106 wants to go to is point F, the destination location that the second user 107 wants to go to is point G, and the destination location that the second user 108 wants to go to is point H, the server 105 recognizes that the current navigation application scenario is a short-distance ride scenario. Then, first, the first preference weights of the first travel preferences of the second users 106, 107, and 108 are determined based on the short-distance ride scenario.

[0113] The following describes in detail the process of determining the first preference weight of the first travel preference of the second user 106 based on a short-distance ride scenario as an example.

[0114] Assuming that the first travel preference of the second user 106 is to prefer fast routes and avoid toll roads, and the current navigation application scenario is a short-distance ride scenario, it can be seen from the query of the above Table 1 that the general user preferences corresponding to the short-distance ride scenario are fast arrival and convenience. By comparison, it can be seen that the first travel preference of the second user 106 is basically consistent with the general user preferences corresponding to the short-distance ride scenario. According to the weight factors corresponding to Fac_short_1 and Fac_short_2, it can be determined that the first preference weight of the first travel preference of the second user 106 is 1.

[0115] After determining the first preference weight of the second user 106, a navigation route is planned based on the first preference weight of the second user 106 and the first traffic congestion state of the first driving section (A→F) (specifically, an ant colony algorithm, a genetic algorithm, etc. may be used for navigation route planning), and an optimal driving trajectory of the first driving section (A→F) is obtained. Finally, according to the driving distance of the optimal driving trajectory and the driving speed of the vehicle, the first shortest driving time of the first driving section (A→F) is calculated.

[0116] The above implementation method, by combining navigation application scenarios and user travel preferences for navigation route planning, can take into account the different travel preferences of each user as much as possible while maximally improving the travel efficiency of the entire vehicle passengers and the user's satisfaction with the navigation service.

[0117] In some embodiments, performing navigation route planning based on the first navigation sequence and the second navigation sequence and outputting a target navigation route includes:

[0118] Obtaining a first navigation requirement for a departure location and a second navigation requirement for a destination location;

[0119] Based on the first navigation requirement and the second navigation requirement, the first navigation sequence and the second navigation sequence are adjusted to obtain a combined navigation sequence; wherein the combined navigation sequence includes the current position of the vehicle, and each departure position and each destination position;

[0120] Generate a target navigation route according to the combined navigation sequence and output the target navigation route.

[0121] The first navigation requirement may be a user's requirement on the order in which vehicles arrive at the starting locations provided by the user.

[0122] The second navigation requirement may be a user's requirement on the order in which vehicles arrive at the destination locations provided by the user.

[0123] The user may input the first navigation requirement / second navigation requirement at the same time as inputting the departure location / destination location.

[0124] As an example, see Fig.10 , assuming that the first navigation sequence obtained according to the above implementation is [A], the second navigation sequence is [G, H, F], and the second navigation requirement is "request to reach the destination location F first", then the map cloud database 701 of the server 105 can first verify whether the navigation sequences [F, G, H] and [F, H, G] can meet the travel preferences and expected travel time of each user. If both can be met, then further compare the estimated driving time of the navigation sequences [F, G, H] and [F, H, G], and determine the navigation sequence with the shortest estimated driving time as the adjusted second navigation sequence. Exemplarily, if the navigation sequences [F, G, H] and [F, H, G] can both meet the travel preferences and expected travel time of each user, and the estimated driving time of [F, G, H] is the shortest, then [F, G, H] can be determined as the adjusted second navigation sequence.

[0125] The first navigation sequence [A] and the adjusted second navigation sequence [F, G, H] are combined to obtain a combined navigation sequence [A, F, G, H]. Finally, a target navigation route A→F→G→H is generated according to the combined navigation sequence [A, F, G, H] and output to the map navigation system 505 of the vehicle 101.

[0126] In the above implementation, by adjusting the first navigation sequence and the second navigation sequence in combination with the navigation needs of the users, the obtained navigation route can better meet the travel needs and actual conditions of each user, thereby further improving the user's satisfaction with the navigation service.

[0127] In some embodiments, the above method further comprises:

[0128] Show the user the target navigation route;

[0129] Obtaining the user's modification instructions for the target navigation route;

[0130] The target navigation route is modified according to the modification instruction to obtain a modified navigation route, and the modified navigation route is output.

[0131] Modification instructions can indicate specific areas that the user wants to avoid or a specific path to take.

[0132] As an example, the map cloud database 701 of the server 105 can transmit the calculated target navigation route to the map navigation system 505 of the vehicle 101, and the central control screen of the map navigation system 505 can display the target navigation route to the user, or the target navigation route can be shared with the user's user device through the map navigation system 505, so that the user can view the target navigation route on the user device and enter the modification instruction. Specifically, the user can enter the modification instruction by touch operation through the cockpit software system 501 of the vehicle 101, voice interaction through the intelligent voice assistant 507, or text entry through the user device.

[0133] After receiving the modification instruction from the user for the target navigation route, the map navigation system 505 transmits the modification instruction to the map cloud database 701. The map cloud database 701 recalculates and modifies the target navigation route based on the modification instruction to obtain the modified navigation route, and returns the modified navigation route to the map navigation system 505.

[0134] The above implementation manner can adjust the target navigation route once or multiple times based on the user's modification instruction, so that the obtained navigation route can meet the personalized needs of each user as much as possible, thereby improving the user's satisfaction with the navigation service.

[0135] In some embodiments, the above method further comprises:

[0136] Obtain real-time traffic data of the target navigation route and the vehicle's current energy consumption information;

[0137] According to the real-time traffic data and the current energy consumption information of the vehicle, the target navigation route is optimized to obtain the optimized navigation route.

[0138] Real-time traffic data, including but not limited to the road slope, curvature, traffic light distribution and real-time traffic congestion conditions of each driving section of the target navigation route.

[0139] The vehicle's current energy consumption information indicates the vehicle's current fuel consumption / power status and power performance, etc.

[0140] As an example, the map cloud database 701 of the server 105 can interact with the traffic management system to obtain real-time traffic data of the target navigation route; and interact with the vehicle 101 to obtain the current energy consumption information of the vehicle. Then, the map cloud database 701 optimizes the target navigation route based on the acquired real-time traffic data and the current energy consumption information of the vehicle, so as to minimize the fuel consumption or power usage of the vehicle while ensuring driving safety. For example, in a traffic-congested urban commuting scenario, the map cloud database 701 can switch the congested section in the target navigation route to a relatively unobstructed but slightly longer section to avoid frequent stopping and starting, thereby reducing fuel consumption or power consumption.

[0141] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0142] In summary, the technical solution provided by the embodiments of the present application has at least the following beneficial effects:

[0143] 1) By integrating advanced AI algorithms and comprehensively considering user travel preferences, real-time traffic conditions, distance, congestion and user's desired travel time, it can significantly improve travel efficiency and greatly increase user satisfaction with navigation services.

[0144] 2) Support multiple interactive methods such as voice and touch screen, broadcast driving information in real time to ensure driving safety, and allow passengers to adjust their destinations during the trip, enhancing flexibility and interactivity.

[0145] 3) It can intelligently and automatically adjust the path planning strategy according to the navigation application scenarios (such as urban commuting scenarios, long-distance travel scenarios, and short-distance ride scenarios) to meet the personalized needs of each user and improve the adaptability of the system.

[0146] 4) It can optimize vehicle fuel consumption / power usage during navigation, reduce resource consumption, and at the same time reduce manual planning costs and save time costs.

[0147] 5) Ensure the system's high efficiency in processing big data and complex calculations, update traffic information in real time, and provide accurate navigation services.

[0148] 6) Through intelligent, personalized route planning and multimodal interaction, it provides users with efficient, convenient and personalized travel experience.

[0149] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0150] Fig.11 is a structural diagram of a navigation route planning device based on multimodal interaction provided in an embodiment of the present application. Fig.11 As shown, the navigation route planning device based on multimodal interaction includes:

[0151] The creation module 1101 is configured to create a navigation position group, wherein the navigation position group includes a current position of the vehicle and a plurality of different departure positions and a same destination position, or includes the current position of the vehicle and a same departure position and a plurality of different destination positions, or includes the current position of the vehicle and a plurality of different departure positions and a plurality of different destination positions;

[0152] The determination module 1102 is configured to determine a first expected travel time period and a first travel preference corresponding to each departure location, and a second expected travel time period and a second travel preference corresponding to each destination location;

[0153] The first acquisition module 1103 is configured to acquire, for each departure location, first historical traffic flow data of a first driving section from the current location of the vehicle to the departure location in a first expected travel period;

[0154] The second acquisition module 1104 is configured to acquire, for each destination location, second historical traffic flow data of a second driving section from the current location of the vehicle to the destination location in a second expected travel period;

[0155] The navigation planning module 1105 is configured to plan a navigation route based on the first expected travel time period, the first travel preference, the first historical traffic flow data, the second expected travel time period, the second travel preference and the second historical traffic flow data, and output a target navigation route.

[0156] In some embodiments, the navigation planning module 1105 includes:

[0157] A first prediction unit is configured to predict a first traffic congestion state of each first driving section in a first expected occurrence time period based on first historical traffic flow data of each first driving section;

[0158] A second prediction unit is configured to predict a second traffic congestion state of each second driving section in a second expected travel period based on second historical traffic flow data of each second driving section;

[0159] A first sorting unit is configured to sort each departure location based on a first traffic congestion state and a first travel preference to obtain a first navigation sequence;

[0160] A second sorting unit is configured to sort each destination location based on a second traffic congestion state and a second travel preference to obtain a second navigation sequence;

[0161] The planning output unit is configured to perform navigation route planning based on the first navigation sequence and the second navigation sequence, and output a target navigation route.

[0162] In some embodiments, the first sorting unit includes:

[0163] A path determination component is configured to determine, for each departure location, a first shortest driving path of a first driving segment from a current location of the vehicle to the departure location based on a first traffic congestion state and a first travel preference; wherein a first shortest driving time of the first shortest driving path is within a first expected travel time period;

[0164] The position sorting component is configured to sort each departure position based on a first shortest travel time to obtain a first navigation sequence.

[0165] In some embodiments, the above-mentioned path determination component may be specifically configured as follows:

[0166] Determine the navigation application scenario;

[0167] Determining a first preference weight of the first travel preference based on the navigation application scenario;

[0168] Performing navigation route planning based on the first preference weight and the first traffic congestion status to obtain an optimal driving trajectory for the first driving section;

[0169] According to the optimal driving trajectory and the vehicle driving speed, the first shortest driving time is calculated.

[0170] In some embodiments, the above-mentioned planning output unit includes:

[0171] A demand acquisition component is configured to acquire a first navigation demand for a departure location and a second navigation demand for a destination location;

[0172] The adjustment component is configured to adjust the first navigation sequence and the second navigation sequence based on the first navigation requirement and the second navigation requirement to obtain a combined navigation sequence; wherein the combined navigation sequence includes the current position of the vehicle, and each departure position and each destination position;

[0173] The output component is configured to generate a target navigation route according to the combined navigation sequence and output the target navigation route.

[0174] In some embodiments, the above apparatus further comprises:

[0175] A display module is configured to display a target navigation route to a user;

[0176] An instruction acquisition module is configured to acquire a user's modification instruction for a target navigation route;

[0177] The route modification module is configured to modify the target navigation route according to the modification instruction, obtain the modified navigation route, and output the modified navigation route.

[0178] In some embodiments, the above apparatus further comprises:

[0179] An information acquisition module, configured to acquire real-time traffic data of a target navigation route and current energy consumption information of a vehicle;

[0180] The route optimization module is configured to optimize the target navigation route according to the real-time traffic data and the current energy consumption information of the vehicle to obtain an optimized navigation route.

[0181] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0182] Fig.12 Schematic diagram of an electronic device 12 provided in an embodiment of the present application. Fig.12 As shown, the electronic device 12 of this embodiment includes: a processor 1201, a memory 1202, and a computer program 1203 stored in the memory 1202 and executable on the processor 1201. When the processor 1201 executes the computer program 1203, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 1201 executes the computer program 1203, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0183] The electronic device 12 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 12 may include, but is not limited to, a processor 1201 and a memory 1202. Those skilled in the art will appreciate that Fig.12The electronic device 12 is merely an example and does not limit the electronic device 12 , and may include more or fewer components than those shown in the figure, or different components.

[0184] Processor 1201 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0185] The memory 1202 may be an internal storage unit of the electronic device 12, for example, a hard disk or memory of the electronic device 12. The memory 1202 may also be an external storage device of the electronic device 12, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 12. The memory 1202 may also include both an internal storage unit of the electronic device 12 and an external storage device. The memory 1202 is used to store computer programs and other programs and data required by the electronic device.

[0186] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0187] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. Computer-readable storage media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0188] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A navigation route planning method based on multimodal interaction, characterized in that: include: Creating a navigation position group, wherein the navigation position group includes a current position of the vehicle and a plurality of different departure positions and a same destination position, or includes the current position of the vehicle and a same departure position and a plurality of different destination positions, or includes the current position of the vehicle and a plurality of different departure positions and a plurality of different destination positions; Determine a first expected travel time period and a first travel preference corresponding to each of the departure locations, and a second expected travel time period and a second travel preference corresponding to each of the destination locations; For each of the departure positions, obtaining first historical traffic flow data of a first driving section from the current position of the vehicle to the departure position during the first expected travel period; For each of the destination locations, obtaining second historical traffic flow data of a second driving section from the current location of the vehicle to the destination location during the second expected travel period; Navigation route planning is performed based on the first expected travel time period, the first travel preference, the first historical traffic flow data, the second expected travel time period, the second travel preference and the second historical traffic flow data, and a target navigation route is output.

2. The method according to claim 1, characterized in that Performing navigation route planning based on the first expected travel time period, the first travel preference, the first historical traffic flow data, the second expected travel time period, the second travel preference, and the second historical traffic flow data, and outputting a target navigation route, including: Predicting a first traffic congestion state of each of the first driving sections during the first expected occurrence time period based on first historical traffic flow data of each of the first driving sections; Predicting a second traffic congestion state of each of the second driving sections during the second expected travel period based on the second historical traffic flow data of each of the second driving sections; Based on the first traffic congestion state and the first travel preference, sorting the departure locations to obtain a first navigation sequence; Based on the second traffic congestion status and the second travel preference, sorting each of the destination locations to obtain a second navigation sequence; Navigation route planning is performed based on the first navigation sequence and the second navigation sequence, and a target navigation route is output.

3. The method according to claim 2, characterized in that Based on the first traffic congestion state and the first travel preference, the departure locations are sorted to obtain a first navigation sequence, including: For each of the departure positions, determining a first shortest driving path from the current position of the vehicle to a first driving segment of the departure position based on the first traffic congestion state and the first travel preference; wherein a first shortest driving time of the first shortest driving path is within the first expected travel time period; The departure locations are sorted based on the first shortest travel time to obtain a first navigation sequence.

4. The method according to claim 3, characterized in that Determining a first shortest travel time of a first travel segment from the current position of the vehicle to the starting position based on the first traffic congestion state and the first travel preference includes: Determine the navigation application scenario; Based on the navigation application scenario, determining a first preference weight of the first travel preference; Performing navigation route planning based on the first preference weight and the first traffic congestion status to obtain an optimal driving trajectory for the first driving section; A first shortest driving time is calculated according to the optimal driving trajectory and the vehicle driving speed.

5. The method according to claim 2, characterized in that: Performing navigation route planning based on the first navigation sequence and the second navigation sequence, and outputting a target navigation route, includes: Acquire a first navigation requirement for the departure location and a second navigation requirement for the destination location; Based on the first navigation requirement and the second navigation requirement, the first navigation sequence and the second navigation sequence are adjusted to obtain a combined navigation sequence; wherein the combined navigation sequence includes the current position of the vehicle, and each of the departure positions and each of the destination positions; A target navigation route is generated according to the combined navigation sequence, and the target navigation route is output.

6. The method according to claim 1 or 2, characterized in that: The method further comprises: Displaying the target navigation route to the user; Obtaining a user's modification instruction for the target navigation route; The target navigation route is modified according to the modification instruction to obtain a modified navigation route, and the modified navigation route is output.

7. The method according to claim 1 or 2, characterized in that: The method further comprises: Obtaining real-time traffic data of the target navigation route and current energy consumption information of the vehicle; The target navigation route is optimized according to the real-time traffic data and the current energy consumption information of the vehicle to obtain an optimized navigation route.

8. A navigation route planning device based on multimodal interaction, characterized in that: include: a creation module configured to create a navigation position group, wherein the navigation position group includes a current position of the vehicle and a plurality of different departure positions and a same destination position, or includes the current position of the vehicle and a same departure position and a plurality of different destination positions, or includes the current position of the vehicle and a plurality of different departure positions and a plurality of different destination positions; A determination module is configured to determine a first expected travel time period and a first travel preference corresponding to each of the departure locations, and a second expected travel time period and a second travel preference corresponding to each of the destination locations; A first acquisition module is configured to acquire, for each of the departure positions, first historical traffic flow data of a first driving section from the current position of the vehicle to the departure position during the first expected travel period; A second acquisition module is configured to acquire, for each of the destination locations, second historical traffic flow data of a second driving section from the current location of the vehicle to the destination location during the second expected travel period; The navigation planning module is configured to plan a navigation route based on the first expected travel time period, the first travel preference, the first historical traffic flow data, the second expected travel time period, the second travel preference and the second historical traffic flow data, and output a target navigation route.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images