Map data processing method, electronic equipment and readable storage medium

Abstract

The invention provides a map data processing method, electronic equipment and a readable storage medium, and relates to the technical field of computers. According to the method provided by the embodiment of the invention, the client can identify the information of each destination, such as the address information and the place sequence, from the multimedia content corresponding to the data input by the user, and then generate the navigation route based on the information of each destination. In the method, the client can automatically identify the information of each destination based on the data input by the user, manual input by the user is not needed, the operation is relatively simple, and the efficiency is relatively high.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a method for processing map data, an electronic device, and a readable storage medium. Background Art

[0002] With the increasingly rich travel life of users, users can find various travel guides online, such as travel guides, cycling guides, hiking guides, food guides, etc. Users can plan travel routes in advance before departure using a client with navigation functions (such as a map application) according to the introductions in these guides, thus facilitating travel.

[0003] However, during the process of users using the client to plan travel routes, users usually need to manually input the information of each location introduced in the guide into the client. For example, when a user wants to travel to location A and finds a video about the travel guide of location A, the user needs to first watch the video to determine each destination to be visited and the order. Then, input the names of these destinations into the client in the order of going to each destination for route planning, which is rather complicated and inefficient. Summary of the Invention

[0004] Some embodiments of this application provide a method for processing map data, an electronic device, and a readable storage medium. The following introduces this application from multiple aspects, and the embodiments and beneficial effects of the following multiple aspects can be referred to each other.

[0005] In a first aspect, an embodiment of this application provides a method for processing map data for a client, the method including: determining first data input by a user; identifying address information of multiple target locations and the location order of the multiple target locations from first multimedia content corresponding to the first data; generating map route data based on the address information and location order of the multiple target locations.

[0006] The client can be the map application mentioned in this application.

[0007] The first data includes at least one of the following: video, audio, picture, text, a link used to indicate at least one of video, picture, text, audio, web page.

[0008] The address information can be the point of interest (POI) information mentioned in this application, including at least one of the following: location name, area, coordinates, category.

[0009] The target location can be the destination mentioned in this application.

[0010] Map route data is data used to generate navigation routes. For example, data such as the coordinates of the starting point, ending point, and waypoints, as well as the track, length, width, direction, etc.

[0011] For example, when a user wants to travel, the user can input a video about travel guides into the map application. Then, the map application can identify the POI information and location order of multiple destinations from the video. Then, the map application can generate map route data based on the POI information and location order of the multiple destinations, and render the navigation route corresponding to the map route data on the map for display.

[0012] For another example, when a user wants to travel, the user can also input the link of a web page about travel guides into the map application. Then, the map application can access the web page corresponding to the link and identify the POI information and location order of multiple destinations from the web page. Then, the map application can generate map route data based on the POI information and location order of the multiple destinations, and render the navigation route corresponding to the map route data on the map for display.

[0013] It can be understood that the client of the present application can directly identify the information of each destination from the multimedia content corresponding to the input data, and then plan a navigation route for the user based on the identified information, without the user inputting the information of each destination, such as the location name. In this way, the user's operation can be reduced and the navigation efficiency can be improved.

[0014] In some embodiments, generating map route data based on the address information and location order of multiple target locations includes: querying the matching points of multiple target locations on the first map from the address information library of the first map of the client based on the address information of the multiple target locations; generating map route data based on the matching points of the multiple target locations on the first map and the location order of the multiple target locations; wherein, the address information library includes the address information of each location on the first map.

[0015] The address information library can be the POI library mentioned in the present application. The matching point can be the POI point mentioned in the present application. The first map can be the map of the map application.

[0016] For example, after the user inputs a video into the map application, the map application can match the location names, regions, and other POI information of each destination with the POI information in the POI information database to determine the corresponding POI points of each destination on the map. Then, based on the corresponding POI points of each destination on the map and the location order, map route data is generated, and the navigation route corresponding to the map route data is rendered and displayed on the map. In this process, the map application queries the corresponding POI points of each destination on the map not only through the location name information, but also through multiple pieces of information. For example, the map application can query the corresponding POI points of each destination on the map through the location name, region, and category of each destination. In this way, not only can the map application avoid identifying multiple similar POI points, but also the accuracy of POI point recognition can be improved, thereby improving the accuracy of route planning.

[0017] In some embodiments, the first multimedia content corresponding to the first data is a video. From the first multimedia content corresponding to the first data, identifying the address information and location order of multiple target locations includes: identifying key frames in the video; extracting the first text in the key frames, and / or extracting the second text corresponding to the audio within the first time interval at the moment of the key frames as the target text; and based on the semantic recognition model, identifying the address information and location order of multiple target locations from the target text.

[0018] The audio within the first time interval can be the audio within a preset time interval (such as 1 second, 2 seconds, 3 seconds, etc., which is not limited thereto) before and / or after the moment of the key frame in the video.

[0019] For example, when the user inputs a video into the client, the client can identify the key frames from the video, then extract the subtitles in the key frames, and / or convert the audio within the first time interval at the moment of the key frames into the corresponding text as the target text, and then input the target text into the trained semantic recognition model to identify the POI information and location order of multiple destinations through the semantic recognition model.

[0020] In some embodiments, identifying the key frames in the video includes: determining the first similarity between two adjacent video frames in the video; corresponding to the first similarity being greater than or equal to a preset first similarity threshold, identifying the image containing text in the two adjacent video frames as the key frame.

[0021] In some embodiments, corresponding to the first data being a link, from the first multimedia content corresponding to the first data, the address information of multiple target locations and the location order of multiple target locations are identified, and it further includes: accessing the multimedia content corresponding to the link; extracting the third text from the multimedia content; based on the semantic recognition model, identifying the address information of multiple target locations and the location order of multiple target locations from the third text.

[0022] For example, when the user inputs the link indicating the web page into the map application, the map application can access the web page corresponding to the link, then extract the corresponding text from the multimedia content such as videos, pictures, texts, and audios in the web page, and input the extracted text into the semantic recognition model, and identify the POI information and location order of multiple destinations from the text through the semantic recognition model.

[0023] For another example, when the user inputs the link indicating the video into the map application, the map application can access and download the video corresponding to the link, extract the text in the video, and then input the text into the semantic recognition model, and identify the POI information and location order of multiple destination points through the semantic recognition model.

[0024] In some embodiments, the method further includes: determining whether the navigation route corresponding to the map route data meets the navigation conditions; corresponding to the navigation route meeting the navigation conditions, displaying the navigation route on the first map of the client; corresponding to the navigation route not meeting the navigation conditions, re-identifying the address information of multiple target locations and the location order from the first multimedia content, and generating new map route data based on the re-identified address information and location order.

[0025] In some embodiments, determining whether the navigation route corresponding to the map route data meets the navigation conditions includes: determining the crossing rate of the navigation route; corresponding to the crossing rate being greater than or equal to the preset crossing rate threshold, determining that the navigation route does not meet the navigation conditions; corresponding to the crossing rate being less than the preset crossing rate threshold, determining that the navigation route meets the navigation conditions; where the crossing rate is the ratio of the number of intersection points on the navigation route to the number of multiple target locations.

[0026] The client can obtain the crossing rate of the navigation route corresponding to the map route data, and then determine whether the crossing rate of the navigation route is greater than or equal to the preset crossing rate threshold (for example, 0.3, 0.4, etc., which is not limited herein). If so, it indicates that the navigation route is not reasonably planned and does not meet the navigation conditions. If not, it indicates that the navigation route is reasonably planned and meets the navigation conditions.

[0027] It can be understood that the intersection point can be a POI point repeatedly passed on the navigation route.

[0028] For example, there are 10 POIs on the navigation route S1: a1 to a10. Among them, a1 is the starting point, a10 is the ending point, and a2 to a9 are waypoints. Among them, waypoint a2, waypoint a5, and waypoint a7 are three intersections on the navigation route S1. The intersection rate of the navigation route S1 can be 3 / 10 = 0.3.

[0029] In some embodiments, determining whether the navigation route corresponding to the map route data meets the navigation conditions further includes: querying the target historical navigation route in the historical navigation database that matches the navigation route; determining the second similarity between the navigation route and the target historical navigation route; corresponding to the second similarity being greater than or equal to the preset second similarity threshold, determining that the navigation route corresponding to the map route data meets the navigation conditions; corresponding to the second similarity being less than the preset second similarity threshold, determining that the navigation route corresponding to the map route data does not meet the navigation conditions.

[0030] The client can query the historical navigation database for the historical navigation route that matches the navigation route, and then use methods based on points (such as the longest common subsequence (LCSS), dynamic time warping (DTW) and other models), methods based on shapes (such as the Frechet distance, Hausdorff distance, etc.), methods based on segments (one way distance, local interpretable model-agnostic predictors (LIP distance), etc.), methods based on specific tasks (such as the trajectory clustering with spatial and temporal alignment (TRACLUS) model, Road Network model, grid model, etc.) and other methods to determine the second similarity between the navigation route and the historical navigation route, and determine whether the second similarity is greater than or equal to the preset second similarity threshold (such as 0.8, 0.85, 0.9, etc., which is not limited). If so, it is determined that the navigation route meets the navigation conditions; if not, it is determined that the navigation route does not meet the navigation conditions.

[0031] It can be understood that the historical navigation database can be a database storing the historical navigation data of all users. The historical navigation data includes but is not limited to the historical data input by the user to the client, the historical map route data generated by the client, etc. In some embodiments, the historical navigation data can be deployed in the cloud.

[0032] In some embodiments, querying for a target historical navigation route that matches the navigation route includes: determining a third similarity between the first multimedia content corresponding to the first data and the historical multimedia content corresponding to each historical input data in the historical navigation database; determining the historical navigation route corresponding to the historical multimedia content with the third similarity greater than or equal to a preset third similarity threshold as the target historical navigation route.

[0033] In some embodiments, determining the third similarity between the first multimedia content corresponding to the first data and the historical multimedia content corresponding to the historical input data in the historical navigation database includes: extracting a fourth text of the first multimedia content corresponding to the first data and a fifth text of the historical multimedia content corresponding to the historical input data; based on a semantic recognition model, determining the similarity between the fourth text and the fifth text as the third similarity.

[0034] For example, the client can extract text 1 from the multimedia content corresponding to the user input data and text 2 from the historical multimedia content corresponding to the historical input data, and then input text 1 and text 2 into the trained semantic recognition model. The semantic recognition model identifies the second similarity between text 1 and text 2 as the second similarity between the multimedia content corresponding to the user input data and the historical multimedia content corresponding to the historical input data. Then, the client can use the historical navigation route corresponding to the historical multimedia content with the second similarity greater than or equal to a preset second similarity threshold (such as 0.8, 0.85, 0.9, etc., which is not limited in this regard) as the historical navigation route that matches this navigation route, that is, the target historical navigation route.

[0035] In a second aspect, an embodiment of the present application provides a map data processing method for a client. The method includes: determining first data input by a user; identifying address information of multiple target locations and the location order of the multiple target locations from the first multimedia content corresponding to the first data; generating map route data based on the address information and location order of the multiple target locations; determining a map data display mode; generating map data based on the map data display mode, and the address information and location order of the multiple target locations.

[0036] The address information includes at least one of the following: location name, region, coordinates, category.

[0037] The first data includes at least one of the following: a file of multimedia content, a link indicating at least one of video, picture, text, audio, and web page.

[0038] The map data includes map route data and map annotation data.

[0039] The map data display mode may include a first mode and a second mode. The first mode may be the navigation mode mentioned in this application, and the second mode may be the exploration mode mentioned in this application.

[0040] The target location may be the destination mentioned in this application.

[0041] After the client receives the file of the multimedia content input by the user, or data such as a link, it can identify the address information and location order of multiple destinations from the multimedia content corresponding to the input data. Then, the client can generate different map data based on the address information and location order of the multiple destinations in different map data display modes to meet different needs of the user.

[0042] For example, in the navigation mode, the client can generate map route data based on the address information and location order of the identified multiple destinations, and then display the navigation route corresponding to the map route data on the map for navigation.

[0043] For another example, in the exploration mode, the client can generate map annotation data for the multiple destinations based on the address information and location order of the identified multiple destinations, so as to mark the POI points corresponding to the multiple destinations on the map through the map annotation data, facilitating the user to understand the position distribution relationship between the current location and each destination on the map.

[0044] That is to say, the client can provide different map services for the user in different map data display modes, such as navigation services and exploration services. Moreover, during the process of the client providing map services for the user, the client can automatically identify the information of each destination from the text or link input by the user without the user manually inputting, which is simple to operate and has high efficiency.

[0045] In some embodiment manners, corresponding to the map data display mode being the first mode, generating map data based on the map data display mode, as well as the address information and location order of multiple target locations, includes: querying the matching points of the multiple target locations on the first map from the address information library of the first map of the client based on the address information of the multiple target locations; generating map route data based on the matching points of the multiple target locations on the first map and the location order of the multiple target locations; wherein, the address information library includes the address information of each location on the first map.

[0046] In some embodiments, corresponding to the map data display mode being the second mode, based on the map data display mode, as well as the address information and location order of multiple target locations, map data is generated, including: querying, from the address information library of the first map of the client, the matching points of the multiple target locations on the first map based on the address information of the multiple target locations; and generating map annotation data of the multiple target locations based on the matching points of the multiple target locations on the first map and the location order of the multiple target locations.

[0047] The map annotation data includes at least one of the following: a link for indicating the multimedia content corresponding to the first data; the location order of the multiple target locations; at least one address data among the location names, regions, coordinates, and categories of the multiple target locations; and the positioning marker data of the multiple target locations.

[0048] For example, Video 1 sequentially introduces Location 1, Location 2, Location 3, Location 4, and Location 5. After the user imports Video 1 into the map application, the map application can display the positioning icons and location names of Location 1, Location 2, Location 3, Location 4, and Location 5 on the map respectively. Among them, the serial number indicating the order in which each destination appears in the video is included in each positioning icon. For example, the serial number 1 is included in the positioning icon of Location 1, indicating that Location 1 appears first in Video 1.

[0049] In addition, the map application can also display the link of Video 1 at each positioning icon. When the user clicks the link corresponding to each positioning icon, the map application can display Video 1 and jump to the playing position of the destination in Video 1.

[0050] For example, the client can also display the distance between the POI points corresponding to each destination and the user's current location on the map, so that the user can understand the positional distance relationship between the current location and each destination on the map, providing a basis for the user to select a destination.

[0051] For example, the client can also display relevant introduction information of the POI points corresponding to each destination on the map, such as evaluation information, description information, etc., to facilitate the user to understand the information of each destination on the map.

[0052] For example, the client can also assign the same value to the location order of each destination. In this case, the client may not display the order in which each destination appears in the multimedia content corresponding to the input data. For example, the map application does not display the corresponding serial numbers in the positioning icons of each destination.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory for storing instructions executed by one or more processors of the electronic device; a processor, when the processor executes the instructions in the memory, enabling the electronic device to execute the method described in any embodiment of the first aspect or the second aspect of the present application. The beneficial effects achievable by the third aspect can be referred to the beneficial effects of the method provided in any embodiment of the first aspect or the second aspect, and will not be elaborated herein.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed on a computer, the computer can be enabled to execute the method described in any embodiment of the first aspect or the second aspect. The beneficial effects achievable by the fourth aspect can be referred to the beneficial effects of the method provided in any embodiment of the first aspect or the second aspect, and will not be elaborated herein.

[0055] In a fifth aspect, an embodiment of the present application provides a chip. The chip system includes a processing circuit and a storage medium, and computer program code is stored in the storage medium; when the computer program code is executed by the processing circuit, the method described in any embodiment of the first aspect or the second aspect is implemented. The beneficial effects achievable by the fifth aspect can be referred to the beneficial effects of the method provided in any embodiment of the first aspect or the second aspect, and will not be elaborated herein.

[0056] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed, the computer is enabled to execute the method described in any embodiment of the first aspect or the second aspect. The beneficial effects achievable by the sixth aspect can be referred to the beneficial effects of the method provided in any embodiment of the first aspect or the second aspect, and will not be elaborated herein. Description of the Drawings

[0057] Figures 1A - 1B is an example diagram of the application scenario of the present application;

[0058] Figure 2 is a flow example diagram of a method for processing map data in a navigation mode provided by an embodiment of the present application;

[0059] Figure 3 is an example diagram of a POI point provided by an embodiment of the present application;

[0060] Figure 4 is an example diagram of a navigation route provided by an embodiment of the present application;

[0061] Figure 5 is an example diagram of a client parsing input resources and displaying a navigation route provided by an embodiment of the present application.

[0062] Figure 6AAn example diagram of a navigation route provided by an embodiment of the present application;

[0063] Figure 6B Another example diagram of a navigation route provided by an embodiment of the present application;

[0064] Figure 7 A flow example diagram of a map data processing method under another navigation mode provided by an embodiment of the present application;

[0065] Figure 8 An example diagram of importing a video in a navigation mode provided by an embodiment of the present application;

[0066] Figure 9 A schematic diagram of identifying key frames provided by an embodiment of the present application;

[0067] Figure 10 An example diagram of text extraction from audio near a key frame provided by an embodiment of the present application;

[0068] Figure 11 An example diagram of planning a navigation route provided by an embodiment of the present application;

[0069] Figure 12 A flow example diagram of a map data processing method under another navigation mode provided by an embodiment of the present application;

[0070] Figure 13 An example diagram of a web page provided by an embodiment of the present application;

[0071] Figure 14 An example diagram of link import provided by an embodiment of the present application;

[0072] Figure 15 A flow example diagram of a map data processing method in an exploration mode provided by an embodiment of the present application;

[0073] Figure 16 An example diagram of importing a video in an exploration mode provided by an embodiment of the present application;

[0074] Figure 17 An example diagram of displaying map annotation data in an exploration mode provided by an embodiment of the present application;

[0075] Figure 18 An example diagram of link jump provided by an embodiment of the present application;

[0076] Figure 19 An example diagram of a system architecture provided by an embodiment of the present application;

[0077] Figure 20 A block diagram of an electronic device provided by an embodiment of the present application is shown. Detailed implementation manners

[0078] Embodiments of the present application are used to provide a method for processing map data. The method for processing map data of the present application will be introduced below in combination with specific embodiments.

[0079] The client in the present application may include an electronic device or an application. The electronic device includes, but is not limited to, a mobile phone with a navigation function, a tablet, a wearable device (such as a smart watch, a smart bracelet, etc.), a vehicle-mounted device (such as a car stereo, etc.), and the application includes, but is not limited to, a map application with a navigation function, a voice assistant application, a location service application, etc. Below, a tablet is used as an example of an electronic device, and a map application on the tablet is used as an example of a client to illustrate the technical solution of the present application.

[0080] Figure 1A and Figure 1B is an exemplary application scenario of the present application.

[0081] Refer to Figure 1A , a map application with a navigation function is installed on the electronic device 100. The user can use the map application on the electronic device 100 to plan a travel route and navigate, thus facilitating travel. For example, when the user wants to travel, the user can first query travel guides, and then plan a corresponding travel route on the map application of the electronic device 100 according to the instructions in the travel guides.

[0082] For example, as Figure 1A shown, when the travel guide queried by the user indicates that the user can drive from location A, pass through location B and location C in xxx district, and finally reach location D, the user can open the route planning interface M1 of the map application on the electronic device 100, then click on the vehicle icon 11 on the route planning interface M1 to plan a driving navigation route, and then input the names of location A, location B, location C, and location D in the input boxes H1, H2, H3, and H4 of the route planning interface M1 in sequence to plan the navigation route. Since the user inputs location names instead of detailed addresses, after the user inputs location names, the map application may identify multiple relevant locations. At this time, the user still needs to select the correct location mentioned in the guide from the multiple relevant locations identified by the map application.

[0083] For example, as Figure 1A shown, when the user inputs the name of location C in the input box H3, the map application identifies the following locations: location C in xxx district, location C (subway station), location C (north gate), location C scenic belt, location C parking lot, location C cruise ship terminal. When the user selects "location C in xxx district" from these locations, the name of "location C in xxx district" can be input into the input box H3.

[0084] As Figure 1BAs shown, after the user enters the names of various locations on the map application of the electronic device 100, when the user clicks the route planning button 12 on the route planning interface M1, the map application can generate three navigation routes: Route 1 (mileage 10.1 kilometers), Route 2 (mileage 10.9 kilometers), and Route 3 (mileage 13 kilometers), and by default display Route 1 with the shortest mileage on the map. Then, the map application can use Route 1 to navigate for the user, thus facilitating the user's travel. However, during this process, the user needs to manually enter the information of each location, and the operation is rather cumbersome and the efficiency is low.

[0085] To solve the above technical problems, the present application proposes a map data processing method. In the method of the embodiments of the present application, the client can identify the information of each destination (as the target location) from the multimedia content corresponding to the user input data, such as the address information and the location order, and then generate map route data based on the information of each destination, and then display the navigation route corresponding to the map route data on the map. In this method, the client can automatically identify the information of each destination based on the user input data, without the need for the user to manually enter, and the operation is relatively simple and the efficiency is high.

[0086] It can be understood that the multimedia content includes but is not limited to video, picture, text, audio and other content.

[0087] It can be understood that the user input data may include at least one of the following: files of multimedia content such as video, picture, text, audio, etc., and links (such as uniform resource locator (URL)) for indicating at least one of video, picture, text, audio, and web page.

[0088] It should be noted that the multimedia content corresponding to the user input data is the content obtained based on the user input data.

[0089] For example, when the user input data is a link indicating a web page, the client can access the web page corresponding to the link and extract the video, picture, text, audio, etc. in the web page as the multimedia content corresponding to the user input data. Or, when the user input data is a link indicating a video, the client can access and download the video corresponding to the link as the multimedia content corresponding to the user input data.

[0090] Also, for example, when the user input data is a video, the client can perform parsing processing on the input video, such as decoding, transcoding, cropping, etc., to obtain the parsed video as the multimedia content corresponding to the user input data.

[0091] It can be understood that the location information (or "POI information") includes, but is not limited to, name, coordinates (such as longitude and latitude, etc.), area, category (such as categories like hotel, scenic spot, parking lot, etc.), description information, etc.

[0092] It should be noted that the map mentioned in this application can be an electronic map.

[0093] It should be noted that the client in the embodiments of this application has different map data display modes, such as the navigation mode and the exploration mode. In different map data display modes, the client can display different map data on the map, such as map route data, map annotation data, to meet the different scenario needs of users.

[0094] The navigation mode is a mode for planning and displaying navigation routes. In the navigation mode, the client can display the navigation route corresponding to the map route data on the map to meet the navigation needs of users and facilitate users' travel.

[0095] The exploration mode is a mode for marking the locations of interest to users on the map. In the exploration mode, the client can display the map annotation data of each location of interest to users, such as the link to relevant guides, location names, positioning icons, etc., to facilitate users to view the locations and relevant information of each location on the map and provide a basis for users' travel choices.

[0096] Next, the navigation mode of the client will be introduced first in combination with specific embodiments.

[0097] Figure 2 It is a flowchart example of the map data processing method in the navigation mode provided for the embodiments of this application.

[0098] Refer to Figure 2 , this method includes the following steps:

[0099] S101: The client obtains the data input by the user.

[0100] In some embodiments, the user can switch the current map data display mode of the client to the navigation mode, and then input files such as videos, pictures, texts, audios, links, or links indicating videos, pictures, texts, audios, or web pages into the client in the navigation mode, so that the client plans corresponding navigation routes for the user based on the data input by the user and conducts navigation.

[0101] S102: The client identifies the POI information and location order of multiple destinations from the multimedia content corresponding to the data input by the user.

[0102] It can be understood that in the navigation mode, the location order can be the arrival order of each destination.

[0103] In some embodiments, after the client receives the data input by the user, the client can perform text extraction processing on the multimedia content corresponding to the data to obtain the corresponding text, and then input the text into the trained semantic recognition model to identify the POI information of multiple destinations and the arrival order from the text through the semantic recognition model.

[0104] For example, when the user inputs a video into the client, the client can extract the subtitles from the video image and perform text conversion processing on the video audio to obtain the corresponding text, and then input the text into the trained semantic model to identify the POI information of multiple destinations and the arrival order in the text through the trained semantic recognition model.

[0105] For another example, when the user inputs audio into the client, the client can convert the input audio of the user into the corresponding text, and then input the text into the trained semantic recognition model to identify the POI information of multiple destinations and the arrival order in the text through the trained semantic recognition model.

[0106] For another example, when the user inputs a picture into the client, the client can extract the text in the picture, and then input the text into the trained semantic recognition model to identify the POI information of multiple destinations and the arrival order in the text through the trained semantic recognition model.

[0107] For another example, when the user inputs text into the client, the client can directly input the text into the trained semantic recognition model to identify the POI information of multiple destinations and the arrival order in the text through the trained semantic recognition model.

[0108] For another example, when the user inputs a link into the client, the client can first access the video, picture, text, audio or web page corresponding to the link, and then extract the corresponding text from the accessed video, picture, text, audio or web page, and input the text into the trained semantic recognition model to identify the POI information of multiple destinations and the arrival order in the text through the trained semantic recognition model.

[0109] It can be understood that the semantic recognition model can be an attribute entity relationship (AER), entity relationship (ER), entity relationship attribute design (ERD), bidirectional encoder representations from transformers (BERT), global vectors for word representation (GloVe), embeddings from language models (Elmo), generative pre-trained transformer (GPT), generalized autoregressive pretraining for language understanding (xlnet), etc., which are models with semantic recognition capabilities.

[0110] The training process of the semantic recognition model will be described below.

[0111] In some embodiments, a large number of texts containing travel, business trips, cycling, hiking, food recommendations, etc. can be collected on the network as training samples, and then these training samples are input into the semantic recognition model. The semantic recognition model performs semantic recognition processing on the input training samples and outputs prediction results. Then, based on the difference between the prediction result and the actual result, a loss value is calculated, and then the parameters of the semantic recognition model are adjusted based on the loss value.

[0112] For example, when the text "On the first day, start from location A and go to location B to visit scenic spots. In the morning of the second day, go hiking at location C in xxx district, and in the afternoon, go boating at location D by bypassing the congested location E" is used as a training sample and input into the semantic recognition model, if the prediction result output by the semantic recognition model is "location A - location B - location C - location E - location D", which does not match the actual result "location A - location B - xxx district location C - location D", a loss value can be calculated based on the difference between the prediction result and the actual result, and then the parameters of the semantic recognition model can be adjusted based on the loss value. In this way, the semantic recognition model is trained multiple times until the semantic recognition model reaches the training end condition, and then the training ends.

[0113] In some embodiments, the training end condition may include at least one of the following: the number of training times reaches a preset number; the model achieves the expected effect; the loss function of the model converges.

[0114] It can be understood that after the semantic recognition model is trained in the above manner, the trained semantic recognition model can identify the POI information of multiple destinations that the user needs to go to and the arrival order from the input text.

[0115] For example, after inputting the text with specific content "Starting from location A on the first day to go to location B to visit scenic spots. In the morning of the second day, go to location C in xxx district to climb mountains, and in the afternoon, bypass the congested location E to go to location D to row a boat" into the trained semantic recognition model, the semantic recognition model can identify from the input text that the user needs to go to "location A", "location B", "location C in xxx district" and "location D" in sequence.

[0116] S103: The client generates map route data based on the POI information of the multiple destinations and the location order.

[0117] In some embodiments, after the client identifies the POI information of multiple destinations, the client can query the matching points (referred to as "POI points" in this article) corresponding to the multiple destinations on the map from the POI library (as an address information library) based on the POI information of the multiple destinations, and then generate corresponding map route data based on the POI points corresponding to the multiple destinations on the map and the location order.

[0118] It can be understood that the POI library is a database for storing the POI information of all POI points on the map. The POI library can be deployed locally or in the cloud, and this is not limited.

[0119] It can be understood that the map route data is data for generating a navigation route, such as the coordinates of the starting point, ending point, and waypoints, trajectory, length, width, direction, and other data.

[0120] It can be understood that the POI information includes but is not limited to information such as location name, coordinates, region, type, etc. In the embodiments of the present application, the client can use the location name, coordinates, region, and category as indexes to query the POI points corresponding to each destination on the map from the POI information library to improve the accuracy of POI point matching. For example, after the map application of the electronic device 100 identifies the destination with the region name "xxx district" and the location name "location C", the map application can query the POI point corresponding to the destination on the map from the POI library with the location name "location C" and the region name "xxx district". As Figure 3As shown, the index of the POI point queried by the map application is "xxxxx", the area is "xxx District", the location name is "Location C", the type is "Scenic Spot", the coordinates are "xxxx", the location type is: POI point, and the confidence level is 5.

[0121] S104: The client determines whether the navigation route corresponding to the map route data meets the navigation conditions.

[0122] If so, step S105 is executed. If not, step S102 is executed.

[0123] In some embodiments, the client can obtain the crossing rate of the navigation route corresponding to the map route data, and then determine whether the crossing rate of the navigation route is greater than or equal to a preset crossing rate threshold (for example, 0.3, 0.4, etc., which is not limited herein). If so, it means that the navigation route planning is unreasonable and does not meet the navigation conditions. If not, it means that the navigation route planning is reasonable and meets the navigation conditions.

[0124] In some embodiments, the crossing rate of the navigation route can be the ratio of the number of crossing points on the navigation route to the number of POI points. It can be understood that the crossing points can be the POI points repeatedly passed on the navigation route.

[0125] For example, as Figure 4 shown, there are 10 POI points on the navigation route S1: a1 to a10. Among them, a1 is the starting point, a10 is the ending point, and a2 to a9 are waypoints. Among them, waypoint a2, waypoint a5, and waypoint a7 are 3 crossing points on the navigation route S1. The crossing rate of the navigation route S1 can be 3 / 10 = 0.3.

[0126] In other embodiments, the client can query the historical navigation database for the historical navigation route matching the navigation route, and then use methods based on points (such as LCSS, DTW and other models), methods based on shapes (such as Frechet model, Hausdorff model, etc.), methods based on segments (such as one way distance, LIPdistance and other models), methods based on specific tasks (such as TRACLUS model, road network, grid and other models), etc., to determine the similarity between the navigation route and the historical navigation route, and determine whether the similarity is greater than or equal to a preset similarity threshold (for example, 0.8, 0.85, 0.9, etc., which is not limited herein). If so, it is determined that the navigation route meets the navigation conditions. If not, it is determined that the navigation route does not meet the navigation conditions.

[0127] It can be understood that the historical navigation database can be a database storing the historical navigation data of all users. The historical navigation data includes, but is not limited to, the historical data input by the user to the client, the historical map route data generated by the client, etc. In some embodiments, the historical navigation data can be deployed in the cloud.

[0128] In some embodiments, the client can extract text 1 from the multimedia content corresponding to the user input data, and extract text 2 from the historical multimedia content corresponding to the historical input data, and then input text 1 and text 2 into the trained semantic recognition model. The semantic recognition model is used to identify the similarity between text 1 and text 2, which is used as the similarity between the multimedia content corresponding to the user input data and the historical multimedia content corresponding to the historical input data. Then, the client can use the historical navigation route corresponding to the historical multimedia content whose similarity is greater than or equal to a preset similarity threshold (such as 0.8, 0.85, 0.9, etc., which is not limited herein) as the historical navigation route matching the navigation route.

[0129] S105: The client displays the navigation route on the map.

[0130] After the client determines that the navigation route meets the navigation conditions, the client can render the navigation route on the map for display.

[0131] In the embodiments of the present application, the client can directly identify the information of each destination based on files such as videos, pictures, texts, and audios input by the user, or links indicating videos, pictures, texts, audios, and web pages, and plan corresponding navigation routes based on the information of each destination, without manually inputting the information of each destination in the client, which is relatively simple to operate and has high efficiency. Exemplarily, as Figure 5 shown, after inputting external resources (such as files such as videos, pictures, texts, and audios, or links indicating videos, pictures, texts, audios, and web pages) into the map application on the electronic device 100, the map application can automatically parse the input resources, generate corresponding navigation routes, and then render the navigation routes on the map for display.

[0132] Moreover, the client has a navigation route correction function. When the navigation route does not meet the navigation conditions, the client can re-plan the navigation route, thereby avoiding unreasonable navigation routes.

[0133] For example, after the user inputs a video indicating that the user departs from location A, passes through location B and location C in xxx district, and finally arrives at location D into the map application of the electronic device 100, the map application misidentifies location C in xxx district as location F in xxx district, resulting in an incorrect route planned by the map application. In the case of misidentification by the map application, asFigure 6A As shown, the route planned by the map application starts from location A, passes through location B and location F in xxx district, and finally reaches location D. In this case, the map application does not display Figure 6A the incorrect navigation route in [the figure], but instead re-identifies the input video to correct the incorrect navigation route. After the map application re-identifies the input video, it can correct the mis-identified location F in xxx district to location C in xxx district, and then display the corrected navigation route on the map. As Figure 6B shown, the corrected navigation route starts from location A, passes through location B and location C in xxx district, and finally reaches location D.

[0134] It can be understood that in some embodiments, after the client generates map route data, it can also directly display the navigation route corresponding to the map route data on the map.

[0135] For ease of understanding, the following takes the client as a map application and the data input by the user to the client as a video as an example to introduce the method for processing map data in the navigation mode of the client.

[0136] Figure 7 This is a flowchart example of another method for processing map data in the navigation mode provided by the embodiments of the present application.

[0137] Referring to Figure 7 , this method includes the following steps:

[0138] S201: The map application obtains the video input by the user.

[0139] Taking the travel scenario as an example, when the user wants to travel, the user can first query videos about travel guides, then download the queried videos to the local, and then import the videos downloaded to the local into the map application in the navigation mode. The map application plans the corresponding navigation route based on the imported video and conducts navigation.

[0140] For example, as Figure 8As shown in the figure, when the user wants to import a local video about travel guides into the map application on the electronic device 100, the user can open the map application on the electronic device 100 and enter the home interface L1 of the map application. Then, the user can click the navigation mode button Q1 on the home interface L1 to control the map application to enter the navigation mode. In the navigation mode, the user can click the import resource button 13 on the home interface L1 of the map application to open the import resource interface L2 of the map application. After the import resource interface L2 is opened, the user can click the local file import button 14 on the import resource interface L2 to open the local file import interface L3. Then, the user can click the import local file button 15 to open the media library of the electronic device 100 and enter the preview interface L4 of the media library. The preview interface L4 of the media library displays the files of the multimedia content stored in the media library. The user can select the file to be imported from the preview interface L4 of the media library. When the user selects the video 2 on the preview interface L4 and clicks the OK button 16, the electronic device 100 can import the video 2 into the map application, so that the map application obtains the imported video 2.

[0141] It should be noted that the methods of importing audio and pictures can be the same as the method of importing video, which will not be elaborated here.

[0142] S202: The map application identifies the key frames in the video.

[0143] In some embodiments, after the user imports the video into the map application, the map application can use algorithms such as histogram comparison algorithm, structural similarity algorithm, feature similarity algorithm, and hash algorithm to obtain the similarity between two adjacent video frames in the video. Then, the map application identifies the video frame containing text (subtitle) as the key frame from the adjacent two video frames whose similarity is greater than or equal to a preset similarity threshold (for example, 0.8, 0.85, 0.9, etc., which is not limited herein). It can be understood that a video frame is a frame image in the video.

[0144] For example, as Figure 9 shown, the video 2 imported by the user into the map application includes n frame images. Among them, the scene in the m1-th frame image of the video 2 is similar to that in the m1-1-th frame image, and its similarity is greater than the preset similarity threshold, and the m1-th frame image includes the subtitle "Location A on the first day". In this case, the m1-th frame image in the video can be used as the key frame. Where m1 is less than or equal to n.

[0145] S203: The map application performs text extraction processing on the key frames and the audio near the key frames to obtain the corresponding text.

[0146] In some embodiments, after the map application identifies a key frame in the video, text recognition technologies such as optical character recognition (OCR) can be used to extract the text in the key frame and perform text conversion processing on the audio near the key frame to obtain the corresponding text.

[0147] It can be understood that the audio near the key frame can be the audio within a preset time interval (such as 1 second, 2 seconds, 3 seconds, etc., which is not limited herein) before and / or after the moment when the key frame is located.

[0148] For example, as Figure 10 shown, the duration of video 2 imported by the user into the client is 2 minutes. The key frame recognized by the client from the video is the m-th frame image, and the moment when the m-th frame image is located in the video is 1 minute and 20 seconds. The client can extract the audio 2 seconds before and after the moment when the key frame is located, that is, extract the audio between 1 minute and 18 seconds and 1 minute and 22 seconds in the video, and then convert the extracted audio into the corresponding text.

[0149] It can be understood that by identifying the key frames in the input video and extracting the text of the key frames and the audio near the key frames, not only can the information of each destination in the video be effectively obtained, but also the data volume can be reduced, thereby improving the efficiency of data processing and the efficiency of route planning.

[0150] Exemplarily, after the user inputs video 2 into the map application, the map application extracts the text shown in Table 1 from the key frames and the audio near the key frames in video 2:

[0151] Table 1

[0152] Serial number Frame ID Text 1 m1 Description of location A 2 m2 Description of location B 3 m3 Description of location C 4 m4 Description of location D …… …… ……

[0153] As shown in Table 1, for the key frame with serial number 1 and frame ID m1, the descriptive text of key location A can be extracted from this key frame and the audio near this key frame. For the key frame with serial number 2 and frame ID m2, the descriptive text of key location B can be extracted from this key frame and the audio near this key frame. For the key frame with serial number 3 and frame ID m3, the descriptive text of key location C can be extracted from this key frame and the audio near this key frame. For the key frame with serial number 4 and frame ID m4, the descriptive text of key location D can be extracted from this key frame and the audio near this key frame. The serial number is the serial number assigned to each key frame according to the order of the key frames in the video.

[0154] It can be understood that by identifying key frames in the video to extract text from the video, the map application can reduce the amount of data, thereby reducing the pressure of data processing, increasing the speed of subsequent text recognition, and thus increasing the speed of route planning.

[0155] It is understandable that in other embodiments, after the user imports the video into the map application, the map application can also extract the text of each frame in the video, and perform text conversion on the audio of the entire video to obtain the corresponding text to avoid missing information.

[0156] S204: The map application identifies POI information and arrival order of multiple destinations from the acquired text.

[0157] In some embodiments, after the map application extracts the text from the input video, the extracted text can be input into a trained semantic recognition model, and the POI information and arrival order of multiple destinations can be identified through the semantic recognition model.

[0158] For example, after the map application extracts text from video 2 imported by the user: "On the first day, depart from Location A to Location B to visit scenic spots. On the second day, go to Location C in District xxx for mountain climbing in the morning, and in the afternoon, avoid the congested Location E to go to Location D for boating", the map application can input the text into the semantic recognition model, and recognize through the semantic recognition model that the user needs to go to "Location A", "Location B", "Location C in District xxx" and "Location D" in sequence.

[0159] S205: The map application queries the POI points corresponding to the multiple destinations on the map from the POI library.

[0160] After the map application identifies POI information of multiple destinations from the video imported by the user, the map application may match the POI information of the multiple destinations with the POI information in the POI library to determine the POI points corresponding to the multiple destinations on the map application.

[0161] It is understood that POI information may include location name, coordinates (such as longitude and latitude), area, category (such as hotel, gas station, etc.), etc. The map application may query the POI point corresponding to the destination on the map from the POI library based on multiple POI information of the destination to improve the accuracy of POI point recognition.

[0162] For example, when the map application recognizes that the location name of the destination that the user needs to go to is "Location C" and the area it belongs to is "District xxx", the map application can first query the POI points in the "District xxx" area on the map from the POI library, and then query the POI points with the location name "Location C" from the POI points in the area as the POI point corresponding to the destination.

[0163] S206: The map application generates map route data based on the POI information and location order of the multiple destinations.

[0164] S207: The map application determines whether the navigation route corresponding to the map route data meets the navigation conditions.

[0165] If not, perform step S204. If so, perform step S208.

[0166] It should be noted that steps S206 - S207 are substantially the same as the above steps S104 - S105 and will not be elaborated here.

[0167] S208: The map application renders the navigation route onto the map for display.

[0168] After the map application determines that the planned navigation route meets the navigation conditions, the map application can render the navigation route onto the map for display.

[0169] Exemplarily, the travel guide in Video 2 instructs the user to drive from Location A, pass through Location B and Location C in xxx District, and finally reach Location D. As Figure 11 shown, after the user imports Video 2 into the map application of the electronic device 100, the map application can plan Route 1 according to the imported Video 2, then render Route 1 onto the map for display, and can display the information on Route 1, such as the starting point, ending point, passing points, etc. Then, the user can start from Location A according to the instructions of Route 1, pass through Location B and Location C in xxx District in sequence, and finally reach Location D.

[0170] In the embodiments of the present application, the map application can directly identify the information of each destination according to the video input by the user, and then plan a navigation route for the user based on the identified information, without the user manually inputting the information of each destination. In this process, the user only needs to import the video, with simple operation and high efficiency.

[0171] The following describes a map data processing method according to another embodiment of the present application. Different from the above embodiment, instead of a file, the initial input is a link indicating a web page.

[0172] Figure 12 This is a flowchart example of a map data processing method under another navigation mode provided by the embodiments of the present application.

[0173] Refer to Figure 12 , this method includes the following steps:

[0174] S301: The map application obtains the link input by the user.

[0175] Taking the travel scenario as an example, when a user wants to travel, the user can query web pages about travel guides on the Internet and then input the link of the web page into the map application, so that the map application can plan a travel route for the user according to the input link and conduct navigation.

[0176] For example, as Figure 13 shown, after the user queries the web page P1 about travel guides on the browser application of the electronic device 100, the user can copy the link of the web page P1 to the clipboard and then paste the link of the web page P1 into the map application for import.

[0177] For instance, as Figure 14 shown, after the user copies the link of the web page P1, the user can open the map application on the electronic device 100 and enter the main interface L1 of the map application. Then, the user can click the import resource button 13 on the main interface L1 of the map application to open the import resource interface L2 of the map application. After the import resource interface L2 is opened, the user can click the link import button 17 on the import resource interface L2 to open the link import interface L4. The link import interface L4 includes an input box 18. The user can paste the link of the web page P1 copied to the clipboard into the input box 18 and then click the confirmation button 19 on the link import interface L4 to input the link in the input box 18 into the map application.

[0178] It can be understood that the way for the client to import text can be the same as the way to import a link.

[0179] S302: The map application accesses the web page corresponding to the link.

[0180] After the user inputs the link into the map application, the map application can access the web page corresponding to the link.

[0181] For example, as Figure 13 shown, after the user imports the link of the web page P1 into the map application, the map application can access the web page P1 through the link to identify information about each destination from the multimedia content carried on the web page P1, such as videos, pictures, texts, audios, etc. Hereinafter, taking obtaining navigation information from a travel guide article published on the web page as an example.

[0182] S303: The map application grabs the corresponding text from the accessed web page.

[0183] After the map application accesses the web page indicated by the link input by the user, it can grab the text in the web page.

[0184] It can be understood that the text extracted by the map application from the web page can be the text displayed on the web page, or the text extracted from multimedia content such as videos, pictures, and audio in the web page.

[0185] In some embodiments, the map application can capture the hyper text markup language (HTML) document of the web page, then parse the HTML document through the document object model (DOM) to obtain a DOM tree, and then extract the node information of the HTML elements in the web page by traversing the DOM tree. The node information of the HTML elements includes, but is not limited to, the information of elements such as titles, paragraphs, lists, hyperlinks, images, controls, and buttons in the web page. Then, the map application can perform text extraction processing on the extracted node information of the HTML elements to obtain the corresponding text.

[0186] It should be noted that in the scenario where the link indicated by the user input is multimedia content such as a video, picture, text, or audio, the map application can access the multimedia content indicated by the link, and then extract the corresponding text from the multimedia content indicated by the link.

[0187] S304: The map application identifies the POI information of multiple destinations and the arrival order from the obtained text.

[0188] S305: The map application queries the POI points corresponding to the multiple destinations on the map from the POI library.

[0189] S306: The map application generates map route data according to the arrival order of the POI points.

[0190] S307: The map application determines whether the navigation route corresponding to the map route data meets the navigation conditions.

[0191] If not, execute step S304. If so, execute step S308.

[0192] It should be noted that steps S306 - S307 are substantially the same as steps S104 - S105 above, and will not be elaborated here.

[0193] S308: The map application renders the navigation route onto the map for display.

[0194] After the map application determines that the navigation route meets the conditions, the map application can render the navigation route onto the map for display.

[0195] Exemplarily, such as Figure 14As shown, after the user imports the link of web page P1 into the map application, since the travel guide in web page P1 indicates that the user needs to drive from location A, pass through location B and location C in xxx district, and finally reach location D, the map application can plan route 1 that meets the navigation conditions: starting from location A, passing through location B and location C in xxx district in sequence, and finally reaching location D. Then the map application can render route 1 on the map for display and navigate for the user according to route 1.

[0196] In the embodiments of the present application, the user can directly input the link of the web page into the map application, and the map application automatically identifies the information of each destination from the web page indicated by the link, without manually inputting the information of each destination into the map application, with relatively simple operation and high navigation efficiency.

[0197] The exploration mode of the client is introduced below in combination with specific embodiments. The exploration mode is a mode of marking locations of interest to the user on the map. In the exploration mode, the client can mark locations of interest to the user, so that the user can understand the relationship between the current location and the distribution of these points of interest on the map. Moreover, the user can freely select and view the relevant information of each point of interest.

[0198] Figure 15 It is a flowchart example of the map data processing method in the exploration mode provided by the embodiments of the present application.

[0199] As Figure 15 shown, the method includes the following steps:

[0200] S401: The client obtains the data input by the user.

[0201] In some embodiments, the user can switch the current mode of the client to the exploration mode, and then input data such as videos, pictures, texts, audios, and links into the client in the exploration mode, so that the client obtains the data input by the user.

[0202] For ease of understanding, the following takes the food exploration scenario as an example for illustration.

[0203] For example, as Figure 16, when the user queries a video 1 about a food guide and wants to go to one of the food locations introduced in video 1, the user can open the map application on the electronic device 100 and enter the first interface L1 of the map application. Then, the user can click the exploration mode button Q2 on the first interface L1 to switch the current map data display mode of the map application to the exploration mode. Then, the user can click the import resource button 13 on the first interface L1 of the map application to open the import resource interface L2 of the map application. After the import resource interface L2 is opened, the user can click the local file import button 14 on the import resource interface L2 to open the local file import interface L3. Then, the user can click the import local file button 15 to open the media library of the electronic device 100 and enter the preview interface L4 of the media library. The preview interface L4 of the media library displays the files of the multimedia content stored in the media library. The user can select the file to be imported from the preview interface L4 of the media library. When the user selects video 1 about the food guide and clicks the OK button 16 on the preview interface L4, the electronic device 100 can import video 1 into the map application, so that the map application obtains the imported video 1.

[0204] S402: The client identifies the POI information and location order of multiple destinations from the multimedia content corresponding to the data input by the user.

[0205] Step S402 is substantially the same as the above step S102 and will not be elaborated here.

[0206] It should be noted that in the exploration mode, the location order of multiple destinations can be the order in which the multiple destinations appear in the multimedia content corresponding to the data input by the user.

[0207] S403: The client generates and displays the map annotation data of the multiple destinations based on the POI information and location order of the multiple destinations.

[0208] In some embodiments, after the client identifies the POI information of multiple destinations, the client can query the POI points corresponding to the multiple destinations on the map from the POI library based on the POI information of the multiple destinations, and then generate and display the map annotation data of the POI points corresponding to the multiple destinations on the map based on the POI points corresponding to the multiple destinations on the map and the location order.

[0209] For example, as Figure 16 shown, after the user inputs video 1 into the map application, the map application can mark the POI points corresponding to each destination mentioned in video 1 on the map.

[0210] It can be understood that the map annotation data includes but is not limited to links for indicating the corresponding media content of the user input data, location order, address data (such as location name, area, coordinates, category, etc.), positioning marker data (such as positioning icons, highlighted display data, etc.), and the like.

[0211] Exemplarily, as Figure 17 shown, Video 1 successively introduces Location 1, Location 2, Location 3, Location 4, and Location 5. After the user imports Video 1 into the map application on the electronic device 100, the map application can respectively display the positioning icons and location names of Location 1, Location 2, Location 3, Location 4, and Location 5 on the map. Among them, each positioning icon contains the serial number of the order in which each destination appears in the video. For example, the positioning icon of Location 1 includes the serial number 1, indicating that Location 1 appears first in Video 1.

[0212] In addition, the map application can also display the link of Video 1 at each positioning icon. When the user clicks on the link corresponding to each positioning icon, the map application can display Video 1 and jump to the playing position of the destination in Video 1.

[0213] For example, as Figure 18 shown, when the user clicks on the link displayed above the positioning icon of Location 4, the map application can pop up a video window K1 and display Video 1 in the video window K1. The current playing position of Video 1 can be the playing position of Location 4.

[0214] In some other embodiments, the client can also display the distance between the POI points corresponding to each destination and the user's current location on the map, so that the user can understand the positional distance relationship between the current location and each destination on the map, providing a basis for the user to select a destination.

[0215] In some other embodiments, the client can also display relevant introduction information of the POI points corresponding to each destination on the map, such as evaluation information, description information, etc., to facilitate the user to understand the information of each destination on the map.

[0216] In some other embodiments, the client can also assign the same value to the location order of each destination. In this case, the client can not display the sequence in which each destination appears in the multimedia content corresponding to the input data. For example, the map application does not display the corresponding serial numbers in the positioning icons of each destination.

[0217] In the embodiments of the present application, in the exploration mode, the client can display the locations of each destination on the map, facilitating the user to clearly understand the location distribution of each destination, thereby facilitating the user to select a suitable location to go.

[0218] Figure 19 Shows an example diagram of the system architecture of the present application.

[0219] As Figure 19 shown, the system includes a client and a cloud. Among them, the client includes a multimedia content parsing module, an event restoration module, an entity association module, a map data generation module, a feasibility judgment module, and a rendering module. The cloud includes a POI library and a historical navigation database.

[0220] Among them, the multimedia content parsing module is used to parse input data such as videos, pictures, texts, audios, links, etc. to obtain corresponding multimedia content. For example,

[0221] The event restoration module is used to identify the POI information and location order of multiple destinations from the multimedia content obtained from the multimedia content module.

[0222] The entity association module is used to query the POI points (entities) corresponding to the multiple destinations on the map based on the POI information of the multiple destinations and the POI library in the cloud.

[0223] The map data generation module is used to generate map route data based on the POI information and address information of multiple destinations in navigation mode. And in exploration mode, generate map annotation data of the multiple destinations based on the POI information and address information of the multiple destinations.

[0224] The feasibility judgment module is used to query, in navigation mode, a historical navigation route that matches the map route data corresponding to the navigation route from the historical navigation database, and then compare the navigation route with the historical navigation route to judge the feasibility of the navigation route, that is, to judge whether the navigation route meets the navigation conditions.

[0225] The rendering module is used to render the navigation route that meets the navigation conditions onto the map for display in navigation mode. And in exploration mode, render the map annotation data of multiple destinations onto the map for display.

[0226] Figure 20The structural schematic diagram of the electronic device 100 is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0227] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0228] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and a SIM card interface.

[0229] The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 may receive the charging input from the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 may also supply power to the electronic device through the power management module 141.

[0230] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as the battery capacity, the number of battery cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0231] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0232] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: The antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0233] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, perform processing such as filtering and amplification on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be disposed in the same device.

[0234] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs an audio signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0235] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be transmitted from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0236] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technologies.

[0237] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0238] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0239] The electronic device 100 can implement the shooting function through the ISP, camera 193, video codec, GPU, display screen 194, application processor, etc.

[0240] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, file information such as music and videos is saved in the external memory card.

[0241] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the image playback function, etc.). The data storage area can store the data created during the use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121 and / or the instructions stored in the memory provided in the processor.

[0242] The electronic device 100 can implement the audio function through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, etc. For example, music playback, recording, etc.

[0243] The embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of this application can be implemented as a computer program or program code executed on a programmable system, and the programmable system includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0244] The program code can be applied to the input instructions to execute the various functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. To achieve the purpose of this application, the processing system includes any system having a processor such as a digital signal processor, microcontroller, application-specific integrated circuit, or microprocessor.

[0245] The program code can be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system. When necessary, the program code can also be implemented in an assembly language or a machine language. In fact, the mechanisms described in this application are not limited to the scope of any specific programming language. In any case, the language can be a compiled language or an interpreted language.

[0246] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored on one or more transient or non-transient machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions can be distributed via a network or via other computer-readable media. Thus, the machine-readable media can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to, floppy disks, optical disks, optical discs, read-only memories, magneto-optical discs, read-only memories, random access memories, erasable programmable read-only memories, electrically erasable programmable read-only memories, magnetic or optical cards, flash memories, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) in electrical, optical, acoustic, or other forms via the Internet. Thus, the machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0247] In the drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or ordering may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Additionally, the inclusion of a structural or method feature in a particular figure does not imply that such a feature is required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0248] It should be noted that each unit / module mentioned in the device embodiments of this application is a logical unit / module. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or can be implemented as a combination of multiple physical units / module. The physical implementation manner of these logical units / module themselves is not the most important. The combination of the functions implemented by these logical units / module is the key to solving the technical problems proposed in this application. In addition, to highlight the innovative part of this application, the above device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that there are no other units / modules in the above device embodiments.

[0249] It should be noted that in the examples and description of this patent, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one" does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0250] Although this application has been illustrated and described by reference to certain preferred embodiments thereof, those of ordinary skill in the art should understand that various changes may be made therein in form and detail without departing from the scope of this application.

Claims

1. A method for processing map data, for a client, characterized in that, The method includes: Determine the first data input by the user, where the first data includes video, audio, pictures, and text; Identify the address information of multiple target locations and the location order of the multiple target locations from the first multimedia content corresponding to the first data; Generate map route data based on the address information and the location order of the multiple target locations.

2. The method according to claim 1, wherein The generating map route data based on the address information and the location order of the multiple target locations includes: Based on the address information of the multiple target locations, query the matching points of the multiple target locations on the first map from the address information library of the first map of the client; Generate the map route data based on the matching points of the multiple target locations on the first map and the location order of the multiple target locations; Wherein, the address information library includes the address information of each location on the first map.

3. The method according to claim 2, wherein The address information includes at least one of the following: location name, region, coordinates, category.

4. The method according to claim 1, wherein Corresponding to the first multimedia content corresponding to the first data being video, the identifying the address information of the multiple target locations and the location order of the multiple target locations from the first multimedia content corresponding to the first data includes: Identify the key frames in the video; Extract the first text in the key frames and / or extract the second text corresponding to the audio within the first time interval at the moment when the key frames are located as the target text; Based on the semantic recognition model, identify the address information and the location order of the multiple target locations from the target text.

5. The method according to claim 4, characterized in that, The identifying the key frames in the video includes: Determine the first similarity between two adjacent video frames in the video; Corresponding to the first similarity being greater than or equal to a preset first similarity threshold, identify the video frame containing text among the two adjacent video frames as the key frame.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Determine whether the navigation route corresponding to the map route data meets the navigation conditions; Corresponding to the navigation route meeting the navigation conditions, display the navigation route on the first map of the client; Corresponding to the navigation route not meeting the navigation conditions, re-identify the address information and location order of the multiple target locations from the first multimedia content, and generate new map route data based on the re-identified address information and location order.

7. The method according to claim 6, wherein The determining whether the navigation route corresponding to the map route data meets the navigation conditions includes: Determine the crossing rate of the navigation route; Corresponding to the crossing rate being greater than or equal to a preset crossing rate threshold, determine that the navigation route does not meet the navigation conditions; Corresponding to the crossing rate being less than the preset crossing rate threshold, determine that the navigation route meets the navigation conditions; Wherein, the crossing rate is the ratio of the number of intersection points on the navigation route to the number of the multiple target locations.

8. The method according to claim 6, wherein The determining whether the navigation route corresponding to the map route data meets the navigation conditions further includes: Query the target historical navigation route in the historical navigation database that matches the navigation route; Determine the second similarity between the navigation route and the target historical navigation route; Corresponding to the second similarity being greater than or equal to a preset second similarity threshold, determine that the navigation route corresponding to the map route data meets the navigation condition; Corresponding to the second similarity being less than the preset second similarity threshold, determine that the navigation route corresponding to the map route data does not meet the navigation condition.

9. The method according to claim 8, wherein The query of the target historical navigation route in the historical navigation database that matches the navigation route includes: Determine the third similarity between the first multimedia content and the historical multimedia content corresponding to each historical input data in the historical navigation database; Determine the historical navigation route corresponding to the historical multimedia content with the third similarity greater than or equal to a preset third similarity threshold as the target historical navigation route.

10. The method according to claim 9, characterized in that, The determination of the third similarity between the first multimedia content corresponding to the first data and the historical multimedia content corresponding to the historical input data in the historical navigation database includes: Extract the fourth text of the first multimedia content corresponding to the first data and the fifth text of the historical multimedia content corresponding to the historical input data; Based on a semantic recognition model, determine the similarity between the fourth text and the fifth text as the third similarity.

11. A method for processing map data, for a client, characterized in that, The method includes: Determine the first data input by the user; Identify the address information of multiple target locations and the location order of the multiple target locations from the first multimedia content corresponding to the first data; Determine the map data display mode; Generate map data based on the map data display mode, and the address information and location order of the multiple target locations.

12. The method according to claim 11, wherein Corresponding to the map data display mode being the first mode, the generation of map data based on the map data display mode, and the address information and location order of the multiple target locations includes: Based on the address information of the multiple target locations, query the matching points of the multiple target locations on the first map from the address information library of the first map of the client; Generate the map route data based on the matching points of the multiple target locations on the first map and the location order of the multiple target locations; Wherein, the address information library includes the address information of each location on the first map.

13. The method according to claim 12, wherein The address information includes at least one of the following: location name, area, coordinate, category.

14. The method according to claim 11, wherein The first data includes at least one of the following: video, picture, text, audio, a link for indicating at least one of video, picture, text, audio, web page.

15. The method according to claim 11, wherein Corresponding to the map data display mode being the second mode, the generation of map data based on the map data display mode, and the address information and location order of the multiple target locations includes: Based on the address information of the multiple target locations, query the matching points of the multiple target locations on the first map from the address information library of the first map of the client; Generate map annotation data based on the matching points of the multiple target locations on the first map and the location order of the multiple target locations.

16. The method according to claim 15, characterized in that, The map annotation data includes at least one of the following: a link for indicating the multimedia content corresponding to the first data; the location order of the multiple target locations; at least one address data of the location name, area, coordinates, and category of the multiple target locations; the positioning marker data of the multiple target locations.

17. An electronic device, characterized in that, Comprising: A memory for storing instructions executed by one or more processors of the electronic device; A processor, when the processor executes the instructions in the memory, enables the electronic device to execute the map data processing method according to any one of claims 1 to 10 or claims 11 to 16.

18. A computer-readable storage medium, characterized in that, Instructions are stored on the computer-readable storage medium, and when the instructions are executed on the computer, the computer is enabled to execute the map data processing method according to any one of claims 1 to 10 or claims 11 to 16.

Images