Model training method, broadcast point prediction method, and computer program product

Abstract

An embodiment of the present application provides a model training method, a broadcast point prediction method, and a computer program product. The model training method includes: performing multimodal fusion on visual image road network data and actual trajectory road network data to obtain a logical road network including trajectory curves, extracting map points from the trajectory curves in the logical road network to obtain multiple sample map points; performing multidimensional feature extraction on the multiple sample map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction includes at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; constructing training samples based on the multidimensional features of the multiple sample map points and the sample map points pre-marked as broadcast points; and using the training samples to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of navigation technology, and in particular to a model training method, a broadcast point prediction method, and a computer program product. Background Art

[0002] When navigating a vehicle via an electronic map, navigation information is generally played via voice to improve driving safety. This allows the driver to understand the driving route directly through the voice broadcast. This makes the timing of the voice broadcast particularly important, especially for driving behaviors such as turning and U-turns. If the voice broadcast is triggered a little late, there may not be enough time or safe distance to perform the corresponding driving action.

[0003] Generally, the broadcast point (or navigation point) corresponding to the broadcast timing will be determined based on the intersection of two roads in the static road network and the preset distance. When the vehicle reaches the broadcast point or the distance between the vehicle and the broadcast point is less than the distance threshold, the voice broadcast of the turning action can be triggered.

[0004] However, due to differences in the number of lanes, lane types, etc. on various roads, the above-mentioned solution of determining the broadcast points based on the preset distance will result in poor accuracy of some of the determined broadcast points, which in turn leads to a poor user experience for the driver. Summary of the Invention

[0005] In view of this, an embodiment of the present application provides a model training solution to at least partially solve the above problems.

[0006] According to a first aspect of an embodiment of the present application, a model training method is provided, comprising: performing multimodal fusion on visual image road network data and actual trajectory road network data to obtain a logical road network including trajectory curves, performing map point extraction on the trajectory curves in the logical road network to obtain a plurality of sample map points; performing multidimensional feature extraction on the plurality of sample map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction comprises at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; constructing training samples based on the multidimensional features of the plurality of sample map points and the sample map points that are pre-marked as broadcast points; and using the training samples to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0007] According to a second aspect of an embodiment of the present application, a broadcast point prediction method is provided, comprising: performing multimodal fusion on visual image road network data and actual trajectory road network data to obtain a logical road network including trajectory curves, performing map point extraction on the trajectory curves in the logical road network to obtain a plurality of candidate map points; performing multidimensional feature extraction on the plurality of candidate map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction comprises at least one of the following: position data of a sample map point, angle data corresponding to the sample map point, and curvature of the trajectory curve at the sample map point; inputting the multidimensional features of the candidate map point into a trained prediction model, and predicting a navigation broadcast point from the plurality of candidate map points through the prediction model, wherein the prediction model is trained by the model training method of the present invention.

[0008] According to a third aspect of an embodiment of the present application, a model training device is provided, comprising: a first acquisition module for performing multimodal fusion of visual image road network data and actual trajectory road network data to obtain a logical road network including trajectory curves, and performing map point extraction on the trajectory curves in the logical road network to obtain a plurality of sample map points; a second acquisition module for performing multidimensional feature extraction on the plurality of sample map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction includes at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; a construction module for constructing training samples based on the multidimensional features of the plurality of sample map points and the sample map points that are pre-marked as broadcast points; and a training module for training a preset machine learning model using the training samples to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0009] According to the fourth aspect of the embodiment of the present application, a broadcast point prediction device is provided, including: a third acquisition module, used to perform multimodal fusion of visual image road network data and actual trajectory road network data to obtain a logical road network including trajectory curves, and perform map point extraction on the trajectory curves in the logical road network to obtain multiple candidate map points; a fourth acquisition module, used to perform multidimensional feature extraction on the multiple candidate map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction includes at least one of the following: position data of sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; a prediction module, used to input the multidimensional features of the candidate map points into a trained prediction model, and predict the navigation broadcast point from the multiple candidate map points through the prediction model, wherein the prediction model is trained by the device as described in the third aspect.

[0010] According to the fifth aspect of the embodiment of the present application, a computer program product is provided, including computer instructions, which instruct a computing device to execute the model training method described in the first aspect or the broadcast point prediction method described in the second aspect.

[0011] According to the solution provided in this embodiment, a logical road network containing trajectory curves is obtained by performing multimodal fusion on visual image road network data and actual trajectory road network data. Map point extraction can then be performed on the trajectory curves in the logical road network to obtain multiple sample map points, and multidimensional feature extraction can be performed on the multiple sample map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction includes at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; training samples are constructed based on the multidimensional features of the multiple sample map points and the sample map points that are pre-marked as broadcast points; and the training samples are used to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points. Since the actual trajectory road network data can reflect the driver's driving habits on the road, by performing multimodal fusion of visual image road network data and actual trajectory road network data, the trajectory curve in the fused logical road network can be made closer to the driving route of the vehicle when driving along the road. Then, the multi-dimensional features of the sample map points of the trajectory curve in the logical road network are obtained, and the preset machine learning model is trained based on this, so that the machine learning model can learn information such as the driver's driving habits on the road, and thus the road broadcast points determined by the broadcast point prediction model obtained through training are more in line with the driver's driving habits. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0013] Figure 1A Schematic diagram of a flow chart of a model training method according to the first embodiment of the present application;

[0014] Figure 1B for Figure 1A A schematic diagram of an example scenario in the illustrated embodiment;

[0015] Figure 2A Schematic diagram of a flow chart of a model training method according to the second embodiment of the present application;

[0016] Figure 2B for Figure 2AA schematic diagram of an example scenario in the illustrated embodiment;

[0017] Figure 3 Schematic diagram of a flow chart of a method for predicting broadcast points according to the third embodiment of the present application;

[0018] Figure 4 This is a structural block diagram of a model training device according to the fourth embodiment of the present application;

[0019] Figure 5 This is a structural block diagram of a broadcast point prediction device according to the fifth embodiment of the present application;

[0020] Figure 6 This is a structural diagram of an electronic device according to embodiment 6 of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0022] The following is an explanation of the technical terms involved in this application.

[0023] Actual travel trajectory: A trajectory aggregated from historical travel trajectory data corresponding to the GPS tracks of multiple users.

[0024] GPS track: refers to the user's movement track based on the GPS positioning system.

[0025] Link: The smallest unit in navigation data, a directed logical road unit divided according to actual roads. Each link has an independent ID and corresponds to a road in the road network.

[0026] Static road network: a road network structure composed of links (road sections) and nodes (intersections).

[0027] Trajectory network: A trajectory network generated by actual trajectories through operations such as pruning.

[0028] Visual image road network: a road network generated based on semantic segmentation from satellite remote sensing images.

[0029] Logical road network: A road network formed by a sequence curve with trajectory road network and visual image road network as input, multimodal fusion and thinning.

[0030] Remote sensing images: Remote sensing images refer to films or photos that record the electromagnetic wave size of various ground objects, which are mainly divided into aerial photographs and satellite photographs.

[0031] Semantic Segmentation: Semantic segmentation is a typical computer vision problem that involves taking images as input and converting them into masks with highlighted regions of interest.

[0032] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0033] Example 1

[0034] See also Figure 1A , shows a flow chart of a model training method provided in an embodiment of the present application, as shown in the figure, which includes:

[0035] S101, performing multimodal fusion on visual image road network data and actual trajectory road network data to obtain a logical road network including trajectory curves, and extracting map points from the trajectory curves in the logical road network to obtain a plurality of sample map points;

[0036] The visual image road network data is road network data corresponding to roads obtained based on the visual image. For example, the visual image road network data may specifically include: the position of the road in the image, the shape of the road, etc.

[0037] The actual track network data is road network data determined based on the actual track of the vehicle. For example, the actual track network data may include a collection of vehicle driving positions on the road. The actual track network data may reflect the vehicle's driving habits on the road.

[0038] In this embodiment, since the visual image road network data is the road network data corresponding to the road obtained based on the visual image, the visual image road network information can only reflect information such as the shape of the road, and cannot determine the driving conditions of the vehicle on the road; while the actual trajectory road network data can reflect the driving conditions of the vehicle on the road, but may lose some information about the road, such as losing relevant information about some areas on the road where the number of vehicles passing through is small.

[0039] In this embodiment, by performing multimodal fusion on the visual image road network data and the actual trajectory road network data, the trajectory curve in the fused logical road network can be made closer to the driving route of the vehicle when traveling along the road.

[0040] It should also be noted that a road may correspond to one or more driving directions, and each driving direction may correspond to a trajectory curve.

[0041] In this embodiment, after obtaining the logical road network, map points can be extracted from the trajectory curves in the logical road network to obtain multiple sample map points. The sample map points can be used to train a preset machine learning model in subsequent steps.

[0042] The specific method of extracting map points can be referred to related technologies, and will not be described in detail in this embodiment.

[0043] S102: Perform multi-dimensional feature extraction on the plurality of sample map points to obtain corresponding multi-dimensional features.

[0044] According to the object of the collected data, multi-dimensional features can be divided into position dimension, angle dimension and curvature dimension. The position dimension can be further divided into two dimensions according to physical position and relative position. Therefore, multi-dimensional features can specifically include features of the following dimensions:

[0045] Physical location dimension features: The physical location data of the sample map point in the road network, such as the physical location of the true value broadcast point in the road network curve;

[0046] Specifically, the physical location data of the sample map point in the road network may be the geographic location data of the sample map point. The geographic location data of the sample map point may be represented by latitude and longitude or by other means, which is not limited in this embodiment.

[0047] Angle dimension feature: The angle data corresponding to the sample map point can be the angle corresponding to the angle formed by the sample map point with other sample map points close to it, such as the angle between the sample map point and the adjacent sample map points, the angle between the sample map point and the starting and ending points of a curve, etc.

[0048] Curvature dimension features: the curvature of the trajectory curve at the sample map point, such as the curvature of the current sample map point, the distance between the current sample map point and the maximum curvature point, etc.

[0049] The curvature dimension may also include location data, specifically the distance data between the current sample map point and the sample map point with the largest curvature on the trajectory curve where it is located. The distance data between the current sample map point and the sample map point with the largest curvature on the trajectory curve where it is located may specifically be the distance data between the current sample map point and the sample map point corresponding to the maximum curvature on the trajectory curve.

[0050] Specifically, the angle data corresponding to the sample map point includes: angle data of the angle formed by the current sample map point and its two adjacent sample map points; angle data of the angle formed by the current sample map point and the starting point and end point of the trajectory curve where it is located.

[0051] Relative position dimension feature: the relative position data of the sample map point relative to the object in the road network, for example, the relative distance between the current sample map point and the standard (SD) road network link point.

[0052] In this embodiment, by performing multi-dimensional feature extraction on a plurality of sample map points to obtain corresponding multi-dimensional features, and constructing training samples based on the multi-dimensional features in subsequent steps, the prediction results of the trained broadcast point prediction model can be made more accurate.

[0053] In this embodiment, the specific method for obtaining multi-dimensional feature information is not limited.

[0054] S103: Construct training samples based on the multi-dimensional features of the plurality of sample map points and the sample map points pre-labeled as broadcast points.

[0055] In this embodiment, the announcement point is a map point that triggers the turn navigation announcement. When the vehicle travels to the announcement point or the distance from the announcement point is less than a distance threshold, the voice announcement of the turning action can be triggered.

[0056] In actual implementation, sample map points that are pre-marked as broadcast points can be obtained as positive samples, and some or all sample map points that are pre-marked as non-broadcast points can be obtained as negative samples; the training samples are constructed based on the multi-dimensional features of the positive samples and the negative samples.

[0057] When training is performed in subsequent steps, the preset machine learning model can be supervised and trained through positive and negative samples to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0058] Supervised training, also known as supervised learning, is mainly based on the features of the samples in the training set and the target variable, adjusting the model parameters so that the model has the ability to predict the target variable based on the features of the samples, where the target variable can be qualitative (such as category) or quantitative (such as continuous value).

[0059] In this embodiment, the feature of the sample may be the multi-dimensional feature obtained in the above steps; the target variable may be the broadcast point or non-broadcast point marked on the sample map point.

[0060] S104: Use the training samples to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0061] The following is an illustrative example of the training program provided in this embodiment through a specific usage scenario.

[0062] See also Figure 1B, the upper left corner shows a schematic diagram of a visual image road network data, and the black lines in the figure represent the roads included in the image. Figure 1B The lower left corner shows a schematic diagram of the actual trajectory road network data, where the black dots represent the trajectory points of the vehicle.

[0063] By performing multimodal fusion of visual image road network data and actual trajectory road network data, a logical road network can be obtained.

[0064] Figure 1B A trajectory curve is shown in the figure for exemplary description. Those skilled in the art should understand that when a road has multiple driving directions, the trajectory curves included in the logical road network may include multiple ones.

[0065] like Figure 1B As shown, a trajectory curve corresponding to the driving direction from the lower right corner to the upper left corner of the figure is shown, and the trajectory curve includes multiple sample map points. For each sample map point, multi-dimensional feature extraction can be performed to obtain multi-dimensional features of each sample map point.

[0066] Technicians can mark multiple sample map points. For example, if one sample map point in the trajectory curve is marked as a broadcast point, the other sample map points will be marked as non-broadcast points by default.

[0067] Afterwards, training samples can be constructed based on the multi-dimensional features of the plurality of sample map points and the sample map points that are pre-marked as broadcast points, and the preset machine learning model can be trained to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0068] The solution provided in this embodiment obtains a logical road network including trajectory curves by performing multimodal fusion on visual image road network data and actual trajectory road network data. Map points can then be extracted from the trajectory curves in the logical road network to obtain multiple sample map points, and multidimensional feature extraction can be performed on the multiple sample map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction includes at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; training samples are constructed based on the multidimensional features of the multiple sample map points and the sample map points that are pre-marked as broadcast points; and the training samples are used to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points. Since the actual trajectory road network data can reflect the driver's driving habits on the road, by performing multimodal fusion of visual image road network data and actual trajectory road network data, the trajectory curve in the fused logical road network can be made closer to the driving route of the vehicle when driving along the road. Then, the multi-dimensional features of the sample map points of the trajectory curve in the logical road network are obtained, and the preset machine learning model is trained based on this, so that the machine learning model can learn information such as the driver's driving habits on the road, so that the road broadcast points determined by the broadcast point prediction model obtained through training are more in line with the driver's driving habits.

[0069] The model training method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc.

[0070] Example 2

[0071] See also Figure 2A , shows a flow chart of a model training method provided in an embodiment of the present application, as shown in the figure, which includes:

[0072] S201: Perform image segmentation processing on a remote sensing image of a road to obtain visual image road network data of the road.

[0073] In this embodiment, a semantic segmentation neural network model can be used to mark each pixel in the remote sensing image, determine the pixel points corresponding to the road in the remote sensing image based on the marking results, and obtain the visual image road network data of the road based on the pixel points corresponding to the road.

[0074] See also Figure 2B , visual image road network data can be obtained based on the "target remote sensing image" in the data source.

[0075] S202: Determine actual driving trajectory road network data of the road based on the historical driving trajectory data corresponding to the road.

[0076] In this embodiment, the map area corresponding to the road can be determined first, and then the historical driving trajectory data within the map area can be obtained. The historical driving trajectory data can then be aggregated to obtain the actual road trajectory network data of the road. The historical driving trajectory data can be generated through data engineering using the actual GPS data corresponding to the vehicle.

[0077] See also Figure 2B , you can obtain the actual walking trajectory road network data based on the "GPS actual walking data" in the data source.

[0078] In this embodiment, the actual travel trajectory road network data includes driving trajectory data corresponding to multiple driving directions of the road.

[0079] For example, taking the road corresponding to an intersection as an example, each intersection has three driving directions: straight, left turn, and right turn. Then, an intersection corresponds to 12 driving directions in total. The actual trajectory road network data can include driving trajectory data corresponding to the 12 driving directions respectively.

[0080] When determining the actual travel trajectory road network data, operations such as pruning can also be performed on the historical travel trajectory data. The specific pruning method can be referred to related technologies and will not be described in detail here.

[0081] Optionally, in this embodiment, when the road is a road connecting an intersection, step S202 may include: obtaining geographic location data corresponding to the intersection from static road network data; filtering the historical driving trajectory data based on the geographic location data corresponding to the intersection, and retaining historical driving trajectory data whose distance from the intersection is less than a preset distance; determining the actual driving trajectory road network data of the intersection based on the retained historical driving trajectory data, the actual driving trajectory road network data including driving trajectory data corresponding to multiple driving directions of the intersection.

[0082] In another implementation of this application, see Figure 2B , the driving trajectory data included in the actual trajectory road network data can also be filtered according to the static road network, and the driving trajectory data whose distance from the intersection is less than a preset distance is retained.

[0083] In this embodiment, static road network data refers to road topology data, which may include links and nodes. A link is the smallest unit of navigation data, specifically the road segment from one intersection to the next. The nodes at the ends of a link are called intersection nodes. Static road network data may also include geographic location data and segment length data corresponding to each link, as well as geographic location data for each node.

[0084] In this embodiment, the geographic location data corresponding to the intersection can be obtained from the static road network data. Therefore, the historical driving trajectory data can be filtered according to the geographic location data of the intersection, and the historical driving trajectory data whose distance from the intersection is less than a preset distance is retained to reduce the data volume of the historical driving trajectory data.

[0085] S203: Perform multimodal fusion on the visual image road network data and the actual trajectory road network data to obtain a logical road network including trajectory curves, and extract map points from the trajectory curves in the logical road network to obtain multiple sample map points.

[0086] See also Figure 2B , perform multimodal fusion of visual image road network data and actual trajectory road network data to obtain a logical road network containing trajectory curves.

[0087] In this embodiment, a multimodal fusion solution based on deep learning can be used to perform multimodal fusion on the visual image road network data and the actual trajectory road network data. The specific algorithm used for multimodal fusion can be referred to in related technologies and will not be described in detail here.

[0088] Preferably, since the purpose of this embodiment is to train a prediction model for predicting announcement points, and announcement points are not required for straight driving, in this step, the trajectory curve corresponding to straight driving may not be generated, or the historical driving trajectory data corresponding to straight driving may be directly filtered out in the above step.

[0089] In this embodiment, after obtaining the trajectory curve, map point extraction processing can be performed on the trajectory curve according to a preset extraction rule to obtain multiple sample map points. The specific extraction rule can be referred to related technologies and is not limited in this embodiment.

[0090] S204: Perform multi-dimensional feature extraction on the plurality of sample map points to obtain corresponding multi-dimensional features.

[0091] The multi-dimensional feature extraction includes at least one of the following: position data of sample map points, angle data corresponding to sample map points, and curvature of the trajectory curve at the sample map points.

[0092] The specific implementation of this step can be referred to the above embodiment and will not be repeated here.

[0093] In addition, in this embodiment, when the road is a road connecting an intersection, the location data of the sample map point also includes distance data between the sample map point and the static road network node corresponding to the intersection. Specifically, the static road network node corresponding to the intersection can be determined, and then the distance data between the sample map point and the static intersection node can be determined.

[0094] S205. Obtain sample map points that are pre-marked as broadcast points as positive samples, and obtain some or all sample map points that are pre-marked as non-broadcast points as negative samples; and construct the training samples based on the multi-dimensional features of the positive samples and the negative samples.

[0095] In this embodiment, when determining negative samples, all sample map points marked as non-broadcast points can be directly used as negative samples; or a preset number of sample map points can be randomly selected from all sample map points marked as non-broadcast points as negative samples. This embodiment is not limited to this.

[0096] S206: Use the training samples to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0097] See also Figure 2B In this embodiment, a preset machine learning model can be trained based on the sample map points included in the logical road network, and the broadcast point prediction model obtained by training can be used for broadcast point mining.

[0098] In this embodiment, after constructing the training samples, the preset machine learning model can be trained by supervised training. The specific training method can be referred to the relevant materials and will not be described in detail here.

[0099] In addition, the machine learning model preset in this embodiment can be any model that can perform supervised training, and this embodiment does not limit the specific implementation of the model.

[0100] In this embodiment, the trained broadcast point prediction model can be used for broadcast point mining. The specific method of broadcast point mining can be referred to the subsequent embodiments.

[0101] The solution provided in this embodiment can be combined with intersection scenarios. By performing multimodal fusion on the visual image road network data and the actual trajectory road network data corresponding to the intersection, and training a neural network model based on the sample map points in the logical road network obtained by fusion, the training samples corresponding to the announcement points can be more accurately determined in combination with the intersection scenario. Moreover, since the actual trajectory road network data is highly correlated with the driver's driving habits, the solution provided in this embodiment can be combined with the driver's driving habits to better model the model, making the predicted announcement points of the trained announcement point prediction model more accurate. In addition, triggering voice announcements based on the predicted announcement points can improve the driver's user experience.

[0102] The model training method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc.

[0103] Example 3

[0104] See also Figure 3 , shows a flow chart of a method for predicting broadcast points provided in an embodiment of the present application, as shown in the figure, which includes:

[0105] S301: Perform multimodal fusion on visual image road network data and actual trajectory road network data to obtain a logical road network including trajectory curves, and extract map points from the trajectory curves in the logical road network to obtain multiple candidate map points.

[0106] In this embodiment, the method for obtaining candidate map points is similar to the method for obtaining sample map points in the above embodiment, and will not be described in detail here.

[0107] S302: Perform multi-dimensional feature extraction on the plurality of candidate map points to obtain corresponding multi-dimensional features.

[0108] The multi-dimensional feature extraction includes at least one of the following: position data of sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points.

[0109] In this embodiment, the method of extracting multi-dimensional features from candidate map points is similar to the method of extracting multi-dimensional features from sample map points in the above embodiment, and will not be described in detail here.

[0110] S303: Input the multi-dimensional features of the candidate map points into a trained broadcast point prediction model, and predict a navigation broadcast point from the multiple candidate map points using the broadcast point prediction model.

[0111] The broadcast point prediction model is trained by the method described in the first or second embodiment.

[0112] The specific prediction method of the broadcast point prediction model can be referred to related technologies and will not be described in detail here.

[0113] The solution provided in this embodiment predicts the broadcast points of the road by using the broadcast point prediction model trained by the method of the above embodiment. The obtained broadcast points are more in line with the driver's operating habits. Triggering voice broadcasts based on the predicted broadcast points can improve the driver's user experience.

[0114] The broadcast point prediction method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a mobile phone, a PAD, etc.) and a PC.

[0115] Example 4

[0116] See also Figure 4 , shows a structural block diagram of a model training device provided in an embodiment of the present application.

[0117] In this embodiment, the device includes:

[0118] A first acquisition module 402 is configured to perform multimodal fusion of the visual image road network data and the actual trajectory road network data to obtain a logical road network including trajectory curves, and extract map points from the trajectory curves in the logical road network to obtain a plurality of sample map points;

[0119] A second acquisition module 404 is configured to perform multi-dimensional feature extraction on the plurality of sample map points to obtain corresponding multi-dimensional features, wherein the multi-dimensional feature extraction includes at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points;

[0120] A construction module 406 is configured to construct a training sample based on the multi-dimensional features of the plurality of sample map points and the sample map points that are pre-labeled as broadcast points;

[0121] The training module 408 is used to train a preset machine learning model using the training samples to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0122] Optionally, the device further comprises:

[0123] The fifth acquisition module is used to perform semantic segmentation processing on the remote sensing image of the road to obtain visual image road network data of the road.

[0124] Optionally, the device further comprises:

[0125] The determination module is used to determine the actual driving trajectory road network data of the road based on the historical driving trajectory data corresponding to the road, wherein the actual driving trajectory road network data includes driving trajectory data corresponding to multiple driving directions of the road.

[0126] Optionally, the determination module is used to obtain the geographic location data corresponding to the intersection from the static road network data when the road is a road connecting an intersection; based on the geographic location data corresponding to the intersection, the historical driving trajectory data is filtered, and the historical driving trajectory data whose distance from the intersection is less than a preset distance is retained; based on the retained historical driving trajectory data, the actual driving trajectory road network data of the intersection is determined, and the actual driving trajectory road network data includes driving trajectory data corresponding to multiple driving directions of the intersection.

[0127] Optionally, the training module 408 is used to obtain sample map points that are pre-marked as broadcast points as positive samples, and obtain some or all sample map points that are pre-marked as non-broadcast points as negative samples; and construct the training samples based on the multi-dimensional features of the positive samples and the negative samples.

[0128] Optionally, the location data of the sample map point includes at least one of the following: geographic location data of the sample map point; and distance data between the current sample map point and a sample map point with a larger curvature on the trajectory curve where the current sample map point is located.

[0129] Optionally, when the road is a road connecting an intersection, the location data of the sample map point further includes: distance data between a static road network node corresponding to the intersection and the sample map point.

[0130] Optionally, the angle data corresponding to the sample map point includes: angle data of the angle formed by the current sample map point and its two adjacent sample map points; angle data of the angle formed by the current sample map point and the starting point and end point of the trajectory curve where it is located.

[0131] The model training device of this embodiment is used to implement the corresponding model training methods of the aforementioned multiple method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the model training device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0132] Example 5

[0133] Reference Figure 5 , shows a structural block diagram of a broadcast point prediction device provided in an embodiment of the present application.

[0134] In this embodiment, the device includes:

[0135] The third acquisition module 502 is configured to perform multimodal fusion of the visual image road network data and the actual trajectory road network data to obtain a logical road network including trajectory curves, and extract map points from the trajectory curves in the logical road network to obtain a plurality of candidate map points;

[0136] A fourth acquisition module 504 is configured to perform multi-dimensional feature extraction on the plurality of candidate map points to obtain corresponding multi-dimensional features, wherein the multi-dimensional feature extraction includes at least one of the following: position data of a sample map point, angle data corresponding to the sample map point, and curvature of a trajectory curve at the sample map point;

[0137] The prediction module 506 is used to input the multi-dimensional features of the candidate map points into the trained prediction model, and predict the navigation broadcast point from the multiple candidate map points through the prediction model, wherein the prediction model is trained by the device as described in the third aspect.

[0138] The broadcast point prediction device of this embodiment is used to implement the corresponding broadcast point prediction methods of the multiple method embodiments described above, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the broadcast point prediction device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, and will not be described in detail here.

[0139] Example 6

[0140] Reference Figure 6 , shows a structural diagram of an electronic device according to embodiment six of the present application. The specific embodiments of the present application do not limit the specific implementation of the electronic device.

[0141] like Figure 6 As shown, the electronic device may include a processor 602 , a communications interface 604 , a memory 606 , and a communication bus 608 .

[0142] in:

[0143] The processor 602 , the communication interface 604 , and the memory 606 communicate with each other via a communication bus 608 .

[0144] The communication interface 604 is used to communicate with other electronic devices or servers.

[0145] The processor 602 is configured to execute a program 610 , which may include multiple instructions.

[0146] Specifically, in an implementation scheme of this embodiment, the processor is used to execute: performing multimodal fusion of visual image road network data and actual trajectory road network data to obtain a logical road network containing trajectory curves, performing map point extraction on the trajectory curves in the logical road network, and obtaining a plurality of sample map points; performing multidimensional feature extraction on the plurality of sample map points to obtain corresponding multidimensional features, wherein the multidimensional feature extraction includes at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; constructing training samples based on the multidimensional features of the plurality of sample map points and the sample map points that are pre-marked as broadcast points; and using the training samples to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points.

[0147] Specifically, in another implementation scheme of this embodiment, the processor is used to execute: performing multimodal fusion of visual image road network data and actual trajectory road network data to obtain a logical road network containing trajectory curves, performing map point extraction on the trajectory curves in the logical road network, and obtaining instructions for multiple candidate map points; performing multidimensional feature extraction on the multiple candidate map points, and obtaining corresponding multidimensional features, wherein the multidimensional feature extraction includes at least one of the following: position data of sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; inputting the multidimensional features of the candidate map points into a trained prediction model, and predicting instructions for navigation broadcast points from the multiple candidate map points through the prediction model, wherein the prediction model is trained by the method described above.

[0148] The specific implementation of each of the above instructions can refer to the corresponding descriptions of the corresponding steps and units in the above-mentioned model training method or broadcast point prediction method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the above-mentioned method embodiments, and will not be repeated here.

[0149] The memory 606 is used to store the program 610. The memory 606 may include a high-speed RAM memory, or may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0150] Another embodiment of the present application further provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to the methods described in the above embodiments one, two or three.

[0151] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0152] The methods according to the embodiments of the present application described above can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored in a remote recording medium or non-transitory machine-readable medium downloaded via a network and then stored in a local recording medium. Thus, the methods described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It will be understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the model training method or broadcast point prediction method described herein is implemented. Furthermore, when a general-purpose computer accesses the code for implementing the model training method or broadcast point prediction method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the model training method or broadcast point prediction method shown herein.

[0153] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of this application.

[0154] The above implementation methods are only used to illustrate the embodiments of the present application, and are not intended to limit the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims.

Claims

1. A model training method, comprising: Performing multimodal fusion on the visual image road network data and the actual trajectory road network data to obtain a logical road network including trajectory curves, and extracting map points from the trajectory curves in the logical road network to obtain multiple sample map points; Performing multi-dimensional feature extraction on the plurality of sample map points to obtain corresponding multi-dimensional features, wherein the multi-dimensional feature extraction includes at least one of the following: position data of the sample map points, angle data corresponding to the sample map points, and curvature of the trajectory curve at the sample map points; Constructing training samples based on the multi-dimensional features of the plurality of sample map points and the sample map points pre-marked as broadcast points; The training samples are used to train a preset machine learning model to obtain a broadcast point prediction model that can be used to determine road broadcast points.

2. The method according to claim 1, wherein The method further comprises: Image segmentation processing is performed on the remote sensing image of the road to obtain visual image road network data of the road.

3. The method according to claim 1, wherein The method further comprises: According to the historical driving trajectory data corresponding to the road, the actual driving trajectory road network data of the road is determined, wherein the actual driving trajectory road network data includes driving trajectory data corresponding to multiple driving directions of the road.

4. The method according to claim 3, wherein: When the road is a road connecting an intersection, determining the actual travel trajectory road network data of the road based on the historical travel trajectory data corresponding to the road includes: Obtaining geographic location data corresponding to the intersection from static road network data; Filtering the historical driving trajectory data according to the geographic location data corresponding to the intersection, and retaining the historical driving trajectory data whose distance to the intersection is less than a preset distance; According to the retained historical driving trajectory data, actual driving trajectory network data of the intersection is determined, and the actual driving trajectory network data includes driving trajectory data corresponding to multiple driving directions of the intersection.

5. The method according to claim 1, wherein The constructing of training samples based on the multi-dimensional features of the plurality of sample map points and the sample map points pre-labeled as broadcast points includes: Obtain sample map points that are pre-marked as broadcast points as positive samples, and obtain some or all sample map points that are pre-marked as non-broadcast points as negative samples; The training samples are constructed according to the multi-dimensional features of the positive samples and the negative samples.

6. The method according to claim 1, wherein The location data of the sample map point includes at least one of the following: Geographic location data of sample map points; The distance data between the current sample map point and the sample map point corresponding to the maximum curvature value on the trajectory curve where it is located.

7. The method according to claim 6, wherein: When the road is a road connecting an intersection, the location data of the sample map point also includes: The distance data between the static road network node corresponding to the intersection and the sample map point.

8. The method according to claim 1, wherein The angle data corresponding to the sample map point includes: The angle data of the angle formed by the current sample map point and its two adjacent sample map points; The angle data of the angle formed by the current sample map point and the start and end points of the trajectory curve where it is located.

9. A method for predicting a broadcast point, comprising: Performing multimodal fusion on the visual image road network data and the actual trajectory road network data to obtain a logical road network containing trajectory curves, and extracting map points from the trajectory curves in the logical road network to obtain multiple candidate map points; Performing multi-dimensional feature extraction on the plurality of candidate map points to obtain corresponding multi-dimensional features, wherein the multi-dimensional feature extraction includes at least one of the following: position data of a sample map point, angle data corresponding to the sample map point, and curvature of a trajectory curve at the sample map point; The multi-dimensional features of the candidate map points are input into a trained broadcast point prediction model, and a navigation broadcast point is predicted from the multiple candidate map points by the broadcast point prediction model, wherein the broadcast point prediction model is trained by the method described in claims 1-8.

10. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the model training method according to any one of claims 1 to 8, or to perform operations corresponding to the broadcast point prediction method according to claim 9.

Images