Navigation action prediction model training method, navigation action generation method and device

Abstract

A navigation action prediction model training method, a navigation action generation method and device are disclosed. A training sample set is constructed, the training sample set including one or more training samples, the training sample being image data including an entry road section and an exit road section at an intersection, and a label of the training sample being used to represent a navigation action from the entry road section to the exit road section; and a navigation action prediction model is trained based on the training sample set. Thus, a navigation action prediction model can be obtained, in which the prediction accuracy can be guaranteed, and the accuracy of the prediction result can be evaluated.

Description

Technical Field

[0001] This disclosure relates to the field of road navigation, and in particular to a method for training a navigation action prediction model, a method for generating navigation actions, and an apparatus. Background Technology

[0002] Navigation action generation is an important component of navigation services.

[0003] Navigation actions refer to the turning actions that should be performed at intersections, such as turning left or going straight.

[0004] Traditional navigation action generation methods mainly involve capturing the entry and exit road segments and calculating the navigation actions to be performed at intersections. However, due to the complexity of road networks, the generated navigation actions need to be validated to ensure reliability.

[0005] The above-mentioned navigation action generation method has the following problems: 1. The key road segment that determines the navigation action may be located outside the intercepted road segment, which makes it impossible to guarantee the accuracy of the generated navigation action; 2. The accuracy of the generated navigation action is highly dependent on the calculation rules. If the calculation rules are not set properly, it will lead to incorrect navigation actions, and the calculation results are easily wrong at the boundary conditions of the calculation rules; 3. The verification of the generated navigation action is highly dependent on human participation, and the labor cost is high.

[0006] Therefore, a new method for generating navigation actions is needed to solve at least one of the problems mentioned above in traditional navigation action generation methods. Summary of the Invention

[0007] One technical problem this disclosure aims to solve is to provide a new method for generating navigation actions, thereby addressing at least one of the aforementioned problems existing in traditional navigation action generation methods.

[0008] According to a first aspect of this disclosure, a method for training a navigation action prediction model is provided, comprising: constructing a training sample set, the training sample set including one or more training samples, the training samples being image data containing entry and exit segments at intersections, the labels of the training samples being used to characterize navigation actions from the entry segment to the exit segment; and training a navigation action prediction model based on the training sample set.

[0009] According to a second aspect of this disclosure, a navigation action generation method is provided, comprising: acquiring prediction samples, wherein the prediction samples are image data containing an entry segment and an exit segment at an intersection; inputting the prediction samples into a pre-trained navigation action prediction model to obtain a prediction result output by the navigation action prediction model, wherein the prediction result is used to characterize the navigation action from the entry segment to the exit segment.

[0010] According to a third aspect of this disclosure, a navigation action generation method is provided, comprising: generating a first navigation action based on the angle between an entry segment and an exit segment at an intersection; inputting image data containing the entry segment and the exit segment into a pre-trained navigation action prediction model to obtain a second navigation action; comparing whether the first navigation action and the second navigation action are consistent; if the first navigation action and the second navigation action are consistent or substantially consistent, then using the first navigation action or the second navigation action as the navigation action from the entry segment to the exit segment.

[0011] According to a fourth aspect of this disclosure, an apparatus for training a navigation action prediction model is provided, comprising: a construction module for constructing a training sample set, the training sample set including one or more training samples, the training samples being image data containing an entry segment and an exit segment at an intersection, the labels of the training samples being used to characterize the navigation action from the entry segment to the exit segment; and a training module for training the navigation action prediction model based on the training sample set.

[0012] According to a fifth aspect of this disclosure, a navigation action generation apparatus is provided, comprising: an acquisition module for acquiring prediction samples, wherein the prediction samples are image data including an entry segment and an exit segment at an intersection; and a prediction module for inputting the prediction samples into a pre-trained navigation action prediction model to obtain a prediction result output by the navigation action prediction model, wherein the prediction result is used to characterize the navigation action from the entry segment to the exit segment.

[0013] According to a sixth aspect of this disclosure, a navigation action generation apparatus is provided, comprising: a first generation module for generating a first navigation action based on the angle between an entry segment and an exit segment at an intersection; a second generation module for inputting image data containing the entry segment and the exit segment into a pre-trained navigation action prediction model to obtain a second navigation action; a comparison module for comparing whether the first navigation action and the second navigation action are consistent; and a determination module for determining whether, if the first navigation action and the second navigation action are consistent or substantially consistent, the first navigation action or the second navigation action is taken as the navigation action from the entry segment to the exit segment.

[0014] According to a seventh aspect of this disclosure, a computing device is provided, comprising: a processor; and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of the first to third aspects above.

[0015] According to the eighth aspect of this disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored, which, when executed by a processor of an electronic device, causes the processor to perform the method described in any one of the first to third aspects described above.

[0016] Therefore, this disclosure trains a navigation action prediction model for predicting navigation actions based on image data including the entry and exit segments at intersections. The trained navigation action prediction model predicts navigation actions by using image analysis. Compared with other types of data, images contain almost all the features needed to determine navigation actions, resulting in high prediction accuracy of the trained navigation action prediction model. The accuracy of the prediction results of the navigation action prediction model can be evaluated. Attached Figure Description

[0017] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0018] Figure 1 A schematic flowchart of a method for training a navigation action prediction model according to an embodiment of the present disclosure is shown.

[0019] Figure 2A A schematic diagram of the entry and exit sections is shown.

[0020] Figure 2B Showing the target Figure 2A A schematic diagram of the constructed training samples.

[0021] Figure 3 It shows Figure 1 The method shown may also include schematic diagrams of operational steps.

[0022] Figure 4 A schematic flowchart of a navigation action generation method according to an embodiment of the present disclosure is shown.

[0023] Figure 5 A schematic flowchart of a navigation action generation method according to another embodiment of the present disclosure is shown.

[0024] Figure 6 A schematic diagram of the structure of an apparatus for training a navigation action prediction model according to an embodiment of the present disclosure is shown.

[0025] Figure 7 A schematic diagram of a navigation action generation apparatus according to an embodiment of the present disclosure is shown.

[0026] Figure 8 A schematic diagram of a navigation action generation apparatus according to another embodiment of the present disclosure is shown.

[0027] Figure 9 A schematic diagram of the structure of a computing device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0028] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0029] Figure 1 A schematic flowchart of a method for training a navigation action prediction model according to an embodiment of the present disclosure is shown. Figure 1 The method shown can be implemented in software using a computer program, or it can be executed using a computing device with a specific configuration. Figure 1 The method shown.

[0030] See Figure 1 In step S110, a training sample set is constructed. The training sample set includes one or more training samples. The training samples are image data of the intersection containing the entry segment and the exit segment. The labels of the training samples are used to characterize the navigation action from the entry segment to the exit segment.

[0031] An intersection, also known as a crossroads, is a place where two or more roads meet. The "entering section" refers to the section of road traversed before reaching the intersection, that is, the section of road behind the intersection in the direction of travel. The "exit section" refers to the section of road traversed after reaching the intersection, that is, the section of road ahead of the intersection in the direction of travel.

[0032] In addition to the entry and exit lanes, the road segment connected to an intersection may typically include several other road segments. For example... Figure 2A Taking the irregular intersection shown as an example, in addition to the entry and exit sections shown in the diagram, it also includes two other sections connected to the intersection.

[0033] To avoid interference with model training due to training samples containing information useless for model prediction, training samples can be image data of the intersection that includes both inbound and outbound road segments but excludes other road segments at the intersection. In other words, the image data excludes road segments connected to the intersection, only the inbound and outbound segments at the intersection.

[0034] As an example, the prediction sample could be image data including the entry segment, the exit segment, and the road portion connecting the entry and exit segments (i.e., the intersection). For example, for... Figure 2A The training samples constructed from the irregular intersections shown can be... Figure 2B The image shown includes an entry section, an exit section, and an intersection where the entry and exit sections connect.

[0035] When constructing the training sample set, image data containing entry and exit road segments can be constructed in one or more ways and added to the training sample set as training samples. For example, the training sample set can be constructed in one or more exemplary ways, including but not limited to the following.

[0036] As an example, road images containing entry and exit segments can be extracted from map data and added to the training sample set as training samples. Specifically, road images containing entry and exit segments can be extracted from existing map data through image cropping. Furthermore, if other road segments exist at intersections connecting to the entry and exit segments, road images containing only the entry and exit segments and the intersections connecting them can be extracted. Map data can be road image data provided by various map software, such as real-world road images or simulated road images.

[0037] As an example, image data obtained by capturing images of roads containing both entry and exit sections can also be added to the training sample set as training samples. For instance, road images captured by cameras positioned at road intersections can be obtained, and image data containing entry and exit sections can be extracted from these images. If other road sections exist at the intersections connected to the entry and exit sections, the portion of the road image containing only the entry and exit sections, as well as the intersections connecting to them, can be extracted and used as training samples.

[0038] As an example, images containing the entry and exit road segments can be generated based on the entry and exit road segment information, and these images can be added to the training sample set as training samples. The entry and exit road segment information can be any form of information that can characterize the entry and exit road segments, such as, but not limited to, textual descriptions of the entry and exit road segments.

[0039] The labels on training samples are used to represent the actual navigation actions from entering to exiting a road segment. Labels on training samples can be obtained through methods including, but not limited to, manual annotation. For example, multiple training samples (i.e., image data containing both entering and exiting road segments) can be packaged into a labeling task and submitted to a labeling platform, which then distributes it to labelers for annotation. Alternatively, multiple training samples can be directly submitted to labelers for annotation.

[0040] In step S120, a navigation action prediction model is trained based on the training sample set.

[0041] The model can be trained using training samples (i.e., image data containing entry and exit points) as input and the labels of the training samples (i.e., navigation actions) as output. The navigation action prediction model can be trained using supervised learning. The specific training process is a mature technology in this field and will not be described in detail here.

[0042] From the perspective of model structure, the navigation action prediction model can be any machine learning model suitable for processing image data, such as, but not limited to, deep learning models based on deep learning algorithms. Optionally, the navigation action prediction model can be a deep learning model based on the residual network algorithm, which has achieved great success in the image domain, i.e., a residual network model.

[0043] Deep learning algorithms are complex machine learning algorithms that primarily improve classification or prediction accuracy by building machine learning models with multiple hidden layers and massive amounts of training data to learn more useful features.

[0044] Residual networks are one of the mainstream models in the field of image processing. They are a type of convolutional neural network composed of stacked residual modules, which effectively alleviates the effects of gradient vanishing and allows for a significant increase in the number of layers in the network model.

[0045] From the perspective of model prediction mechanism, navigation action prediction model can be regarded as a multi-classification model. The input of navigation action prediction model can be image data containing the entry segment and the exit segment (i.e., the prediction sample mentioned below). The navigation action prediction model can process the input to obtain the probability value of different navigation actions and output the navigation action with the highest probability value as the prediction result of the prediction sample, that is, the predicted navigation action from the entry segment to the exit segment.

[0046] As an example, at the start of training, the parameters of the navigation action prediction model can be randomly initialized. During each iteration, one or more training samples can be input into the model, and the predicted results are compared with the labels of the training samples to obtain the value of the loss function. The model is then adjusted to reduce the loss function, and the next iteration begins, continuing until the loss function converges. The loss function evaluates the degree to which the model's predicted values ​​differ from the true values. Loss functions and their use in evaluating model performance are mature techniques in this field and will not be elaborated upon here.

[0047] To improve the generalization ability of the navigation action prediction model, this disclosure can also construct a test sample set. The test sample set includes one or more test samples, which are image data containing the entry and exit segments at intersections. The labels of the test samples are used to characterize the actual navigation actions from the entry segment to the exit segment (for ease of distinction, they can be referred to as the first actual navigation actions). The test samples and training samples are image data for different entry and exit segments; that is, the test samples and training samples are sample data constructed for different entry and exit segments. The test sample set can be constructed using the training sample set construction method described above, which will not be elaborated further in this disclosure.

[0048] During the training of the navigation action prediction model based on the training sample set, test samples can be input into the navigation action prediction model to obtain the prediction result of the test samples output by the navigation action prediction model, that is, the predicted navigation action (for ease of distinction, it can be called the first predicted navigation action). The navigation action prediction model can be adjusted based on the difference between the first predicted navigation action and the first real navigation action, and the adjusted navigation action prediction model can be trained based on the training sample set.

[0049] The difference between the first predicted navigation action and the first real navigation action reflects the performance of the navigation action prediction model trained on the training sample set on the test sample set. In other words, the test sample set can be used to preliminarily evaluate the capability of the navigation action prediction model trained on the training sample set. For example, it can be used during or after training to assess whether the navigation action prediction model is overfitting and to estimate the generalization error. Based on the evaluation results, the navigation action prediction model can be adjusted. Adjusting the navigation action prediction model mainly refers to adjusting its hyperparameters, and optionally, the model parameters. Hyperparameters are parameters whose values ​​are set before the learning process begins, rather than parameter data obtained through training the model; examples include the learning rate and the number of hidden layers in a deep neural network.

[0050] For example, if the difference between the first predicted navigation action and the first real navigation action indicates that the navigation action prediction model is diverging on the test sample set and that mAP (Mean Average Precision) is not increasing or is increasing very slowly, training can be terminated in time and the hyperparameters or model parameters used in training can be readjusted without waiting for the training to end.

[0051] This disclosure can also construct a verification sample set, which includes one or more verification samples. Each verification sample is image data containing entry and exit road segments. The label of the verification sample is used to characterize the actual navigation action from the entry segment to the exit segment (which can be referred to as the second actual navigation action for easy distinction). The verification samples, test samples, and training samples are all image data for different entry and exit road segments; that is, the verification samples, test samples, and training samples are sample data constructed for different entry and exit road segments, respectively. The verification sample set can be constructed using the training sample set construction method described above, which will not be repeated here.

[0052] like Figure 3 As shown, after adjusting the navigation action prediction model based on the test sample set and training the adjusted model based on the training sample set, validation samples can be input into the trained model to obtain the prediction results of the validation samples, i.e., the predicted navigation actions (which can be referred to as the second predicted navigation actions for ease of distinction). The navigation action prediction model is then evaluated based on the difference between the second predicted navigation actions and the second true navigation actions. Furthermore, the accuracy of the navigation action prediction model can be comprehensively evaluated based on the differences between the second predicted navigation actions and the second true navigation actions from multiple validation samples.

[0053] The evaluation results can be compared with a preset iteration termination condition. If the evaluation results do not meet the iteration termination condition, the steps of adjusting the navigation action prediction model based on the test sample set, training the adjusted navigation action prediction model based on the training sample set, and evaluating the navigation action prediction model based on the validation sample set are executed iteratively until the evaluation results meet the iteration termination condition. The evaluation results can be, but are not limited to, metrics that characterize the performance of the navigation action prediction model, such as accuracy. The iteration termination condition can be that the accuracy reaches a threshold.

[0054] In one specific embodiment of this disclosure, a training sample set, a test sample set, and a validation sample set can be constructed first. For example, a certain amount of raw information on entry and exit road segments of intersections can be randomly selected from across the country; based on this raw information, road condition images containing entry and exit road segments are generated; the above two steps are repeated three times, and the resulting three sets of images are used as the training sample set, the test sample set, and the validation sample set, respectively. The number of samples in each sample set of the training sample set, the test sample set, and the validation sample set can be set as needed. Sample labels can be obtained through annotation.

[0055] After obtaining the training sample set, test sample set, and test sample set, the residual network image algorithm can be used to continuously adjust the parameters (hyperparameters) based on the training sample set and test sample set, and train the model based on the adjusted parameters until the loss function converges; then, the model is validated on the validation sample dataset; repeat the above steps until the validation result (corresponding to the evaluation result mentioned above) meets the requirements of the residual network model (corresponding to the navigation action prediction model mentioned above).

[0056] The navigation action prediction model obtained based on this disclosure uses image analysis to predict navigation actions. Compared with other types of data, images contain almost all the features needed to determine navigation actions, which makes the prediction accuracy of the trained navigation action prediction model high, and the accuracy of the prediction results of the navigation action prediction model can be calculated.

[0057] The navigation action prediction model obtained based on the above method can be used to predict navigation actions from entering a road segment to exiting a road segment. For example, the navigation action prediction model can be applied to a navigation system (such as navigation software installed on a device). While generating route navigation information based on the starting and ending positions and providing navigation services to users, the navigation system can use the navigation action prediction model to generate navigation actions for the entering and exiting road segments at an intersection when the user is traveling or about to reach that intersection, guiding the user from the entering segment to the exit segment. As another example, the navigation system can also use the navigation action prediction model to pre-generate navigation actions for various travel routes at intersections within a predetermined geographical area. This allows the system to directly obtain pre-generated navigation actions based on the user's location when providing online navigation services, eliminating the need for online generation and thus improving the user's navigation experience.

[0058] The navigation action prediction model obtained based on the above method can also be used in combination with other navigation action generation methods to replace manual verification of navigation actions obtained by other navigation action generation methods, saving manpower costs and improving the reliability of verification results.

[0059] Figure 4 A schematic flowchart of a navigation action generation method according to an embodiment of the present disclosure is shown. Figure 4 The method shown can be executed by a navigation system that provides navigation services to users. The navigation system can be, but is not limited to, navigation software installed on devices such as mobile phones and vehicles.

[0060] See Figure 4 In step S410, prediction samples are obtained. The prediction samples are image data containing the entry and exit segments at the intersection. The prediction samples can be constructed using the training sample construction method described above, which will not be repeated here.

[0061] In step S420, the prediction sample is input into the pre-trained navigation action prediction model to obtain the prediction result output by the navigation action prediction model. The prediction result is used to characterize the navigation action from entering the road segment to exiting the road segment.

[0062] From the perspective of model structure, the navigation action prediction model can be any machine learning model suitable for processing image data, such as, but not limited to, deep learning models based on deep learning algorithms. Optionally, the navigation action prediction model can be a deep learning model based on the residual network algorithm, which has achieved great success in the image domain, i.e., a residual network model.

[0063] From the perspective of model prediction mechanism, navigation action prediction model can be regarded as a multi-classification model. The navigation action prediction model can process the input prediction samples to obtain the probability values ​​of different navigation actions, and output the navigation action with the highest probability value as the prediction result of the prediction sample, that is, the predicted navigation action from the entry segment to the exit segment.

[0064] Navigation action prediction models can be developed using the methods described above in this disclosure. Figure 1 The training method described above was used to train the equipment. For details of the training process, please refer to the relevant description above.

[0065] As an example, when a user is walking or about to approach an intersection, the navigation action prediction model disclosed herein can be used to generate navigation actions for the entry and exit segments of the intersection. After obtaining the navigation actions, corresponding prompts can be generated and output to remind the user of the navigation actions to follow when crossing the intersection. These prompts can be, but are not limited to, one or more forms of information such as text, voice, and images.

[0066] Figure 5 A schematic flowchart of a navigation action generation method according to another embodiment of the present disclosure is shown. Figure 5 The method shown can be executed by a navigation system that provides navigation services to users. The navigation system can be, but is not limited to, navigation software installed on devices such as mobile phones and vehicles.

[0067] This disclosure does not limit the execution order of steps S510 and S520. That is, step S510 can be executed first and then step S520, or step S520 can be executed first and then step S510, or steps S510 and S520 can be executed simultaneously without any order.

[0068] See Figure 5 In step S510, a first navigation action is generated based on the angle between the entry and exit road segments at the intersection.

[0069] The angle between the entry segment and the exit segment can refer to the angle between the road travel direction represented by the entry segment and the road travel direction represented by the exit segment.

[0070] Based on the magnitude of the angle between the entry and exit segments, as well as their relative orientation, a navigation action (i.e., the first navigation action) can be generated from the entry segment to the exit segment.

[0071] As an example, multiple angle ranges can be preset. The first navigation action is generated based on the angle range between the entry and exit road segments and with reference to the relative orientation between the entry and exit road segments.

[0072] For example, if the angle is within the range of 0 to 45°, a "go straight" navigation action can be generated; if the angle is within the range of 45° to 135° and the exit segment is on the right side of the entry segment, a "turn right" navigation action can be generated; if the angle is within the range of 135° to 180° and the exit segment is on the right side of the entry segment, a "U-turn on the right" navigation action can be generated; if the angle is within the range of 45° to 135° and the exit segment is on the left side of the entry segment, a "turn left" navigation action can be generated; and if the angle is within the range of 135° to 180° and the exit segment is on the left side of the entry segment, a "U-turn on the left" navigation action can be generated.

[0073] The above-mentioned method of generating navigation actions based on the included angle may generate incorrect navigation actions at the critical point of the angle range. Furthermore, due to the complexity of road conditions in the real world, the accuracy of the method of generating navigation actions based on angle calculation cannot be guaranteed.

[0074] Therefore, this embodiment proposes that a navigation action prediction model can be used to verify the navigation actions generated based on the above method, so as to discover and / or correct erroneous navigation actions.

[0075] Specifically, in step S520, image data containing the entry and exit segments are input into a pre-trained navigation action prediction model to obtain the second navigation action.

[0076] That is, image data containing the entry and exit points of the road can be used as prediction samples and input into a pre-trained navigation action prediction model to obtain the prediction result (i.e., the second navigation action) output by the navigation action prediction model.

[0077] In step S530, the first navigation action and the second navigation action are compared to see if they are consistent.

[0078] If the first navigation action is consistent with or substantially consistent with the second navigation action, it can be assumed that both the first navigation action generated by the angle-based navigation action generation scheme and the second navigation action generated by the navigation action prediction model are accurate. Therefore, if the first navigation action is consistent with or substantially consistent with the second navigation action, either the first navigation action or the second navigation action can be used as the navigation action from entering the road segment to exiting the road segment (step S540).

[0079] If the first navigation action and the second navigation action are inconsistent, it indicates that at least one of them is inaccurate (or even both are inaccurate). In this case, the entry and exit segments can be marked as suspected samples. The suspected samples can be handed over to relevant personnel for manual verification to determine the navigation actions from the entry segment to the corresponding exit segment represented by the suspected sample.

[0080] Therefore, the navigation action prediction model can be used to verify the navigation actions (i.e., the first navigation action) obtained by the angle-based navigation action generation scheme, thereby solving the problem of relying solely on roadside or user feedback to obtain incorrect navigation actions.

[0081] Taking the navigation action generation method of this disclosure embodiment executed by a navigation system providing navigation services as an example, when generating navigation actions from an entry segment to an exit segment at an intersection, the navigation system can use a navigation action generation method based on the included angle and a navigation action generation method based on a navigation action prediction model to obtain a first navigation action and a second navigation action. Then, the first and second navigation actions are compared. If they match, the navigation action is considered accurate and can be stored in a database in association with the entry and exit segments. If they do not match, the entry and exit segments can be marked as suspect samples. Suspect samples can be manually reviewed to confirm the navigation actions from the entry to the exit segment represented by the suspect sample. The navigation system can pre-generate navigation actions for various driving routes at intersections within a predetermined geographical area, allowing the system to directly obtain pre-generated navigation actions based on the user's location when providing online navigation services, without needing to generate them online, thereby improving the user's navigation experience.

[0082] The method for training a navigation action prediction model disclosed herein can be implemented as an apparatus for training a navigation action prediction model. Figure 6 A schematic diagram of an apparatus for training a navigation action prediction model according to an embodiment of the present disclosure is shown. The functional units of the apparatus for training the navigation action prediction model can be implemented by hardware, software, or a combination of hardware and software that implement the principles of the present disclosure. Those skilled in the art will understand that... Figure 6The described functional units can be combined or divided into sub-units to achieve the principles of the invention described above. Therefore, the description herein can support any possible combination, division, or further limitation of the functional units described herein.

[0083] The following is a brief description of the functional units that a device for training navigation action prediction models can have and the operations that each functional unit can perform. For details, please refer to the relevant descriptions above, which will not be repeated here.

[0084] See Figure 6 The apparatus 600 for training a navigation action prediction model may include a construction module 610 and a training module 620.

[0085] The construction module 610 is used to construct a training sample set, which includes one or more training samples. Each training sample is image data containing the entry and exit road segments at an intersection. The labels on the training samples are used to represent the navigation actions from the entry to the exit road segment. For details on how the training sample set is constructed, please refer to the relevant description above.

[0086] Training module 620 is used to train a navigation action prediction model based on a training sample set.

[0087] As an example, the construction module 610 can also construct a test sample set, as described above. The apparatus 600 for training the navigation action prediction model may further include an input module and an adjustment module. The input module is used to input test samples into the navigation action prediction model to obtain the first predicted navigation action of the test samples output by the navigation action prediction model; the adjustment module is used to adjust the navigation action prediction model based on the difference between the first predicted navigation action and the first actual navigation action. The training module 620 is used to train the adjusted navigation action prediction model based on the training sample set.

[0088] Optionally, the construction module 610 can also construct a verification sample set, as described above. The apparatus 600 for training the navigation action prediction model may further include an evaluation module. The input module is used to input the verification sample into the trained navigation action prediction model after training the adjusted model based on the training sample set, to obtain a second predicted navigation action of the verification sample output by the navigation action prediction model; the evaluation module is used to evaluate the navigation action prediction model based on the difference between the second predicted navigation action and the second actual navigation action.

[0089] Optionally, the apparatus 600 for training the navigation action prediction model may further include an iteration module for the instruction adjustment module, the training module 620, and the evaluation module to iteratively execute the steps of adjusting the navigation action prediction model based on the test sample set, training the adjusted navigation action prediction model based on the training sample set, and evaluating the navigation action prediction model based on the validation sample set, until the evaluation result meets the iteration termination condition.

[0090] The navigation action generation method disclosed herein can be implemented as a navigation action generation device.

[0091] Figure 7 A schematic diagram of a navigation action generation apparatus according to an embodiment of the present disclosure is shown. The functional units of the navigation action generation apparatus can be implemented by hardware, software, or a combination of hardware and software that implement the principles of the present disclosure. Those skilled in the art will understand that... Figure 7 The described functional units can be combined or divided into sub-units to achieve the principles of the invention described above. Therefore, the description herein can support any possible combination, division, or further limitation of the functional units described herein.

[0092] The following is a brief description of the functional units that the navigation action generation device can have and the operations that each functional unit can perform. For details involved, please refer to the relevant descriptions above, which will not be repeated here.

[0093] See Figure 7 The navigation action generation device 700 may include an acquisition module 710 and a prediction module 720. The acquisition module 710 acquires prediction samples, which are image data containing the entry and exit segments at the intersection. The prediction module 720 inputs the prediction samples into a pre-trained navigation action prediction model to obtain the prediction result output by the navigation action prediction model. The prediction result characterizes the navigation action from the entry segment to the exit segment. The navigation action prediction model may be obtained using the training method described above in this disclosure.

[0094] The navigation action generation device 700 may further include a generation module and an output module. The generation module is used to generate prompt information corresponding to the navigation action; the output module is used to output the prompt information.

[0095] Figure 8 A schematic diagram of a navigation action generation apparatus according to another embodiment of the present disclosure is shown. The functional units of the navigation action generation apparatus can be implemented by hardware, software, or a combination of hardware and software that implement the principles of the present disclosure. Those skilled in the art will understand that... Figure 7The described functional units can be combined or divided into sub-units to achieve the principles of the invention described above. Therefore, the description herein can support any possible combination, division, or further limitation of the functional units described herein.

[0096] The following is a brief description of the functional units that the navigation action generation device can have and the operations that each functional unit can perform. For details involved, please refer to the relevant descriptions above, which will not be repeated here.

[0097] See Figure 8 The navigation action generation device 800 may include a first generation module 810, a second generation module 820, a comparison module 830, and a determination module 840.

[0098] The first generation module 810 generates a first navigation action based on the angle between the entry and exit segments at the intersection; the second generation module 820 inputs image data containing the entry and exit segments into a pre-trained navigation action prediction model to obtain a second navigation action; the comparison module 830 compares whether the first and second navigation actions are consistent; and the determination module 840, if the first and second navigation actions are consistent or substantially consistent, determines either the first or second navigation action as the navigation action from the entry segment to the exit segment. The navigation action prediction model can be obtained using the training method described above in this disclosure.

[0099] The navigation action generation device 800 may also include a marking module for marking the entry and exit segments as suspected samples if the first navigation action is inconsistent with the second navigation action.

[0100] Figure 9 A schematic diagram of a computing device is shown, according to an embodiment of the present disclosure, which can be used to implement the above-described navigation action prediction model training method or navigation action generation method.

[0101] See Figure 9 The computing device 900 includes a memory 910 and a processor 920.

[0102] Processor 920 may be a multi-core processor or may contain multiple processors. In some embodiments, processor 920 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, processor 920 may be implemented using custom circuitry, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0103] Memory 910 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 920 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 910 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 910 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0104] The memory 910 stores executable code. When the executable code is processed by the processor 920, the processor 920 can execute the navigation action prediction model training method or navigation action generation method described above.

[0105] The above description, with reference to the accompanying drawings, details the navigation action prediction model training method, navigation action generation method, apparatus, and computing device according to this disclosure.

[0106] Furthermore, the method according to this disclosure can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing the steps defined in the above-described method of this disclosure.

[0107] Alternatively, this disclosure may be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) that, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the steps of the method described above according to this disclosure.

[0108] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0110] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for training a navigation action prediction model, comprising: Construct a training sample set, which includes one or more training samples. The training sample is image data containing the entry and exit segments at the intersection but not other segments at the intersection. The labels of the training samples are used to characterize the navigation action from the entry segment to the exit segment. A navigation action prediction model is trained based on the training sample set. The navigation action prediction model is used to predict navigation actions from entering the road segment to exiting the road segment at the intersection. The navigation action prediction model is a deep learning model and a multi-classification model. The navigation action prediction model processes the training samples to obtain probability values ​​corresponding to different navigation actions and outputs the navigation action with the highest probability value as the prediction result of the training samples.

2. The method according to claim 1, wherein, The navigation action prediction model is a deep learning model based on the residual network algorithm.

3. The method according to claim 1, wherein the step of constructing the training sample set includes: Road images containing entry and exit sections at intersections are extracted from map data, and these road images are added to the training sample set as training samples. And / or Image data obtained by capturing images of roads including entry and exit points at intersections will be added to the training sample set as training samples; and / or Based on the entry and exit information at the intersection, an image containing the entry and exit sections is generated, and this image is added to the training sample set as a training sample.

4. The method according to claim 1, further comprising: Construct a test sample set, which includes one or more test samples. The test sample is image data containing the entry and exit segments at the intersection. The label of the test sample is used to characterize the first real navigation action from the entry segment to the exit segment. The test sample is input into the navigation action prediction model to obtain the first predicted navigation action of the test sample output by the navigation action prediction model; The navigation action prediction model is adjusted based on the difference between the first predicted navigation action and the first actual navigation action; and The adjusted navigation action prediction model is trained based on the training sample set.

5. The method according to claim 4, further comprising: Construct a verification sample set, which includes one or more verification samples. The verification sample is image data containing the entry and exit segments at the intersection. The label of the verification sample is used to characterize the second real navigation action from the entry segment to the exit segment. After training the adjusted navigation action prediction model based on the training sample set, the verification sample is input into the trained navigation action prediction model to obtain the second predicted navigation action of the verification sample output by the navigation action prediction model. The navigation action prediction model is evaluated based on the difference between the second predicted navigation action and the second actual navigation action.

6. The method according to claim 5, further comprising: The process iteratively executes the steps of adjusting the navigation action prediction model based on the test sample set, training the adjusted navigation action prediction model based on the training sample set, and evaluating the navigation action prediction model based on the validation sample set, until the evaluation result meets the iteration termination condition.

7. A method for generating navigation actions, comprising: Obtain prediction samples, which are image data that include the entry and exit sections at the intersection but do not include other sections at the intersection; The predicted sample is input into a pre-trained navigation action prediction model to obtain the prediction result output by the navigation action prediction model. The prediction result is used to characterize the navigation action from the entry segment to the exit segment at the intersection. The navigation action prediction model is a deep learning model and a multi-classification model. The navigation action prediction model processes the predicted sample to obtain the probability value corresponding to different navigation actions and outputs the navigation action with the highest probability value as the prediction result of the predicted sample.

8. The method according to claim 7, wherein, The navigation action prediction model is obtained using the method described in any one of claims 1 to 6.

9. The method according to claim 7, further comprising: Generate prompt information corresponding to the navigation action; Output the aforementioned prompt message.

10. A method for generating navigation actions, comprising: The first navigation action is generated based on the angle between the entry and exit sections at the intersection. Image data containing the entry and exit road segments but excluding other road segments at the intersection is input into a pre-trained navigation action prediction model to obtain a second navigation action. The navigation action prediction model is used to predict the navigation action from the entry to the exit road segment at the intersection. The navigation action prediction model is a deep learning model and a multi-classification model. The navigation action prediction model processes the image data to obtain probability values ​​for different navigation actions and outputs the navigation action with the highest probability value as the second navigation action. Compare whether the first navigation action and the second navigation action are consistent; If the first navigation action is the same as or substantially the same as the second navigation action, then the first navigation action or the second navigation action shall be used as the navigation action from the entry segment to the exit segment.

11. The method of claim 10, further comprising: If the first navigation action is inconsistent with the second navigation action, the entry segment and the exit segment will be marked as suspected samples.

12. The method according to claim 10, wherein, The navigation action prediction model is obtained using the method described in any one of claims 1 to 7.

13. An apparatus for training a navigation action prediction model, comprising: A construction module is used to construct a training sample set, which includes one or more training samples. The training samples are image data that includes the entry and exit segments at the intersection but does not include other segments at the intersection. The labels of the training samples are used to characterize the navigation action from the entry segment to the exit segment. The training module is used to train a navigation action prediction model based on the training sample set. The navigation action prediction model is used to predict the navigation actions from entering the road segment to exiting the road segment at the intersection. The navigation action prediction model is a multi-classification model. The navigation action prediction model processes the training samples to obtain the probability values ​​of different navigation actions and outputs the navigation action with the highest probability value as the prediction result of the training samples.

14. A navigation action generation device, comprising: The acquisition module is used to acquire prediction samples, which are image data that include the entry and exit sections at the intersection but do not include other sections at the intersection. The prediction module is used to input the prediction sample into a pre-trained navigation action prediction model to obtain the prediction result output by the navigation action prediction model. The prediction result is used to characterize the navigation action from the entry segment to the exit segment at the intersection. The navigation action prediction model is a deep learning model and a multi-classification model. The navigation action prediction model processes the prediction sample to obtain the probability value corresponding to different navigation actions and outputs the navigation action with the highest probability value as the prediction result of the prediction sample.

15. A navigation action generation device, comprising: The first generation module is used to generate the first navigation action based on the angle between the entry segment and the exit segment at the intersection. The second generation module is used to input image data containing the inbound road segment and the outbound road segment but excluding other road segments at the intersection into a pre-trained navigation action prediction model to obtain a second navigation action. The comparison module is used to compare whether the first navigation action and the second navigation action are consistent; The determination module is used to determine the navigation action from the entry segment to the exit segment if the first navigation action is the same as or substantially the same as the second navigation action. The navigation action prediction model is used to predict the navigation action from the entry segment to the exit segment at the intersection. The navigation action prediction model is a deep learning model and a multi-classification model. The navigation action prediction model processes the image data to obtain the probability values ​​corresponding to different navigation actions and outputs the navigation action with the highest probability value as the second navigation action.

16. A computing device, comprising: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 12.

17. A non-transitory machine-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as claimed in any one of claims 1 to 12.

Images